Melanoma Skin Lesion Classification Using Neural Networks: A systematic review

Abstract


INTRODUCTION
One of the deadliest types of skin cancer is melanoma [1], It is becoming more common both in women and men Throughout the world [2] [3]. Ultraviolet exposure is the main cause of melanoma, according to [4] melanoma arises through some mutations that occur on melanocytes due to excessive exposure to these rays. Although it is one of the deadliest types of skin cancer, early detection of this disease, as shown by many studies, is an important stage as it leads to a cure rate of up to 90% of cases [5]. Currently, visual analysis by a specialist is one of the most standard methods of diagnosing melanoma. However, this method may take a long time. Furthermore, due to the complexity of the diagnostic procedure, this can lead to a misdiagnosis. Many aspects must be taken into account, the number of variables that must be examined (asymmetry ,shape, edge, texture, color, etc.), inexperienced specialists, and fatigue [6] [7] [8]. A dermatologist obtains and examines dermoscopy images in

Al-Rafidain Journal of Computer Sciences and Mathematics (RJCM)
www.csmj.mosuljournals.com various cases. in which case an examination accuracy of 84% (ACC) can be achieved [9] [10], considered insufficient. Therefore, to diagnose melanoma, it is necessary to use a computer-aided diagnosis (CAD) system [11].
Several attempts have been made by studies to construct a machine learning-based technique for detecting data automatically by putting their ideas together that provides a high accuracy with a fast result, even if the complex analysis of skin lesions images generates many problems [12] [13]. Because of the presence of artifacts, finding a suitable diagnostic algorithm is a rather difficult task, for example, the appearance of hair in a skin lesion, and the difference in the dimensions of the lesion, as well as its shape, color, vessels of blood, and other relics [14]. Those lesions are depicted in Figure 1. The researchers have greatly expanded their research due to the inconvenience caused by these pests, where preprocessing is used as the first step, segmentation, feature extraction, and then classification is used. Figure 2 shows the workflow for pre-processing.

Theoretical Background 2.1 Pre-Processing
The initial operations are applied as one of the preprocessing steps as follows: grayscale conversion or brightness correction, resizing, noise removal, data augmentation, binary conversion, Optimization of the contrast and density [15]. Given the wide variety of content of melanoma images, the segmentation step is a challenging task. Where the image can be divided into a group of pixels, which is part of the algorithm [16], and using an automatic or semi-automated process, the regions of interest (RoI) are extracted [17]. For data detection and segmentation, methods based on artificial neural networks (NNs) are used and are considered the most widely used techniques.

Feature Extraction and Selection
The feature extraction step is usually applied after the segmentation process as the dimensions of the data representation is reduced to make it more manageable. This leads to ease and speed in data processing, without losing any important information. Because it contains a large number of variables, it can be considered a large consumer of resources. Thus, the accuracy will be greatly increased if the feature is extracted perfectly. Most authors in the past [18] [19] [20] have used the ABCD (colour, differential structure, asymmetry, boundary) as a method for extracting features for melanoma detection, deep learning methods are currently being used by some to optimize extraction of features. classification step is the last and most discussed step in this research. The main objective of this phase is to give the image ROI a category. Given the difficulty of manual classification, In recent years, it has become increasingly necessary to design an accurate automated classification method. Currently, Artificial intelligence algorithms such as machine learning and deep learning have been utilized to get high results in segmentation, feature extraction, and classification. The main goal of using artificial intelligence is to clone human intelligence, with applications in multiple fields, including medical diagnosis. Optimal results have been obtained by applying artificial intelligence to detect melanoma. Whereas typical machine learning methods were first presented as a solution for automatic melanoma detection, a field of artificial intelligence has been proposed as a solution for automatic detection of melanoma. Melanoma, for the most part, relies on prior experience to enhance outcomes [21]. To create training data, the system first extracts the required features. Then, supervised learning is based on training a data sample from the data source with correct classification already assigned or unsupervised learning refers to the ability to learn and organize information without providing an error signal to evaluate the potential solution, And it is used in the learning process after obtaining this data [22]. In general, most researchers have used supervised learning for being accurate. It was also noted that the traditional methods based on machine learning have shown good results, but there are some limitations. The system in this case when it performs training, will need a large amount of data and therefore the learning process will take a long time, and machine learning is also subject to error. Hence, researchers turned to neural networks and deep learning techniques.

Neural Networks
A neural network is a collection of neurons that work together to promote the neural system in the body. These neurons are linked in a network like this, and every connection between these cells is assigned a weight, and therefore these weights help the neurons to give important outputs. Because of the benefits offered by neural networks, they are preferred by researchers, for example, distributed memory, as though a small amount of information it is possible to obtain good results, and also the ability to process data in parallel. When it comes to training, the system errors can be computed by subtracting the expected value from the target value. After obtaining the value of this error, it is possible to reduce the error by adjusting the weights of the system. To obtain high accuracy, the papers for the detection of melanoma were used, which were directed to the use of forward-directed abstraction and recurrent neural networks. By using models of convolutional neural networks or recurrent neural networks, the researchers obtained the best results. A convolutional neural network is a neural network that contains a minimum of one layer. Currently, many various applications can obtain the best results, including detection systems for melanoma. The study of new methods for detecting skin lesions automatically, particularly melanoma, is the main objective of this work. This paper focuses on the use of neural network techniques when we develop a system like this. The remainder of the paper is structured as follows: The second section presents, material, methods, stimulates and select recent and relevant research to establish new directions for detecting melanoma using neural networks. The third section deals with the dataset used in the selected research. The fourth section describes and analyzes the most important neural networks currently utilized to detect melanoma, such as classification and segmentation. In the fifth section, new directions for implementing the neural network for detecting melanoma are presented, considering individual neural networks decision fusion based on multiple neural network elements, neural networks and also other intelligent classifiers, as well as hybrid component. Lastly, in the sixth section, which is in discussion section, The study results are compared to those of other peer-reviewed articles. Many neural networks techniques were utilized to classify melanoma. Among them: AlexNet [23,24,25,26,27,12], the number of neurons for each layer was determined in Figure 3, GoogLeNet/Inception [28,29,30,27,5], Figure 4 depicts the elements of the GoogLeNet integrator, GoogLeNet has 22 layers in its simplified architecture as seen in Figure 5, The general architecture of the Inception v3 network as seen in Figure 6, VGG Networks [31,27,32,33], VGG16 as seen in Figure 7, VGG19 as seen in Figure 8, ResNet [34,35,30,32,36,5,37,38,39,40,29], Residual network as seen in in Figure 9, figure 10 Show the Basic architecture of the ResNet-152, YOLO Networks [41,42,43,44,45,46], the network has certain alternating 1 x 1 convolution filters as seen in Figure 11, Xception Network [37,30,47], the Xception architecture seen in Figure 12, MobileNet [48,30,32,49], EfficientNet [50,51,32], the EfficientNet network as seen in Figure 13, DenseNet [1,35,32,52], the DenseNet architecture as seen in figure 14, U-Net [53,54], this architecture is divided into 4 parts: encoder, bottleneck, decoder, and skip connections ( Figure 15), Generative Adversarial Network [55,56], the GAN is built up of two independent networks as seen in Figure 16.

Methodology
We analyzed nearly 150 articles and therefore selected 120 articles for this study. The most important criteria for choosing articles were their publication dates, new neural network-based data disclosure developments, the vision, in addition to the contribution's influence (publishing in important journals and conferences, as well as citations). About new trends through the use of neural networks in the detection, segmentation, and classification of melanoma, The following was observed: Convolutional neural networks with a single layer are often modified to detect melanoma, as for systems that use multiple convolutional neural networks, and systems that use convolutional neural networks alongside other classification systems. We discuss it in detail in Section Five. Despite the fact that the quantity of citations is small, it is generally higher than new research (2022) for old research. Thus, Due to various exclusions, we did not establish a restriction on the number of citations. Many papers related to developments during the preceding period (correlated to melanoma, data sets, neural networks, merged networks, and decision fusion) are explained in Sections 3 and 5. We carried out the analysis to compare each search. We then evaluated the performance. Accuracy, precision, sensitivity, specificity, F1 score, and Jaccard index are the most often used measurements for melanoma detecting, segmenting, and classification. table 1 shows these formulas, TP denotes true positive, TN denotes true negative, FP denotes false positive, and FN denotes false negative. The focus is on accuracy (ACC), F1 score (F1 -dice factor), and Jaccard index in this case (IoU -Intersection over Union). Table 1. Indicators of performance used in this study [58] Indicator Formula Accuracy Precision Dice Coefficient Sensitivity Specificity Jaccard index

Datasets
The algorithms in this research are artificial intelligencebased, which means can train from a variety of big and small sets of data. In collaboration with clinicians and medical professionals, data sets were built. The datasets consist of well-selected, high-quality images that have been analyzed earlier, categorized, and segmented by health professionals. The goal of this research is to show the growing trend of these automated systems that can diagnose problems, stratifying, or detecting skin lesions, especially melanoma Based on previous research study. Through this section, we will present some of the frequently used datasets that have been shown in much research on skin lesions. Among them, The datasets PH2, ISIC 2016, 2017, 2018, 2019, 2020, HAM10000, DermNet Atlas, Dermatology Atlas, DermIs, and MED-NODE are all available (Table 2). For such study, PH2 is the most extensively utilized dermoscopic dataset. The datasets are created in Portugal at Pedro Hispano Hospital, which is one of multiple medical institutions, as stated in [59]. There are roughly 200 dermoscopic images in this dataset (80 common nevi, 80 atypical, and 40 melanoma). They are 8-bit RGB color images with a resolution of 768560 pixels that were carefully chosen for quality, resolution, dermoscopic properties. For every image in the database has a manual analysis and clinical diagnosis of skin lesions, and also recognition of other significant dermoscopic criteria. ISIC (International Imaging Dermatology Collaboration), which provides datasets containing digital images of expertly annotated skin lesions from different editions (2016, 2017, 2018, 2019, and 2020) to assist in these Computer-aided diagnoses in diagnosing many diseases [60] [61], has also provided important and sustainable datasets in this subject. These datasets have been presented at international biomedical imaging symposiums (ISBI). The ISIC 2016 dataset [14] includes 900 JPEG skin lesion images from training data and 379 images from test data in the same format. Images in this collection range in resolution from 576 X 768 to 2848 X 4288, hinting that scaling may be necessary in some cases. ISIC 2017 [62] features a total of 2,750 skin lesions, with 2,150 skin lesions suitable for training and 600 skin lesions suitable for testing. The resolution of these images ranges from 540 x 722 to 4499 x 6748. Resizing may be required in some cases, as it was in the previous dataset. The cutaneous lesion was analyzed for a melanoma detection challenge [63] using the ISIC 2018 dataset challenge [62]. The data set is quite huge (10.1 Gb), Using a total of 2594 images and 12,970 ground-truth response masks (five per image) for training and 1000 images for testing (2.2 Gb). Skin lesions are RGB JPG images, but the masks are grayscale PNG images [64]. There were three challenging tasks in the ISIC 2018 Challenge. Participants used 2594 images in the first two tasks, while in the third task, which was a classification test, they used the HAM10000 dataset, a further noteworthy data collection that may be found in the ISIC archives for free. There are 10,015 images in HAM10000, 1,113 for skin cancer. To improve contrast and color accuracy, all images in dataset were meticulously cropped with the lesion centered to 800 600 px at 72DPI and hand histogram corrections were performed [65]. ISIC 2019 and ISIC 2020 are new ISIC dataset types [25] that include more images than previous versions. The MED-NODE dataset [66], which contains 70 melanoma and 100 nevus images from the digital image library of the department of dermatology at the University Medical Center Groningen, is another prominent dataset used in skin cancer detection systems. The University of Edinburgh Dermofit Image Library is an exclusive resource for medical imaging study. Melanoma (76), melanoma nevus/mol (331), seborrheic keratosis (257), basal cell carcinoma (239), and so on are among the 1,300 high-quality images of skin lesions grouped into ten groups [67]. Each image in the set was taken with a high-quality SLR camera with illumination. Expert opinion (dermatologists and dermatologists) is used to classify the images, and binary segmentation masks are used to separate the lesions themselves. Only a one-time license is required to access this data set. Another dataset utilized in skin lesion detection, segmentation, and classification studies is the DermNet Skin Disease Atlas. There are almost 22,000 images in this data collection (only 21,844 were judged to be significant) classified into 23 skin disease types (supercategories) [68]. JPEG images are RGB, with resolution varying from one image to another [69]. DermIS is an image of dermatological dataset of 300 melanoma images that is widely utilized in the papers for skin lesion detection, segmentation, and classification [70]. [71] You can look for dermoscopic images by type in this collection (face, hands, legs, etc.). Dermquest is a web-based medical atlas for dermatologists and dermatologist-based healthcare practitioners [72], was another prominent dataset for skin lesion identification, segmentation, and classification. The collection, which includes approximately 22,000 clinical images, was once open to the public (but is no longer available.  Table 2. It describes the characteristics of the database/dataset used in the references examined and shows the availability and mostly used datasets discussed in our survey. The ISIC, PH2, HAM10000, MED-NODE, DermIS, and Dermquest historical databases are free and publically available for skin lesion diagnosis study. When it comes to deep learning algorithms designed to learn diverse patterns, watermarks usually represent image noise. As a result, researchers who want to use high-resolution DermNet images without watermarks must get a permission One of the causes DermNet isn't widely used is because of this, even though is a huge dataset where high-quality images that are not watermarked may make a difference in deep learning processing. According to our findings, the ISIC archives have the most commonly used databases in skin lesions diagnosis study (also contains H10000). The first reason could be that these databaese are relatively static and are adequately classified by subject specialists, while the second one could be the annual difficulty in keeping stable prices. 2nd goes to PH2, a small database. According to our findings, the tendency is to employ small datasets for verification and large databases for learning systems like deep learning and transfer learning. As stated in Section 4, increasing data is routinely used. This section describes the design of the fundamental neural networks commonly used in these types of programs, because we're particularly curious on the use of neural networks in melanoma detection. The most commonly utilized neural networks in such applications are shown in Table 3.

A Single Convolutional Neural Network for Melanoma Classification
Melanoma and other skin lesion detection systems constructed with only one convolutional neural network are presented in several studies. The majority of them compared the findings of several convolutional neural networks before selecting the one that performed best. [27] presents a DLbased technique for skin lesion identification employing different distinct convolutional neural network designs such as AlexNet, GoogLeNet, VGG, and ResNet using the ISIC 2017 dataset. [32] shows one example of a neural network-based support system that can assist doctors in diagnosing the seven most common pigmented skin lesions.  [116] reported an automated melanoma detection and segmentation method that used a modified deep regional convolutional neural network to decrease the investigation area and a fuzzy C means technique for precise segmentation. ISIC 2016 dataset provided the dermoscopic images. [117] shows existing networks that have been tweaked to make them more accurate. A modified version of U-Net was proposed by the authors. This innovative structure improved the performance of U-Net in skin lesion segmentation by combining DenseNet and ResNet. Convolutional layers in the encoder are interlaced with densely connected context modules. These are leftover modules from a previous project. Localized modules are also intercalated into the decoder's up- FrNet FrNet [114] Mask R_CNN Mask R_CNN [115] sampling stages. The Dense Skip Connection is a novel skip connection between the encoder and decoder. 3D convolutional neural network represents a new tendency in creating such systems with precise outcomes in skin lesion identification. For example, the authors of [118] proposed the Hyper-net, a 3D fully convolutional neural network for more accurate melanoma segmentation from hyperspectral pathology pictures. Hyperspectral pictures are represented by 256 × 256 x 16 cubes as input for Hyper-net. The authors mix regular convolution with dilated convolution for multi-scale features. A fusion path connects the encoder and decoder blocks. The decoder generates a 3D cube of the same size as the input cube. V-net [119] inspired residual learning to improve training efficiency. Another novel avenue in the use of neural networks for melanoma detection is preprocessing tasks. [120], a recently released research that presented an encoder-decoder convolutional neural network for hair removal, is an example of such a network ( Figure 17).
The new melanoma detection method described in [38] is based on deep learning. The two primary modules of the system are (RoI detection using Mask R CNN and RoI classification using transfer learning for ResNet152, which was previously trained with ImageNet database).
In [1] two convolutional neural networks merged to provide a 95% accurate skin lesion categorization. As shown in Figure  18, an image containing a skin lesion passes through the first convolutional neural network (encoder-decoder type) designed for segmentation, and the segmented skin lesion is then used as an input in the next convolutional neural network composed of merged dense blocks, which is used for classification. The HAM10000 dataset was used for the studies.

Multiple Convolutional Neural Networks for Melanoma Classification
The capacity to detect and classify skin lesions can be boosted by integrating many networks into a complex system. This viewpoint reveals two concepts: first, use multiple networks for the same function and the fusion of individual results for the final decision, and second the use of multiple networks for the same function and the fusion of individual results for the final decision. For example, in [77], the authors proposed a multi-task deep learning system based on a Feature Pyramid Network (FPN), a Region Proposal Network (RPN), and three subnets for exact skin lesion (for classification, detection, and segmentation). The data from the FPN and RPN (which determine the RoI) are sent to the subnets, which run in parallel to produce a combined and more precise result for skin lesions analysis and prediction. To handle the skin lesion classes imbalance issue for the image dataset, the framework is built on the creation of a loss function based on focal loss (RPN loss function) and the Jaccard distance. The datasets for the ISIC 2016 and 2017 contests were used. A similar system concept is described in [35], a recent article that combined two modules: a prerequisite segmentation module and a classification module.  [5] suggested a melanoma detection method with the following novel features: (a) using multiple convolutional neural networks as individual classifiers, (b) using a hybrid structure combining four convolutional neural network-based classifiers with a texture featurebased classifier, and (c) using another convolutional neural network as a global classifier with the probabilities of individual classifiers as input (considered as weights). The convolutional neural networks used included a custom neural network, GoogLeNet, ResNet-101, NasNet-Large, and a Perceptron ( Figure 20).

Figure 20.
Melanoma detection based on decision fusion in a multi-network system architecture [5]. [27] proposes utilizing a convolutional neural network ensemble (GoogLeNet, AlexNet, ResNet, and VGGNet) to categorize skin lesions using convolutional neural network output interpretation, indicating that it is a viable technique. [79] proposes a multi-convolutional neural network and voting scheme-based complicated system for melanoma and skin lesion diagnostics. As illustrated in Figure 21, Each sub-module of the classification system votes on a single dermoscopic image and assigns it a value. After then, the greatest value is compared to a threshold. If it is less than the threshold, a large module (Vote) made up of other convolutional neural networks will make a group decision. As a result, a more accurate final classification decision is made. It is possible to move the global classifier away from subjective network classification decisions and toward a more objective decision, which is likewise represented by a neural network, with the proper employment of numerous neural networks [6]. In this specific study report, the scientists proposed a system based on a choice made by several neural networks, as shown in Figure 22. Six (neural network-based) classifiers are coupled to two operational levels in this system. Five subjective (individual) classifiers are found on the first level, whereas a Perceptron classifier is found on the second level, which determines whether the final judgment is melanoma or not. The learning-adjusted weights from the first level are used to make the final selection. During the learning phase, each subjective classifier is given a weight based on its classification accuracy. The probabilities offered are the results of these classifiers during the testing phase. The final classifier's convolution law is made up of the subjective classifiers' weights and probabilities. The final classifier is thought to be objective. Subjective classifiers include two ResNet 101 and AlexNet neural networks, two perceptrons with LBP histogram and HOG as inputs, and a GAN-based ABCDbased classifier for major segmentation. Figure 21. The group decision ensemble strategy [79].

Figure 22.
The suggested melanoma classification system [6] is built on a network of several neural networks coupled on two classification levels.

Discussion
The research discussed the most widely used neural network-based algorithms for detecting, classifying, and segmenting skin lesions, particularly melanoma. The focus was on emerging developments in these types of applications. To that goal, we looked at 134 sources, the majority of which were published between 2017 and 2021. Multiple deep convolutional neural networks are grouped together based on decision fusion or with other classifiers based on texture, shape, and color variables in the most performing systems for melanoma detection. We move away from subjective classifications specific to individual classifiers (neural network) and toward a more objective classification, the global classifier in this way. This classifier examines the results of the classifiers, but makes its own choice based on pre-determined criteria. Individual classifiers should be selected in such a way individual classification can be compensated for by the objective classifier faults. The pipeline kind of neural network, which is based on jobs, is another fascinating combination of neural networks; The first network, for example, handles primary processing, while the second handles segmentation and the third handles classification. Other networks were built using intermediary modules, or smaller networks, into the framework of an existing network. Both the suggested network solution and the data collection employed have an impact on the final results (additionally containing the chosen images). The vast majority of the publications examined came from Web of Science, which is widely regarded as the most reliable global citation database. The following criteria were utilized in the searches: (a) themes such as melanoma and neural networks, (b) new tendencies (articles published among 2017 and 2022), (c) citations number, (d) impact factors in journals, and (e) the rate at which conference papers are indexed by ISI.

Conclusions and Suggestions
Neural networks, as part of AI algorithms, are fast being researched as a support system for identifying skin lesions and detecting melanoma in imaging applications. Skin lesion classification databases and problems are always expanding. That's why there's a push to enhance these classifiers so that they can reliably detect and track the evolution of skin lesions even when they're far away. The best results were achieved by using many neural networks for various functions and combining decisions. We might argue that this area of interest and the technique in which problems are treated are targets of great interest in the integration of artificial intelligence in medicine, based on the rising use of neural networks in detecting melanoma. Use neural network for melanoma detection could be part of system of assistance for dermatologists who must make the final decision on whether to recommend a biopsy if at least one of the dermatologist's diagnoses and the support system (a helpful method) indicate melanoma or to investigate if another type of cancerous lesion exists. In the latter situation, the system can be trained to recognize distinct types of cancerous skin lesions. On the other hand, the system is incapable of making final decisions. Given neural networks' evolutionary patterns, updated, changed, and integrated networks are expected to increase the performance of such systems. In future's work, the application of these technologies to classification of melanoma that develops under the nails, which is now a more difficult condition to diagnose, is a potential topic to study. We are not aware of such a process and have searched the literature for it. If the nail is still translucent, an image enhancement technique can be used to tell the melanoma apart. If the melanoma has attacked the network, it must be learned with the nail.