# A Reinforcement Learning Algorithm for Automated Detection of Skin Lesions
###### Abstract  
Skin cancers are increasing at an alarming rate, and detection in the early stages is essential for advanced treatment. The current segmentation methods have limited labeling ability to the ground truth images due to the numerous noisy expert annotations present in the datasets. The precise boundary segmentation is essential to correctly locate and diagnose the various skin lesions. In this work, the lesion segmentation method is proposed as a Markov decision process. It is solved by training an agent to segment the region using a deep reinforcement-learning algorithm. Our method is similar to the delineation of a region of interest by the physicians. The agent follows a set of serial actions for the region delineation, and the action space is defined as a set of continuous action parameters. The segmentation model learns in continuous action space using the deep deterministic policy gradient algorithm. The proposed method enables continuous improvement in performance as we proceed from coarse segmentation results to finer results. Finally, our proposed model is evaluated on the International Skin Imaging Collaboration (ISIC) 2017 image dataset, Human against Machine (HAM10000), and PH2 dataset. On the ISIC 2017 dataset, the algorithm achieves an accuracy of 96.33% for the naevus cases, 95.39% for the melanoma cases, and 94.27% for the seborrheic keratosis cases. The other metrics are evaluated on these datasets and rank higher when compared with the current state-of-the-art lesion segmentation algorithms. Keywords:image segmentation; skin lesion; deep deterministic policy gradient; multistep manner; skin cancer; melanoma; semantic segmentation; instance segmentation +
Footnote †: journal: Medical  
the detection algorithms employ \"contextual\" images gathered within the same patient to analyze whether the images represent a melanoma [5]. If the process is successful, the classifiers will be more precise and assist dermatological clinic work. The ABCDE recommendations [6; 7] provide physicians and patients with a clear and comprehensive framework for diagnosing melanoma, asymmetry, abnormalities along the border, skin discoloration, unbalanced color palette, lesions larger than 6 mm in diameter, evolved lesions (shape, color, or size), indicating the potential melanoma [8; 9]. The uneven melanoma borders are usually notched or scalloped, and uneven margins are defined vaguely. Consequently, the lesion segmentation process is often given the first performance preference to provide the regions of interest (ROI) or boundary information that has proven to aid the detection and classification tasks [10; 11]. Still, the issue of automatic lesion segmentation remains unsolved. In some skin lesions with light pigment, the color and the visual patterns of the pigment patches and the surrounding skin regions are highly similar, resulting in fuzzy and unclear boundaries, making the segmentation extremely difficult. Figure 1 shows the skin sample lesion images taken from the ISIC 2017 dataset [12], which have distinct class differences due to the various variations such as severe, blurred boundary, inhomogeneity, hair appearance, etc. Furthermore, due to the high quality of the original dermoscopic images, direct processing takes a long time and requires many resources. Therefore, to reduce the image's size, downsampling is utilized first. The subtlety and finer textures are lost during this process, making it much more difficult to distinguish the boundaries of the lesion. The lesions include color-makers, veins, hairs, glues, and rulers, all of which change the texture and color distribution of the lesions and impede efficient learning. Because the images are collected from various institutions and hospitals, they have a diverse set of characteristics. Hairs and color-makers are seen in specific images, complicating the process of segmenting lesions. The lesion segmentation is challenging due to the problems mentioned above.