# A Reinforcement Learning Algorithm for Automated Detection of Skin Lesions
###### Abstract
Several multi-scale information in CNN architectures [13; 14; 15] or multi-task learning frameworks [16; 17] are mentioned in the literature to address these problems. The fundamental idea behind the methods is to generate reliable predictions by using as much data as possible. On the other hand, these methods either incorporate detailed new training parameters or need additional labeling information, both of which are inapplicable in practice. However, in the methods using fully convolutional networks (FCNs) [18; 19], the FCN smoothens out the complicated structures while ignoring the minor details. In the stand-alone post-processing, step CRF is employed and is disconnected from the FCN. A downsampling sequence followed by a greater sampling rate rapidly results in a detailed loss and poor segmentation results. The network makes errors while dealing with oversized objects. Despite their large representative capacity, the multi-scale approach uses skip connections from the low-level encoder features that tend to utilize redundant data. Furthermore, the encoder feature's contextual information is insufficient at the start of the network when coupled with the proper high-level decoder feature map, resulting in poor pixel-wise recognition performance. A series of downsampling and a higher sampling rate leads to loss of details and rough lesion segmentation results. When dealing with oversized objects, it assigns wrong labels and ignores them. Figure 2 presents the segmentation masks generated from our reinforcement learning (RL) algorithm on the PH2 dataset. As can be seen in the figure, the masks are clear. RL is prominent in artificial intelligence applications, such as anomaly detection, robot control, computer vision, autonomous driving, and computer gaming. DL and RL are used to solve increasingly complex problems resulting from a breakthrough in deep learning (DL). Deep reinforcement learning (DRL) trains an intelligent agent to deal with the Markov decision process (MDP) issue that combines DL with RL GrabCut [20] is a conventional interactive lesion segmentation method that divides the foreground and background regions using a coarse bounding box. [PERSON] et al. [21] proposed a CNN-based model that improved the lesion segmentation results by scribbling on the initial segmentation results. When physicians delineate the ROI on a lesion image, they first perform the coarse segmentation to determine the majority of the ROI's area. The clinicians then fine-tune the coarse segmentation procedure in the form of a multi-step process. This segmentation approach is similar to interactive segmentation [22]. Since it includes painting many strokes with a brush on the foreground and background or creating a box around the foreground, the interaction is considered prior knowledge for segmentation. This strategy gathers past information and improves the segmentation algorithm's efficacy by interacting with the user. Inspired by these concepts, the proposed method is proposed as a multi-step segmentation process. Our method has the advantage of improving the segmentation performance by automatically gathering the previous knowledge, thus eliminating the need for human involvement. The segmentation process is proposed as an MDP. In the next step, the segmentation mask predicted previously is taken as prior knowledge in the following step during the segmentation process. The agent executes an action for the segmentation process in every step that depends on the current segmentation mask and the input image. Our method derives its inspiration from the stroke-based stylization approach [23]. We propose a segmentation executor that helps draw a brushstroke for the input segmentation mask to designate the ROI. A neural network is used in the segmentation executor, which converts the continuous action parameters to brushstrokes. It can be used in different ways, and after some specific number of phases, the final segmentation results are achieved.