# A Reinforcement Learning Algorithm for Automated Detection of Skin Lesions
## 2 Related Work
[44] proposed pyramid-pooling networks with dilated residual networks to segment skin lesions. The combination of endpoint error loss results in negative log-likelihood sharp boundaries. [PERSON] et al. [16] suggested skin lesion segmentation as a mutual bootstrapping CNN method and classification, in which one job bootstraps the other. [PERSON] et al. [45] first suggested a fully convolutional network (FCN) with a skip architecture based on a standard classification network to segment an entire image swiftly. [PERSON] et al. [46] used the Leaky ReLU and FCN in the final model layers frameworks to separate the ischemic lesions. [PERSON] et al. [47] proposed a V-Net-based architecture to segment the medical images in 3D and 2D formats. Many interactive segmentation algorithms are developed, with the physicians assisting the whole segmentation task. [PERSON] et al. [36] proposed a CNN and FCN-based biomedical image (cell) segmentation architecture. The classification network, led by a coarse segmentation network trained explicitly for this purpose, is guided by the expected coarse mask. Simultaneously, class-specific localization maps are produced by classification activation mapping (CAM). Then the concatenation is done into a U-Net-like network to improve the coarse mask prediction. DEXTR (deep extreme cut) [34] showed that using extreme points (contours corner points) as CNN input may improve the nature images instance segmentation results [34]. On the other hand, reference [34] uses extreme point inputs, the quality of which define the segmentation's efficiency. According to research [48; 49], the auxiliary function of boundary/edge prediction helps in instance segmentation. A loss function based on the Dice index is proposed to enhance the segmentation network. In addition to FCN-based techniques, several DL-based image segmentation algorithms have been proposed, including polygon-RNN [50], DeepLab V3+ [51], and multi-task network cascades [52]. In recent years, novel approaches for various applications, such as area extraction [53], wound intensity correction [54], and automated lung nodule categorization, have been developed [55]. Although there are positive effects of the therapies discussed above, a few pieces of literature have examined how physicians compute the ROI in skin imaging. The RL helps in imitating the demarcation technique of a physician. RL is significantly progressing in many applications, by combining RL with DL, DQN [24], DDPG [25], proximal policy optimization (PPO) [56], and asynchronous advantage actor critic (AC) are examples of deep neural networks used in DRL techniques for agent training. DeepMind achieves human-level game-playing abilities using DRL [23]. Therefore, other researchers have begun to use DRL for a range of problems, such as recommendation systems [57], game simulators, the Internet of Things, and adaptive packet scheduling [58]. DRL methods show promise in image classification, landmark identification [59], object localization [60], visual navigation [61], large -scale 3D point clouds semantic parsing [62], and face recognition [63]. [PERSON] et al. [64] developed a system for the image segmentation of prostate images based on RL Q-learning [65], which helps in finding the best values suitable for the subcategory of images and enhances the extraction of the ROI from the image. In contrast, Q-learning is limited to a narrow set of states and actions. Several researchers have tried to use DQN for image segmentation in recent years, combining Q-learning with CNN. DeepOutline [66] is an end-to-end deep RL framework for semantic image segmentation that works similarly to a user sketching the outlines of objects in an image with a pen. This approach is also proposed as an MDP. SeedNet is a game-changing seed generation method for interactive segmentation [67]. In each of these methods, DQN is utilized for training an image segmentation agent. DQN, on the other hand, is unable to maintain continuous activity, demanding additional operations to address the issue.