# Skin Lesion Segmentation by U-Net with Adaptive Skip Connection and Structural Awareness
## 3 Proposed Method
### Selective Kernel Module  
Skin lesions come in a wide range of sizes. In some cases, they occupy less than one percent of the total area of the image and, in other cases, the remaining background is so small that the algorithm could not find enough reference points to discern the two classes. Furthermore, the convolutional kernel of DenseNet [29] or other convolutional neural networks are built with a stable size (3 \(\times\) 3). This makes conventional deep learning networks unable to extract features from larger receptive fields. Nonetheless, to some extent, large receptive fields do not equal adequate outputs. The appropriate receptive field should reconcile the object localization with the contextual information extraction. The accuracy of localization decreases in relation to the excessive size of the receptive field, while the ability to capture the contextual information is confined to the undersized one. Based on that observation as well as the variation in size of the skin lesions, we opt for the SK module [18] which is exceptional at incorporating local and global information into the receptive field. This module is constructed with the ideas of an automatic selection operation and a lightweight design. The architecture is illustrated in Figure 3 and could be briefly described with three steps, in the following order: Split, Fuse and Select. At first, a group of two or three depthwise convolutional blocks (with Batch Normalization [30] and ReLU Activation [31]) with different dilation sizes are applied to the prior feature map \(\mathrm{X}\in\mathbb{R}^{\mathrm{C}\times\mathrm{W}\times\mathrm{H}}\). This can extend to multiple branches with bigger kernel sizes to cover a larger receptive field. In our case, the combination of kernel 3 \(\times\) 3 and kernel 5 \(\times\) 5 achieves the best result. Subsequently, in the Fuse process, an element-wise sum function assembles information from \(\mathrm{U}_{3}\) and \(\mathrm{U}_{5}\) (\(\mathrm{U}=\mathrm{U}_{3}+\mathrm{U}_{5}\)). To embed global spatial information into the channel, a global average pooling function \(\mathrm{F}_{\mathrm{gp}}\) is placed, which generates channel-wise statistics for \(\mathbf{s}\), followed by is a fully- connected layer \(\mathrm{F}_{\mathrm{fc}}\) which performs the guidance for the precise and adaptive selections. The feature map \(\mathrm{z}\) contains the encoded attention weights for features in different scales. At the Select stage, the channel selection weights are computed by a classification layer based on their correspondence with the split feature maps in the beginning. We choose sigmoid as the classification function on an exchange  
Figure 2: Architecture of the proposed model. with the softmax function in the original paper, since the task is binary classification in our method. The softmax function will be in charge when more than two data branches are generated in the Split stage. The learned parameters are applied to each feature map that is derived from the beginning, and all of them are then aggregated to be a new feature map. The new feature map is  
\[\hat{\mathbf{X}}=\hat{\mathbf{U}}_{3}+\hat{\mathbf{U}}_{5}=a\cdot\mathbf{U}_{3 }+b\cdot\mathbf{U}_{5}, \tag{1}\]  
where \(\hat{\mathbf{U}}_{3}\) and \(\hat{\mathbf{U}}_{5}\) are the selected feature maps of the kernel size 3 and 5, respectively. \(a\) and \(b\), in the following order, are the attention weight vectors for each feature map in which the sum of the elements with the same order from two vectors is 1. SK modules are embedded into the skip connections of U-Net, as shown in Figure 2. The corresponding level features from the encoder are adaptively selected before passing to decoder. In recent papers about object segmentation, researchers focused on the spatial attention and channel attention [32; 33; 34] to guide the learning process towards only relevant local representation, reducing the distraction from unnecessary features and improving the representation power.