Segmentation of Skin Lesions by Superpixel
Classification with Graph-Context CNN
Ángel Vı́ctor Juanco Müller1,2 , João F.C. Mota2[0000−0001−7263−8255] , andCorné Hoogendoorn1[0000−0002−4914−9936]
211
Canon Medical Research Europe
Heriot Watt University aj112@hw.ac.uk

Introduction

Accurate delineation of skin lesions is prone to inter-expert variability [4], whichcalls for automatic segmentation methods. In this paper, we process superpixel
graphs with a novel Graph Convolution layer to segment skin lesions.

2Methods & DataFirst, the Simple Linear Iterative Clustering (SLIC) algorithm [2] divides aninput image into n superpixels of approximately 250 pixels each. We then forma Region Adjacency Graph (RAG) over these partitions, which is the input ofthe proposed network (figure 1). Input features are the output of feeding 50 × 50
image patches, centred at the superpixel centroids, to a convolution layer.
Positional Encoding. The distances from superpixels to the central one,normalised to the interval (0, 1), are element-wise added to the input features.
Graph-Context CNN (gc-CNN). We build on the Graph Convolution
e =D
b −1/2 · A·b D
b −1/2 guides the propagation of information.
Network [9], where A
b is the adjacency matrix with self loops and Db its degree matrix. At layer l,Al
we convolve a tensor X of n feature maps, each with c channels, height h andwidth w, with a learnable kernel KΘ . Fold and unfold operations are used toenable the propagation of features through the RAG. Putting it all together, thegc-CNN layer can be written as:e · (unfoldc×h×w→f (X l ∗ KΘ )) + bΘ ),X l+1 = foldf →c×h×w (A(1)

where ∗ denotes the 2-D convolution operation, and bΘ is a learnable bias vector.
Sequences of gc-CNN, the ReLU non-linearity and layer normalization (LN) [3]are combined with skip connections [7] in the feature extraction block.
Feature Extraction Block. This block (figure 1) comprises a residual blockand a transition block. The latter halves the spatial dimension, increases theb by re-utilizing the feature maps corresponding
channel dimension and updates Ato neighbouring nodes. The mean value of their element-wise product, mapped tothe range (−1, 1) with the Tanh function and weighted by a parameter β = 0.001,bare added to the entries of A.

2Ángel Vı́ctor Juanco Müller , João F.C. Mota, and Corné Hoogendoorn

MLP block. A Multi Layer Perceptron (MLP) with layers of 2048, 1024,512, 128, 32, 16 and 2 neurons, respectively, with ReLU activation and LN [3],predicts the lesion probability for the superpixels.

Fig. 1. Overall network, comprising k feature extraction blocks and a MLP. The featureextraction block is formed by a residual block and a transition block.

Training. We label a superpixel as lesion if more than a 60% of its pixels
have that value in the ground truth3 . The ADAM optimizer [8] minimizes theCross Entropy function for 100 epochs with learning rate λ = 5 · 10−3 and batch
size 1. We use Pytorch Geometric [5] on an 11GB NVIDIA GeForce GTX 1080
Ti.
Datasets. We train on the ISIC-2017 [6] training split; 6 crops are extracted
and flipped for each image. We run inference on two dermatoscopic datasets,(ISIC-2017 [6] and PH2 [10]), and a non-dermatoscopic dataset (DermQuest [1])which has less controlled illumination conditions.
Post-processing. We keep only the connected component closest to theimage centre, which reduces false positives caused by illumination gradients in
DermQuest. We also apply binary opening and hole filling operations, which isuseful on ISIC-2017 although it decreases performance on PH2 (see figure 2).

3Results & Conclusion

Results. Table 1 reports Sensitivity (SEN), Specificity (SPE), Accuracy (ACC)and F1 score (F1) for the different categories in the PH2 dataset, comparing
against the results reported in [11], where a colour distance criterium was used
to merge superpixels. Figure 2 shows the F1 histograms for all three datasets.
Conclusion. We explore a segmentation technique that combines superpixel
graphs with Deep Learning. Our results for melanoma lesions surpass those in[11], and our model generalises well to unseen data. Future work will seek toimprove performance and apply the method to other medical domains.
360% yields an acceptable F1 score, around 90%, compared to the original masks.

Title Suppressed Due to Excessive Length3

Table 1. Our results on PH2 , in terms of mean and standard deviation, after postprocessing, for models with various k, and those reported in [11]. NN stands for Normal
Nevi, AN for Atypical Nevi, M for Melanoma. The highest figures are in bold.
RefClassSEN
SPEACCF1
Patiño et al. [11] NN (80 images)
92.12
96.42
95.24±6.37
92.92
ours (k = 1)
97.27±5.66
91.01±6.08
92.88±3.50
82.46±9.46
ours (k = 2)
94.21±13.87 86.86±16.42 89.52±12.52 79.72±16.17
ours (k = 3)
97.39±7.84 86.29±15.54 89.23±12.28 79.23±16.36
Patiño et al. [11] AN (80 images)
92.25
93.54
93.14±8.41
91.12
ours (k = 1)
96.87±4.70
90.31±8.69
92.98±4.20
86.65±7.18
ours (k = 2)
96.39±7.06 85.76±15.96 89.78±10.77 83.54±12.16
ours (k = 3)
97.12±4.45 87.73±13.89
91.38±9.60 85.32±11.76
Patiño et al. [11] M (40 images)
86.45
68.70
75.19±22.16
77.79
ours (k = 1)
84.07±23.70 83.19 ±17.07 81.27±20.09 83.31±18.08
ours (k = 2)
89.26 ± 16.29 80.71 ±14.00 83.76±13.18 85.80±12.36
ours (k = 3)
86.32 ±20.54 86.18 ±10.39 84.04±17.61 86.15±14.76
Patiño et al. [11] All (200 images)
91.04
89.73
90.39±14.19
89.18
ours (k = 1)
94.47±12.69 89.16±10.60 90.60 ±10.70 84.31±11.19
ours (k = 2)
94.09±12.52 85.19±15.94 88.47±12.23 82.47±14.14
ours (k = 3)
95.07±11.67 86.85±14.00 89.05 ±12.90 83.05±14.69

Normalized
frequency

20.0%
10.0%
0.0%0%ISIC 2017 Test Split (600 images). Model with k = 3

Original F1=77.81%±19.62%
Post-processed F1=79.84%±19.27%

20%

40%

ID: ISIC_0012136 F1: 24.73%

Original
Post-processed
Ground Truth

Normalized
frequency

20.0%
10.0%
0.0%0%

ID: ISIC_0012086 F1: 70.15%

Normalized
frequency

10.0%
0.0%0%

80%

100%

ID: ISIC_0012092 F1: 91.07%

Original
Post-processed
Ground Truth

Original
Post-processed
Ground Truth

PH2 (200 images). Model with k = 1

Original F1=86.18%±10.81%
Post-processed F1=84.31%±11.19%

20%

40%

ID: IMD417 F1: 23.09%

ID: IMD009 F1: 77.24%

Original
Post-processed
Ground Truth20.0%

60%

60%

80%

100%

ID: IMD015 F1: 92.15%

Original
Post-processed
Ground Truth

Original
Post-processed
Ground Truth

DermQuest (137 images). Model with k = 3

Original F1=77.96%±21.90%
Post-processed F1=80.23%±16.16%

20%

40%

ID: D25 F1: 21.59%

Original
Post-processed
Ground Truth

ID: D17 F1: 67.98%

60%

Original
Post-processed
Ground Truth

80%

100%

ID: D11_2 F1: 92.65%

Original
Post-processed
Ground Truth

Fig. 2. F1 histograms and sample outputs for the ISIC-2017 test set (top), PH2 (middle) and the DermQuest (bottom).4Ángel Vı́ctor Juanco Müller , João F.C. Mota, and Corné Hoogendoorn

References
1. University of Waterloo VIPlab, https://uwaterloo.ca/vision-image-processinglab/research-demos/skin-cancer-detection
2. Achanta, R., Süsstrunk, A.S., Smith, K., Lucchi, A., Fua, P.S.: SLIC Superpixels
Compared to State-of-the-Art Superpixel Methods. IEEE Transactions on PatternAnalysis and Machine Intelligence 34(11), 2274–2282 (2012)
3. Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer Normalization. In: NIPS 2016. vol. 1050,p. 21 (2016), http://arxiv.org/abs/1607.06450
4. Bogo, F., Peruch, F., Fortina, A.B.., Peserico, E.: Where’s the Lesion?: Variabilityin Human and Automated Segmentation of Dermoscopy Images of Melanocytic
Skin Lesions. In: Dermoscopy image analysis, pp. 67–96 (2015)
5. Fey, M., Lenssen, J.E.: Fast graph representation learning with pytorch geometric.
In: ICLR 2019. pp. 1–9. No. 1 (2019)
6. Halpern, A., Helba, B., Gutman, D., Codella, N., Celebi, M.E.: Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposiumon Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC). arXiv: 1710.05006 https://arxiv.org/abs/1710.05006
7. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition.
In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
8. Kingma, D.P., Ba, J.L.: Adam: A method for stochastic optimization. In: ICLR
2015. pp. 1–15 (2015)
9. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutionalnetworks. In: ICLR. pp. 1–14 (2017)
10. Mendonça, T.F., Ferreira, P.M., Marçal, A., Catarina Barata, J., Joana Rocha, J.:
Ph2: A public database for the analysis of dermoscopic images. Dermoscopy imageanalysis. (2015)11. Patiño, D., Avendaño, J., Branch, J.W.: Automatic skin lesion segmentation on
dermoscopic images by the means of superpixel merging. In: MICCAI 2018. vol. 1,
pp. 728–736 (2018)