# Smart Embedded System for Skin Cancer Classification
## 4 Results
### Single Network Model  
Several known CNN models can be utilized to classify skin cancer lesions. The most adequate for the problem at hand depends on the tradeoffs between the classification accuracy, complexity and facility to be quantized. Three variants of ResNet (ResNet18, ResNet50 and ResNet101) were considered for this purpose. The metrics used to compare the models were accuracy, precision, recall, and F1-score. These metrics are defined as follows:  
\[Accuracy=\frac{TP+TN}{TP+FP+FN+TN}\]  
\[Precision=\frac{TP}{TP+FP}\]  
\[Recall=\frac{TP}{TP+FN}\]  
\[F1\text{-}score=2\times\frac{Recall\times Precision}{Recall+Precision}\]  
where  
1. True Positive (_TP_): Correctly predicted positive values;
2. True Negative (_TN_): Correctly predicted negative values. Both predicted and actual values are negative;
3. False Positive (_FP_): The predicted value is positive but the correct value is negative;
4. False Negative (_FN_): The predicted value is negative but the correct value is positive. Accuracy is the most intuitive measure and is simply a ratio of correctly predicted observations to total observations. This metric is enough for symmetric datasets where values of false positives and false negatives are almost the same. The precision determines how much of the images identified as positive detections are correct. Recall determines how many positive cases were detected out of all of the positive cases. Finally, the F1-score is useful when there is an uneven class distribution. The four network models were trained for 50 epochs with the HAM10000 dataset and the results of the four metrics were determined (see results in Table 2). In the first experiment, all images used in the network were resized to \(224\times 224\). \begin{table}
\begin{tabular}{l c} \hline \hline
**Lesion Name** & **Lesion Abreviation** \\ \hline Actinic keratoses and intraepithelial carcinoma & akiec \\ basal cell carcinoma & bcc \\ benign keratosis-like lesions & bkl \\ dermatofibroma & df \\ melanoma & mel \\ melanocytic nevi & nv \\ vascular lesions & vasc \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Image classes of HAM10000 dataset. The table includes the number of parameters (\(Param\)) and the number of floating-point operations (FLOPS) of a single inference. As can be seen from the results, ResNet18 is close to the results of ResNet50, using only half of the parameters and less than half of the operations. Since the results obtained with the models are close to each other, smaller variants of ResNet18 model were considered. These are identified as ResNet18-X-Y-M:  
1. X: Indicates the repetition pattern of the basic block. Two repetition patterns are considered: (F) the original ResNet18 repetition pattern, [2; 2; 2; 2], and (S) a reduced repetition pattern, [1; 1; 1; 1];
2. Y: Indicates the number of filters. Three different variations are considered: the original (F), one with half of the filters in all layers (S) and another with a quarter of the filters in all layers (VS);
3. M: Indicates the size of the images. Three different resizes are considered: 224 \(\times\) 224 (F), 112 \(\times\) 112 (S) and 56 \(\times\) 56 (VS). All variants were trained and evaluated with the HAM10000 dataset. The dataset was divided into two parts: 80% of the images for training and 20% for evaluation. The models were trained for 50 epochs with a batch size of 32 using the Adam optimization algorithm and a cross entropy loss function. The learning rate was initially set to 0.001 and a cosine annealing learning rate adjustment strategy was used. The results of the four metrics were determined (see results in Table 3). \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Model** & **Param (\(\times\)10\({}^{6}\))** & **FLOPS (\(\times\)10\({}^{9}\))** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline ResNet18 & 11.18 & 1.83 & 0.88 & 0.87 & 0.88 & 0.86 \\ ResNet50 & 23.5 & 4.13 & 0.89 & 0.88 & 0.88 & 0.87 \\ ResNet101 & 44.5 & 7.61 & 0.90 & 0.90 & 0.89 & 0.88 \\ \hline \hline \end{tabular}
\end{table}