## 3 Materials and Methods
### The Genetic Algorithm  
This paper uses a genetic algorithm that has been shown to effectively select the optimal hyperparameters to achieve skin lesion classification [34; 35; 36]. However, the application of the algorithm used is unique as shown below.  
Figure 4: The generated image sample uses the augmented technique.  
Figure 3: The original datasets without and with the augmentation technique.  
Hyperparameter Optimization Problem Formulation: A CNN model \(\mathcal{A}_{\lambda}\) can be defined as a function \(\mathcal{A}_{\lambda}:\mathcal{D}\longrightarrow\mathcal{M}\) where \(\mathcal{D}\) is the set of all datasets, \(\mathcal{M}\) presents the space of all models and \(\lambda\in\Lambda\) is the chosen hyperparameter configuration with \(\Lambda=\Lambda_{1}\times\cdots\times\Lambda_{p}\) being the p-dimensional hyperparameter space of CNN model \(\mathcal{A}\). The learning model estimates a model \(\mathcal{M}_{\lambda}\in\mathcal{M}\) that minimizes a loss function \(\mathcal{L}\) (e.g., misclassification rate) with its model regularization term \(\mathcal{R}\):  
\[\mathcal{A}_{\lambda}:=\arg\min_{\mathcal{M}_{\lambda}\in\mathcal{M}}\left( \mathcal{M}_{\lambda},\mathcal{D}^{\text{Train}}\right)+\mathcal{R}(\mathcal{ M}_{\lambda},\lambda) \tag{1}\]  
Then, hyperparameter optimization aims to find the optimal hyperparameter configuration \(\lambda_{\text{optimal}}\) using a validation set, i.e.,  
\[\lambda_{\text{optimal}}:=\arg\min_{\lambda\in\Lambda}\mathcal{L}\left(\geq \left(\mathcal{D}^{\text{Train}}\right),\mathcal{D}^{\text{Valid}}\right)=: \mathcal{F}_{\mathcal{D}}(\lambda) \tag{2}\]  
The \(\mathcal{F}\) is the miscalculation rate or loss function rate. The hyperparameter space includes all possible values that are usually defined as acceptable bounds for each hyperparameter and the number of the hyperparameter is the dimension of the function.  
Contribution: this basic structure of the genetic algorithm considers only one function called the loss function as the test point, which can be sufficient for some applications. However, in biomedical image applications, the success of a CAD system is associated with tuning multi-evaluation metrics and each metric has the potential to improve the performance of the CAD system. For example, choosing the Recall metric to tune the CNN model without considering other metrics could lead to misclassification if other metrics are kept static. The genetic algorithm used for this work considers six evaluation metrics to tune the CNN model to overcome these issues. Thus, this genetic algorithm focuses on finding the optimal hyperparameters that correspond to the highest evaluation metrics and Equation (2) becomes:  
\[\lambda_{\text{optimal}}:=\arg\max_{\lambda\in\Lambda}\mathcal{L}\left(\geq \left(\mathcal{D}^{\text{Train}}\right),\mathcal{D}^{\text{Valid}}_{i} \right)=:\mathcal{F}_{\mathcal{D}_{i}}(\lambda) \tag{3}\]  
where \(\mathcal{D}^{\text{Valid}}_{i}\) is a vector that contains five validation metrics, and \(i\) denotes each of these metrics (DICE, ACC, SN, SP, PREC and F-score). The hyperparameter response function \(\mathcal{F}_{D_{i}}\) is the misclassification rate for the corresponding \(i\) metric. Thus, the genetic algorithm selects the optimal hyperparameter that maximizes the six validation metrics [37; 38; 39]. In the context of a CNN model, a fitness function, based on six metrics, determines how well a particular set of model parameters performs on a skin lesion classification task. Unlike existing applications, the fitness function used here incorporates all the metrics. The fitness function is used to evaluate the fitness of different candidate solutions (i.e., sets of model parameters) and guide the selection of new solutions for further evolution. This modification of using multi-metrics was developed to ensure the optimized CNN model could classify the type of skin lesion with the lowest misclassification rate for various validation metrics. More details about the evaluation metrics are explained below.