## 3 Materials and Methods
### Wound Alteration Classification Model
#### 3.3.4 Feature and Model Selection  
After the extraction of a high dimensional feature vector, the number of input variables needs to be reduced in order to decrease the computational cost and improve the model's performance. To select the optimal features, the principal component analysis (PCA)technique was employed to reduce dataset dimensionality by removing the predictors with low variance.  
A nested cross-validation technique was applied to optimize the model's hyperparameters along with model selection and evaluation to overcome the optimistically biased evaluation with only cross-validation. A three-time repeated five-fold stratified cross-validation was applied as the outer loop in which the final model is evaluated. The hyperparameters search was conducted on the inner loop by four-fold stratified cross-validation, returning the optimized pipeline with the ideal number of feature components.  
The choice of the number of folds relied on the dataset division of 60% training, 20% validation, and 20% test, where a stratified k-fold was applied instead of standard k-fold cross-validation due to the problem of class imbalance. This stratification divides the dataset into folds while preserving the percentage of samples for each class.  
A total of seven ML models were considered for model selection, such as SVM, LDA, KNN, NB, RF, DT, and logistic regression. Each of these ML algorithms has changeable hyperparameters, which can modify the algorithm's behavior towards the specific dataset and final performance.  
The optimization was performed with a grid search for the three classification models and the full dataset model, along with the PCA for feature reduction.  
The hyperparameter search was performed to optimize two scoring parameters, F1 and F2, due to the imbalance class problem since accuracy gives us a biased evaluation of the model. These two metrics evaluate the system by combining precision and recall. While F1 balances the two metrics, F2 adds more weight to recall, minimizing FN. The ideal system would have high F1 and F2 scores while balancing out both precision and recall, but in practice, it is more important to have fewer FN because overlooking an alteration would be harmful to patients.  
The training phase occurred for multiple experiments, such as evaluating the oversampling techniques and with a single classifier for all wound types. As such, for every model type, 2 score optimizations were made for 3 experiments: with no oversampling, with oversampling, and with a combination of oversampling and undersampling, giving a total of 18 training steps with an additional 3 for the single classifier. For each grid search, the final pipeline model with the best hyperparameters and the optimal feature number used was obtained.  
Finally, the model evaluation is obtained by averaging the chosen metrics along the outer loop test folds. Other metrics, such as accuracy, precision, recall, F1, and F2, were used to prevent a biased interpretation of the model's performance. The metrics are expressed in Equations (3-(8). Moreover, the receiver operating characteristics (ROC) curve ROC were used for evaluating the performance of the machine learning classifier. The graphical plot of the ROC curve includes a true-positive rate (also called recall) y-axis and a false-positive rate x-axis (Equation (6)). This performance can be quantitatively evaluated using the area under the ROC curve (AUC) [37].  
\[Accuracy=\frac{TP+TN}{FP+FN+TP+TN} \tag{4}\]  
\[Recall=\frac{TP}{FN+TP} \tag{5}\]  
\[False\,Positive\,Rate=\frac{FP}{FP+TN} \tag{6}\]  
\[F_{1}=2\frac{precision\cdot recall}{precision+recall}=\frac{TP}{TP+\frac{1}{2}( FP+FN)} \tag{7}\]  
\[F_{2}=\frac{(1+2^{2})(precision\cdot recall)}{2^{2}\cdot precision+recall} \tag{8}\]