# Machine Learning Models for the Classification of Histopathological Images of Colorectal Cancer
## 2 Materials and Methods
### Data Preprocessing and Model Fine-Tuning  
The algorithms that are presented in this study were implemented in the R programming language (version 4.2). The caret library was used to train and evaluate the random forest, SVM, conditional inference tree, naive Bayes, and k-NN algorithms. For these algorithms, cross-validation was used to determine the parameter values that yield the highest accuracy across all tissue types (optimal parameters). The doParallel library was used, which allowed for parallel CPU processing. The xgboost library was used to train and evaluate the XGBoost algorithm. For the training and evaluation of the CNN, the keras and tensorflow libraries were used. For feature extraction from the images, the OpenImageR (version 1.3), EBlImage (version 4.32), wvtool (version 1.0), and glcm (version 1.6.5) libraries were used. The ggplot2 and reshape2 libraries were used for data visualizations.  
Initially, 14 features were extracted from each image. In order to find the most relevant features, we implemented different classifying algorithms combining subsets of features, and we completed this procedure for all feature combinations, in order to detect a subset of optimal features. However, the model containing all features was associated with the highest classifying accuracy, and it was selected as the primary model for the subsequent analyses. For each algorithm, a 10-fold cross-validation procedure was applied five times to fine-tune the algorithm and provide a better estimate for accuracy. All the features were standardized before the algorithmic procedure. For the random forest algorithm, the optimal parameter for the parameter _mtry_, which denotes the number of features that are considered at each iteration, was equal to eight. For the SVM algorithm, the tuning parameters were sigma = 0.124 and C = 1. For the k-NN algorithm, the optimal value for the number of neighbors was set to k = 144. For the XGBoost algorithm, the default parameters of the xgboost library were used. The algorithm was executed for 100 iterations.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & \multicolumn{5}{c}{**True Classes**} \\ \hline  & \(C_{1}\) & \(C_{2}\) &... & \(C_{k}\) \\ \hline \multirow{3}{*}{Predicted classes} & \(C_{1}\) & \(C_{11}\) & \(C_{12}\) &... & \(C_{1k}\) \\  & \(C_{2}\) & \(C_{21}\) & \(C_{22}\) &... & \(C_{2k}\) \\ \cline{1-1}  & \(\cdots\) & & & & \\ \(C_{k}\) & \(C_{k1}\) & \(C_{k2}\) &... & \(C_{kk}\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Multi-class confusion matrix representation with k classes.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Metrics** & **Formula** \\ \hline Accuracy & \(\frac{\sum_{i=1}^{k}TP(C_{i})}{\sum_{i=1}^{k}\sum_{i=1}^{k}C_{i}}\) \\ \hline Sensitivity(\(C_{i}\)) & \(\frac{TP(C_{i})}{TP(C)+FN(C)}\) \\ \hline Specificity(\(C_{i}\)) & \(\frac{TN(C_{i})}{TN(C_{i})+TP(C_{i})}\) \\ \hline Precision(\(C_{i}\)) & \(\frac{TP(C_{i})}{TP(C_{i})+TP(C_{i})}\) \\ \hline F1(\(C_{i}\)) & \(2\cdot\frac{Recall(C_{i})-Precision(C_{i})}{Recall(C_{i})+Precision(C_{i})}\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: Indicators—metric evaluations of the classifying algorithms.  
The objective function was the multiclass log loss. The minimum loss reduction gamma was set to 0. The maximum depth of a tree was set to 6. The parameter eta, which is the step size shrinkage used to prevent overfitting for the next iteration, was set to 0.3. The algorithmic procedures were conducted on a desktop configuration equipped with an AMD Ryzen 7 CPU, an NVIDIA RTX 2070 Super GPU, 32 GB of RAM, and a 1 TB SSD. Due to the use of a pre-trained CNN model, the computational workload remained comparable across algorithms. Execution times for all algorithms were below two hours.