# Improving Foraminifera Classification Using Convolutional Neural Networks with Ensemble Learning
## 3 Results  
The testing protocol for the dataset is the 4-fold cross-validation, and the performance metric is the F-score (\(F_{1}\)), defined as:  
\[F_{1}=\frac{T_{p}}{T_{p}+\frac{1}{2}\ \left(F_{p}+F_{n}\right)}\]  
where \(T_{p}\), \(F_{p}\), \(F_{n}\) are, respectively, the total number of True positives and the False positive/negative predictions made by the model.  
In Table 1, we compare our ensemble, coupled with different network topologies, with that proposed in [3]. With \(X(y)\), we report the performance of \(y\) CNNs trained with the \(X\) RGB approach for colorizing the images.  
In the column ResNet50_DA we couple the ResNet50 with a data augmentation approach. The tested data augmentation takes a given image, and randomly reflects it top-bottom and left-right for two new images. The third transform linearly scales the original image along both axes with two factors randomly extracted from the uniform distribution [1,2].  
The conclusions that can be obtained from the results reported in Table 2 are the following:  
* The best-performing ensemble produces results that significantly improve those obtained by the method presented in [3] (percentile), whose \(F_{1}\) score was reported as 85%;
* Among stand-alone approaches the best performance is obtained by GraySet;
* It appears that, in general, increasing the diversity of the ensemble yields better results. The approaches combining multiple pre-processed images sets consistently rank higher in \(F_{1}\) scores than any individual method, iterated ten times. Combining fewer iterations of all the approaches yielded the best results overall. Similar conclusions are obtained with the different topologies;
* The ensemble based on ResNet50_DA obtains performance similar to the one based on ResNet50, but clearly data augmentation improves the stand-alone approaches.  
The breakdown of the 4-fold classification on the full dataset is presented in Figure 6 as a confusion chart, where we can see the number of true positives, false positives, and false negatives. The _N. Incompta_ species was classified correctly the least which, alongside the high number of false positives predicted, is consistent with the classification rate in [3], although it found the false negatives to be the larger issue.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**4-Fold Cross-Validation** & & & & & \\ \hline ensemble [3] & & & 0.850 & & & \\ \hline Vgg16 [3] & & & 0.810 & & & \\ \hline  & ResNet50 & ResNet18 & GoogleNet & MobileNetV2 & ResNet50\_DA \\ \hline Percentile(1) & 0.811 & 0.803 & 0.785 & 0.817 & 0.870 \\ \hline Percentile(10) & 0.853 & 0.860 & 0.807 & 0.874 & — \\ \hline Luma Scaling(10) & 0.870 & 0.845 & 0.794 & 0.856 & — \\ \hline Means Reconstruction(10) & 0.874 & 0.859 & 0.810 & 0.879 & — \\ \hline Gaussian(10) & 0.873 & 0.850 & 0.805 & 0.867 & — \\ \hline HSVPP(10) & 0.843 & 0.833 & 0.789 & 0.841 & — \\ \hline GraySet(10) & 0.885 & 0.864 & 0.831 & 0.892 & — \\ \hline Percentile(3) + Luma Scaling(3) + Means Reconstruction(3) & 0.877 & 0.868 & 0.821 & 0.881 & 0.895 \\ \hline Gaussian(3) + Luma Scaling(3) + Means Reconstruction(3) & 0.879 & 0.859 & 0.798 & 0.872 & 0.889 \\ \hline Percentile(2) + Gaussian(2) + Means Reconstruction(2) + Means Reconstruction(2) + Means Reconstruction(3) & 0.885 & **0.880** & 0.840 & 0.888 & 0.903 \\ \hline Percentile(1) + Gaussian(1) + Luma Scaling(1) + Means Reconstruction(1) + HSVPP(1) + GraySet(5) & **0.906** & 0.865 & **0.841** & **0.897** & **0.906** \\ \hline \hline \end{tabular}
\end{table}
Table 1: \(F_{1}\) scores across the specified training cycles.