# Improving Foraminifera Classification Using Convolutional Neural Networks with Ensemble Learning
## 4 Discussion  
Comparing the final results of the ensemble with the study reported in [3], the precision averaged across the six labelled classes achieved by six human experts ranged from 59% to 83%, with a mean of 74%. In contrast, five human novices ranged from 49% to 65%, with 56% on average. The ensemble approach proposed in [3] was found to achieve an average precision of 80%, which is comparable to that of the experts. The best ensemble presented in Table 1, which is percentile(1) + Gaussian(1) + luma scaling(1) + means reconstruction(1) + HSVPP(1) + GraySet(5), was found to achieve an average precision of 91%, a much better average than that of the experts and higher than any of them individually. Similar results have been obtained for recall and the \(F_{1}\) score, which was always averaged across the six labelled classes. The six experts achieved recall between 32% and 83% (mean 60%), while the novices scored 47-64% (mean 53%). The ensemble proposed in [3] (ResNet50 + Vgg16) reported an average recall of 82%, while our ensemble reached 90%. The \(F_{1}\) score of the six experts ranged from 39% to 83% (mean 63%), the five novices obtained \(F_{1}\) scores between 47% and 63% (mean 53%), and the [3] reported an average of 85%. The ensemble presented in this paper scored 90.6%. These results are illustrated in Table 2.  
On a finer level, we have also charted the precision, recall and \(F_{1}\) scores across all the different classes in our study, as seen in Table 3. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Precision (\%)** & **Recall (\%)** & \(F_{1}\) **Score (\%)** & **Accuracy (\%)** \\ \hline Novices (max) & 65 & 64 & 63 & 63 \\ \hline Experts (max) & 83 & 83 & 83 & 83 \\ \hline ResNet50 + Vgg16 [3] & 84 & 86 & 85 & 85 \\ \hline Vgg16 [3] & 80 & 82 & 81 & 81 \\ \hline Percentile(1) & 80.8 & 81.5 & 81.6 & 81.6 \\ \hline Percentile(10) & 85.1 & 85.8 & 85.2 & 85.3 \\ \hline GraySet(10) & 88.2 & 89.1 & 88.4 & 88.0 \\ \hline
**Proposed Ensemble** & **90.9** & **90.6** & **90.6** & **90.7** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Precision, recall, accuracy and \(F_{1}\) score comparison between [3] and the best ensemble presented in the paper. Figure 6: Confusion chart of the 4-fold cross validation results obtained by our ensemble. In both precision and \(F_{1}\) score, _N. incompta_ comes out the lowest by a significant margin, achieving 0.77 in precision, while the other six classes average 0.94; the \(F_{1}\) score for _N. incompta_ is 0.83, while the other six classes average 0.92. _N. pachyderna_ has the poorest recall, achieving 0.84, while the other six classes average 0.92. In contrast, the highest recall and \(F_{1}\) score were achieved on the _G. ruber_ class, alongside the third-highest precision, which may indicate it has features that are easier to distinguish than all the others. Out of the false positives for _G. ruber_, three were _G. sacculifer_, two were _G. bulloides_, and six were _Others_, while the false negatives were only misclassified as _G. sacculifer_ or as _Other_. The differences and similarities between the false positives and real data are illustrated in Figure 7.  
The highest precision was instead achieved on the _N. duttertrei_ class, along with the second-highest \(F_{1}\) score and third-highest recall. Only six patterns were misclassified as false positives, of which two were _G. sacculifer_, one was _N. incompta_, and one _N. pachyderna_, as well as two _Others_, shown in Figure 8.  
It is important note that all the performance indicators for our method achieved scores higher than 77%, while in [3] some classes like _N. duttertrei_ had precision less than 70%. The main disadvantage of the proposed ensemble is the increase in computational time; the preprocessing approaches are not computationally expensive, the main computational requirement is the inference with the set of CNNs. In any case, with a Titan RTX 24 GB, we can classify a batch of one thousand images in 1.945 s.