# Instance Segmentation of Microscopic Foraminifera
## 3 Results
### Evaluating the Model Performance  
After the 10 training runs had concluded, we evaluated each model on the test data using their respective parameters from the final training epoch. Note that all precision and recall evaluations presented from this point onward are based on a maximum of 256 detections per image, which is an increase from the maximum of 100 detections per image used during training due to lower computational costs and faster evaluation. The mean AP across all of the 10 run was evaluated as 0.78 for the detection task, and 0.80 for the segmentation task. Table 2 shows a summary of the AP and AR metrics for both tasks, where each result is the mean and standard deviation of all training runs. Note that this table shows results averaged over all four classes, as well as with the \"sediment\" class omitted from each respective evaluation, which will be discussed later. The precision-recall curves computed by averaging over all 10 training runs can be seen in Figure 7. Note the sharp and sudden drop in the curve around the recall threshold of 0.75, for both tasks. \begin{table}
\begin{tabular}{l c c c c} \hline \hline  & \multicolumn{2}{c}{**All Classes**} & \multicolumn{2}{c}{**Sans “Sediment” Class**} \\ \cline{2-5}  & **Bound. Box** & **Segm. Mask** & **Bound. Box** & **Segm. Mask** \\ \hline AP\({}_{50}\) & 0.90 & 0.90 & 0.94 & 0.95 \\ AP\({}_{75}\) & 0.88 & 0.90 & 0.94 & 0.94 \\ AP & 0.78 & 0.80 & 0.84 & 0.86 \\ \hline AR & 0.83 & 0.84 & 0.89 & 0.90 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: AP and AR scores for different IoU thresholds, evaluated with all object classes being considered and with the “sediment” class excluded. Figure 6: Evolution of the AP for both the detection and segmentation task, for each of the 10 runs, per epoch of training. Note that these results are based on evaluations using a maximum of 100 detections per image. (**Left**): The AP for the detection (bounding box) task. A plateau is reached after about 30 epochs. (**Right**): The AP for the segmentation mask task. We observe that the same type of plateau is reached here as with the detection task. In order to investigate the sharp drop in precision and recall, we computed per-class precision and recall; the results can be seen in Figure 8. From the curves in the figure, it is clear that the model is finding the \"sediment\" class particularly challenging. Notice how the precision rapidly goes towards zero slightly after the recall threshold of 0.75. We also wanted to determine how well the model performed at different IoU thresholds, so precision and recall were evaluated for the IoU thresholds \(\{0.5,0.75,0.85,0.95\}\). Figure 9 shows the precision-recall curves for all object classes, and Figure 10 shows the sans \"sediment\" class curves. From these results, it is clear that the model performs quite well at IoU thresholds up to and including 0.85, but, at 0.95, the model does not perform well. Based on the per-class and per-IoU results, it became evident that some test images containing only \"sediment\" class objects were particularly challenging. This can in part be explained by the object density in these images, with multiple objects sometimes overlapping or casting shadows on each other. In the COCO context, these types of object clusters are referred to as a \"crowd\", and they receive special treatment during evaluation. Importantly, none of the objects in our dataset have been annotated as being part of a \"crowd\" due to the resources required to annotate more than 7000 objects based on their proximity to other objects with sufficient precision and recall. Some examples of these dense object clusters can be seen in Figures A2 and A3. By removing the \"sediment\" class from the evaluation, the AP score for the bounding box increased to 0.84, and for instance segmentation it increased to 0.86. Recall also increased significantly, which means that more target objects were correctly detected and segmented.