# Potential of Deep Learning Methods for Deep Level Particle Characterization in Crystallization
## 4 Results
### Mask-RCNN Applied to Three-Class Task  
The three-class task is addressed by use of the Mask-RCNN method. Since this is a complex semantic segmentation with high demands concerning accuracy, different methods are developed to achieve the best result possible. Figure 3 shows a comparison of the resulting COCO metrics of the different Mask-RCNN methods used and developed in this paper: direct application of Mask-RCNN (M3_1), combined use of two Mask-RCNNs, one detecting only ag and sc, and the other one detecting pc\({}_{\text{ag}}\) (M3_2), and the latter variant with additional post-processing (M3_2 PP). Table 2 evaluates the different methods regarding the target variables of agglomeration. The plain Mask-RCNN variant (M3_1) achieves detection accuracy of \(AP=49.2\) and \(AP_{50}=66.9\). Notable differences can be spotted in per-class AP, with the value of \(AP^{ag}=74.2\) for ag being significantly higher than both \(AP^{pc_{ag}}=38.5\) and \(AP^{sc}=34.8\) for sc and pc\({}_{\text{ag}}\), respectively. This indicates that size distribution across object classes correlates with detection and classification accuracy. Agglomerates are larger on average and score highest in detection accuracy, while the smallest particles are usually single crystals scoring at lower values. Primary crystals in agglomerates achieve low accuracy, which is most likely due to the higher complexity of detection. As primary crystals merge in agglomerates, they overlap or at least touch each other by forming the agglomerate leading to the challenge of identifying the contours even in the annotation phase. Often the details in the images  
Figure 3: Comparison between Mask-RCNN-based methods developed in this paper. are insufficiently pronounced, so the more detailed the annotations, the more subjective they are. To further complicate matters, on the one hand, the distinction between single crystals and primary crystals in agglomerates is tricky. Both, sc and pc\({}_{\text{ag}}\), belong to the primary crystals formed by nucleation. The only difference is that sc are free in solution while pc\({}_{\text{ag}}\) are bound in agglomerates and therefore touch or overlap with others. On the other hand, primary crystals in agglomerates are always part of the agglomerate, so there is a lot of overlap, ideally even by 100% making it difficult to determine the object borders. This results in a underestimation of \(\overline{pc/ag}\) whereas single crystals and agglomerates agrees well in terms of \(Ag\) (see Table 2). As a first attempt reducing these dependencies, the second variant (M3_2) uses a separate Mask-RCNN network specifically to detect pc\({}_{\text{ag}}\). This network is of course also trained separately on the relevant subset of the original training data. Evaluation of the second variant shows that the use of two separate Mask-RCNN networks gives an improvement over using only one network. Both, \(AP=51.6\) and \(AP_{50}=72.1\), show higher detection accuracy over the first variant, and per-class APs are increased as well, although not uniformly, with \(AP^{sc}=38.1\) and \(AP^{pc_{\text{ag}}}=39.1\) being increased more significantly than \(AP^{ag}=77.6\). Furthermore, \(\overline{pc/ag}\) of 2.91 approaches the ground truth of 3.08, since more pc\({}_{\text{ag}}\) are detected for M3_2 PP. Since the CNN learns on image features and does not recourse to theoretical knowledge, the results can be further improved by the use of post-processing. Analogously, the knowledge about pc\({}_{\text{ag}}\) forming agglomerates and consequently having to be part of an agglomerate are implemented separately. For that purpose the pairwise intersection over union (IoU) of all detected pc\({}_{\text{ag}}\) and ag is calculated and the sum of all IoUs of one primary crystal is calculated. All pc\({}_{\text{ag}}\) with a sum \(\geq 0\) are part of an ag and will be kept, the others are identified as false positives and are withdrawn.