# Overview of Algorithms for Using Particle Morphology in Pre-Detonation Nuclear Forensics
## 2 CCR Results  
Results based on the 22 MAMA measurements [2] are given in Table 1 for nine ML options including decision trees, flexible discriminant analysis (FDA), mixture DA (MDA), linear DA (LDA), k-nearest neighbor, approximate Bayesian computation (ABC), learning vector quantization (LVQ), multivariate adaptive regression with splines (MARS), and support vector machines (SVM). When all three pathways are in the data, the CCRs (using 70% of the data to train and 30% of the data to test) for decision trees, MDA, LDA, FDA, k-nearest neighbor, LVQ, SVM, ABC, and MARS, are 0.83, 0.76, 0.74, 0.74, 0.81, 0.75, 0.76, 0.87, and 0.87, respectively. All analyses are done in R [12]. When only pathway 1 is in the data for each of ADU, AUC, SDU, and MDU, the CCRs (again using 70% of the data to train and 30% of the data to test) for decision trees, MDA, LDA, FDA, k-nearest neighbor, LVQ, SVM, ABC, and MARS, are all increased, to 0.93, 0.82, 0.81, 0.81, 0.85, 0.87, 0.83, 0.89, and 0.89, respectively. Precipitation conditions SDU and MDU are the most difficult to distinguish, so if SDU and MDU are merged into one group, reducing the number of groups from 4 to 3, then all methods have higher CCRs, some as large at 0.98. ML options 1-9 are described in Appendix B. In addition, LDA is discussed in Section 4, and ABC is discussed in Section 5 because the simulation framework used in ABC is helpful for including various sources of model uncertainty. It is not surprising that the CCR decreases when all three pathways are represented in the data. Figure 3 plots principal coordinate (PC, [13]) 7 versus PC 4 for particles from the ADU, AUC, SDU, and MDU. PCs are coordinates from which the matrix of pairwise distances in the 22-dimensional space can be closely approximated. In Figure 3, there is qualitative evidence of moderately-good group separation, particularly considering that only two PCs are used (up to 22 PCs are available). Figure 4 is the same as Figure 3, but is for PC1, PC2, and PC4, and illustrates that using three PCs leads to better group separation than using the two PCs in Figure 3. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline  & **Tree** & **MDA** & **LDA** & **FDA** & **knn** & **LVQ** & **SVM** & **ABC** & **MARS** \\ \hline
3 pathways & 0.83 & 0.76 & 0.74 & 0.74 & 0.81 & 0.75 & 0.76 & 0.87 & 0.87 \\
1 pathway & 0.93 & 0.82 & 0.81 & 0.81 & 0.85 & 0.87 & 0.83 & 0.89 & 0.89 \\ \hline \end{tabular}
\end{table} 
Table 1: CCRs for 9 ML methods for recognizing ADU, AUC, SDU, and MDU with 3 pathways present in the data, or with 1 pathway present in the data. The boldface entries are the largest entries in each row, and all entries are repeatable to within approximately \(\pm\)0.01. Decision trees, MARS, and ABC have the highest CCRs. Methods such as MDA have the potential to recognize clusters within the 4 precipitation conditions and thereby mitigate the decrease in classification accuracy [13; 14]. Figure 5 plots the Bayesian information criterion (BIC) used in model-based clustering versus the number of clusters (\"components\") in the ADU data with all three pathways (U3O8, UO2, UO3) present. There is modest evidence for 3-7 clusters within ADU. Figure 6a plots the case in which mclust is asked to find the best 3-cluster fit in 22 MAMA features in the ADU data (using MDS coordinates 7 and 3 as one good option to display the data in 2-dimensions). The three clusters are somewhat distinct, but the same types of plots for 4, 5, 6, or 7 clusters look equally plausible, and there are small sub-clusters in 2 of the 4 clusters. Figure 6b is the \"true\" clusters as defined by the three pathways. Note in comparing Figure 6a,b that some of U3O8 cases are clustered in the UO2 group. Figure 7 plots the raw MAMA feature 5 (perimeter of convex hull covering the particle) versus feature 1 (particle vector area) without transformation into PCs.