# Overview of Algorithms for Using Particle Morphology in Pre-Detonation Nuclear Forensics
## 5 ABC
### Model Uncertainty Accommodated by ABC for Inverse Problems in Forensics
It should be noted that the calibration checks indicate that ABC is working: the actual coverages of 0.99, 0.95, and 0.90 are the same as the nominal coverages, and the estimated RMSE is 0.05, in agreement with the observed average RMSE, using threshold \(\varepsilon=0.01\). Although this is a simple model selection example, it illustrates how ABC can accommodate both model selection and parameter estimation effects. Alternatively, [25] used a smoothed bootstrap method to assess model selection effects, which were non-negligible even for the case of selecting among linear, quadratic, or higher order polynomials to fit a response, as done here. For ABC-based model selection effects in the classification context, recall that Figure 5 plots the BIC used in model-based clustering versus the number of clusters (\"components\") in the ADU data with all 3 pathways (U308, UO2, UO3) present. There is modest evidence for 3-7 clusters within ADU. Recall that Figure 6 is a two-dimensional plot of the identified clusters when the mclust function in R is asked to find 4 clusters. ABC can be implemented with a fixed number of clusters per group, or with the probability of 1, 2, 3, 4, 5, 6, or 7 clusters per group calculated from the BIC (with the relative model probability assumed to be well approximated by \(e^{-\frac{BIC}{2}}\)[14] and see below). In the case of distinguishing group 1 (ADU) from group 2 (AUC), the 4-cluster model has an ABC-based CCR estimate of 0.92 and the random-cluster model has an ABC-based CCR estimate of 0.88. The corresponding actual CCRs in simulated data are 0.93 and 0.88, in good agreement with prediction, and with the anticipated feature that the CCR is larger if the assumed forward model has a known number (4) of clusters in each group. Returning to ABC applied to the 9255 sets of 22 MAMA measurements on particles, recall that the CCR in Table 1 is 0.87, which was for the case of using MAMA measurements [1; 2; 3; 4; 5; 6; 7; 8; 9; 23; 24] and using 70% of the data to train and 30% to test. If leave-one-out cross validation (CV, [14]) is used, the CCR is again 0.87. If model-based clustering (mclust in R) is use to model the data, and 3 clusters are selected for each of ADU, AUC, SDU, and MDU, then the ABC-based CCR is 0.93 (unrealistically high CCR due to over-simplified forward model in ABC). If instead the BIC-based probability of 1, 2, 3, 4, 5, 6, or 7 clusters is used, so that model uncertainty is accommodated in the forward model used by ABC, then the ABC-based CCR is 0.88, only 0.01 larger than that observed in real data, and lower than the over-simplified 3-clusters-per-group forward model. The BIC-based probability of a candidate model M, such as one with M = 1-7 clusters, is approximated as \(P(M)=e^{-\frac{BIC}{2}}\), with  
\[BIC=-2\log(\text{likelihood of model})+p\ \log(n) \tag{7}\]  
where the model likelihood in Equation (7) depends on the assume parametric form and measures how well the model fits the data, \(p\) is the number of parameters, and \(n\) is the number of observations. The number of parameters \(p=2 kM+\frac{Mk(k-1)}{2}\) in the clustering example,  
Figure 19: The ABC-based posterior pdf for the factor \(x_{1}\) if model 1 is the true model and model 1 is always the chosen model, if model 2 is always the chosen model, and if BIC-based model selection chooses between model 1 and 2 in the ABC training data. for the central composite design with 20 runs. where k is the dimension of the data and there are k means, k variances, and k(k \(-1\))/2 covariances per cluster. There are other ways to approximate the probability of a candidate model, such as the integrated complete-data likelihood, but the point of this example is to illustrate how ABC can accommodate the model selection process, no matter what type of model selection [25] is used. One goal for ABC applications is that the data model used in ABC lead to essentially the same CCR as that from using the real data within the ABC framework, in which case the fitted data model is a convenient model-based summary of the data.