# Machine Learning Subjective Opinions: An Application in Forensic Chemistry
## 4 Results
### Training Data Set Size Effects  
The results from varying the total number of samples in the training data set are shown in Tables 3 and 4. The total number of samples (first column) in each case consists of an equal number of samples containing ILR (class \(x\)), and samples containing no ILR (class \(\overline{x}\)). Data sets with equal representation from classes \(x\) and \(\overline{x}\) are balanced. The number of base models in each ensemble was 100. The results shown in the \"Bimodal\" column in Table 3 reflect the number of the 1117 validation samples that produced a set of posterior probabilities that were best fitted by a beta distribution with both \(\alpha\) and \(\beta\) less than 1, which corresponds to a bimodal distribution. In the LDA model, training data sets composed of 2000 randomly chosen samples were required to eliminate the bimodal fits. When the beta distribution fit parameters for a single validation sample both fell below a value of 1, the model was considered as showing significant uncertainty. In those cases, the values each fitting parameter were assigned a value of 1, which results in \(u_{x}^{ML}\) of 1 and each of \(b_{x}^{ML}\) and \(d_{x}^{ML}\) equal to 0. In contrast, even the minimum training data set sample size (50 \(x\) and 50 \(\overline{x}\)) did not produce bimodal posterior probability distributions from the ensemble of RF and SVM models. This is attributed to the greater complexity of these models. The RF models each contained 500 trees and optimized the mtry parameter. The SVM models produce a complex support vector with tuning parameter'sigma' held constant at a value of 0.1325 and tuning parameter 'C' was held constant at a value of 2. The \"Median \(u_{x}^{ML}\)\" column in Table 3 provides the median \(u_{x}^{ML}\) value obtained for the \(1117\) validation samples. The \"Max. \(u_{x}^{ML}\)\" column gives the maximum uncertainty from the validation opinions. The Wilcoxon rank sum test was applied to each successive sample size to determine if the median \(u_{x}^{ML}\) was statistically decreasing as the number of training samples increased. The Wilcoxon \(p\)-value is given in the last column of Table 3. The \(p\)-values were very small and indicate that the decreases in median \(u_{x}^{ML}\) were statistically significant. Shapiro-Wilk normality tests showed the \(u_{x}^{ML}\) values were not normally distributed in each case. The Wilcoxon rank sum test is non-parametric and does not rely on an assumption of normality. The LDA results show a maximum of total uncertainty, \(u_{x}^{LDA}=1\), for some validation samples predicted with models developed with training data sets of 1000 or less samples. The maximum uncertainty of the LDA model ensemble falls to a very low value of 5.60 \(\times\) 10\({}^{-2}\) at a training data set size of 60,000 samples. The training samples were bootstrapped to ensure that all 100 training sets were not equivalent. The very low uncertainty for the LDA base model is attributed to the in silico data creation method [7] which mixes IL into SUB at varying contributions. The covariances for \(x\) and \(\overline{x}\) are nearly identical, which is a requirement for the LDA model; however, the training data set size must be large enough to sufficiently represent the validation data. The low uncertainty of the LDA base model does not necessarily mean that the model discriminates well between \(x\) and \(\overline{x}\) classes.