# Machine Learning Subjective Opinions: An Application in Forensic Chemistry
## 1 Introduction  
In some applications (e.g., forensic science [1], medical image analysis [2], molecular design [3]) the user of ML technology may be well-served by a method that provides anopinion, rather than a single numerical answer. In some cases, the ML user is required to formulate an opinion, and in other cases, the overall outcome benefits from knowledge of a predicted sample's uncertainty, rather than assuming that the previously unseen sample is representative of the training data. A comprehensive survey of applications of evidential deep learning (EDL), a related methodology, is available [4]. The work presented here focuses on a forensic chemistry binary classification problem in the analysis of evidence of potential argon crimes and builds on previous work [1]. In this example, the forensic science expert is allowed to provide the court with an opinion regarding the outcome of the evidence analysis, as discussed in Section 2.2. The ML opinion can facilitate the formulation of an expert opinion, but should not replace the expert's role. This work builds on [PERSON]'s prior publication [1], while differing in several respects. The previous work relied on the use of laboratory-generated ground truth fire debris data for training on bootstrapped and class-balanced data (1000 samples per training data set) and a subset of the data (200 samples) were used for validation. [PERSON] et al. pretreated a set 129 features by principal components analysis following centering and variance scaling. The work presented here bootstraps training data from a reservoir of ground truth data computationally generated in silico. A set of 33 features with chemical significance to the problem was pretreated by scaling and removing the low variance and highly correlated features, leaving a set of 26 training features. The influence of training data size on the distribution of predicted class probabilities is tested in this work, but was not addressed by [PERSON]. The entire set of 1117 laboratory-generated data is used for validation in this report. Testing on casework or \"real\" fire debris is not included in this work or [PERSON]'s work, since it is not possible to know the ground truth of these samples. While the previous work only examined the use of support vector machine (SVM) learning, in this work, SVM results are compared with linear discriminant analysis (LDA) and random forest (RF). The term \"opinion\", in this text, will refer specifically to a \"subjective opinion\", which is composed of belief, disbelief and uncertainty masses, as well as a specified base rate [5]. The uncertainty mass can be interpreted as a degree of \"I don't know\". The belief, disbelief and uncertainty are required to sum to one, see Equation (2) below, which allows subjective opinions to be visualized on a ternary plot, as shown in Figure 1. Subjective opinions reflecting no uncertainty are \"dogmatic\" opinions, while dogmatic opinions expressing total belief or disbelief are \"absolute\" opinions. Formulating an ML subjective opinion can be accomplished by fitting predicted posterior probabilities from an ensemble of ML models to an appropriate probability distribution function. In the case of a binary opinion between two mutually exclusive states, the appropriate probability distribution is a beta distribution [5]. If the distribution of posterior probabilities for a sample is narrow the uncertainty is low, and as the width of the distribution increases, so does the uncertainty. This basic construct extends Dempster-Shaffer Evidence Theory [6]. The basis for the ensemble approach used here is briefly discussed in Section 2.5. There is an important distinction between an \"opinion\" and a \"decision\" in classification problems, as discussed in Section 2.5. When the uncertainty mass goes to zero in a subjective opinion, the opinion becomes either an \"absolute opinion\" or a \"dogmatic opinion\", see Figure 1. Nonetheless, the absolute or dogmatic opinion remains designated as a \"subjective opinion\".