# Probabilistic Mixture Model-Based Spectral Unmixing
## 3 Methods
#### Data Generation and Inference  
Sampling under this model is performed once per mixture component \(k\) to determine endmember parameters (for a total of \(K\), the number of endmembers); per \(d\)th spectral dimension to scale the variance of observation noise in each dimension (for a total of \(D=K-1\), the number of spectral dimensions); and per \(i\)th observed data point. The choice of \(D=K-1\) follows from the desire for a direct comparison between the Bayesian hierarchical model and N-FINDR [6]. N-FINDR, in turn, specifies this value of \(D\) because its convex set geometry formulation establishes a correspondence between the spectral space and the rank \(K-1\) space of allowed mixing coefficients.  
Figure 1: Plate diagram for the Bayesian endmember mixture model. The top-level variables \(\vec{w}_{k},s_{d}\) and \(\vec{c}_{k}\) are given by Equation (1) and represent the prevalence of each mixture component, the extent of noise, and the endmember coordinates, respectively. ‘obs’, for ‘observation’, represents an observed mixture, of which \(N\) samples from the underlying distribution constitute one generated dataset.  
This coupling of \(D\) to \(K\) simplifies the approach but would, in a practical setting, require prior dimensionality reduction in the data. Second, and more significantly, we have assumed that \(K\) is known. The suitability of this assumption is likely domain-dependent. To take a particular example, in the context of phase identification in materials science, the assumption may be reasonable to the extent that \(K\) is constrained by considerations such as the Gibbs phase rule.  
In a bird's eye overview of our method, we generate simulated data of multiple points and then rely on two methods for inference of endmembers, Hamiltonian Monte Carlo (HMC) and variational inference (VI). Once a probabilistic model has been derived and established in the form of the model expression, one needs to infer the distributions of the parameters in the model expression. This inference is generally intractable for any realistic model due to the integrals involved in the Bayesian estimation [18]. To circumvent this intractability, alternative approaches are utilized. The gold standard in deriving parameter uncertainty is sampling-based Markov Chain Monte Carlo (MCMC) [19] methods, such as Metropolis Hastings. However, these have poor scaling and are computationally infeasible for high-dimensional problems. Recent developments of HMC and, specifically, the No U-Turn Sampler (NUTS) have transformed this status quo [20]. HMC avoids the random walk dynamics of classical MCMC algorithms by using the information in the gradient of the target distribution to inform sampling. NUTS regulates the frequency of sampling and together, HMC-NUTS enables sampling-based uncertainty estimation approaches to be extended to higher dimensions. However, in many cases with higher dimensions, HMC-NUTS may still not scale well and be time consuming. Thus, we also test inference using VI via the mean field approximation. Stochastic VI [21] reduces the sampling problem to an optimization exercise, wherein the divergence between the posterior distribution and a family of parameterized distributions is minimized. This enables the use of classical gradient-based optimizers to solve the problem rapidly and scale to even higher dimensions. However, in line with the no free lunch theorem, these advantages in scaling are accrued at the cost of being unbiased. Based on the assumptions made during VI to simplify the problem for computational and scaling expense, the final solution can be biased [19; 22]. This may lead to over-confidence and under-prediction of the uncertainty. To this end, we compare and contrast the solutions that we generate via HMC-NUTS and mean-field VI.