# On the Influence of Data Imbalance on Supervised Gaussian Mixture Models
## 2 Materials and Methods
### Gaussian Mixtures for Classification  
Classification models based on Gaussian mixtures make the assumption that the density within each class follows a Gaussian mixture distribution:  
\[f(\mathbf{x}_{i}\mid C_{k})=\sum_{g=1}^{C_{k}}\pi_{g|k}\;\mathcal{N}(\mathbf{x}_{i} \mid\mathbf{\mu}_{g|k},\mathbf{\Sigma}_{g|k}), \tag{1}\]  
where \(G_{k}\) represents the number of components within class \(k\). The \(\pi_{g|k}\) terms denote the mixing probabilities for class \(k\) (\(\pi_{g|k}>0\) and \(\sum_{g=1}^{C_{k}}\pi_{g|k}=1\)), and \(\mathbf{\mu}_{g|k}\) along with \(\mathbf{\Sigma}_{g|k}\) stand for the mean vectors and covariance matrices for component \(g\) within class \(C_{k}\).  
One earlier proposal is the _Mixture Discriminant Analysis_ (MDA) model [10], where it is posited that the number of mixture components is known and equal across classes, with equal full covariance matrix across classes, so \(\mathbf{\Sigma}_{g|k}=\mathbf{\Sigma}\) for all \(g=1,\ldots,G_{k}\) and \(k=1,\ldots,K\) in Equation (1).  
A much more constrained model is the _Eigenvalue Decomposition Discriminant Analysis_ (EDDA) model [11], which assumes that the density for each class can be characterized by a single Gaussian component, so \(G_{k}=1\) for all \(k\) in Equation (1). Furthermore, in this model, the covariance structure of each class is factorized as:  
\[\mathbf{\Sigma}_{k}=\mathbf{\lambda}_{k}\mathbf{U}_{k}\mathbf{\Delta}_{k}\mathbf{U}_{k}^{\top}. \tag{2}\]  
A list of models obtained from the eigen-decomposition in (2), with associated geometric characteristics, is reported in Table 2.1 of [PERSON] et al. [12]. Through this decomposition, several classification models can be derived. When each component shares the same covariance matrix, i.e., \(\mathbf{\Sigma}_{k}=\mathbf{\lambda}\mathbf{U}\mathbf{\Delta}\mathbf{U}^{\top}\), the corresponding model is equivalent to the classical _Linear Discriminant Analysis_ (LDA) model. On the other hand, if the component covariance matrices are unrestricted and differ between components, i.e., \(\mathbf{\Sigma}_{k}=\mathbf{\lambda}_{k}\mathbf{U}_{k}\mathbf{\Delta}_{k}\mathbf{U}_{k}^{\top}\), EDDA is equivalent to the _Quadratic Discriminant Analysis_ (QDA) model. _Naive-Bayes_ models can also be obtained by assuming conditional independence of features within each class, so component covariance matrices are all diagonal, eventually with equal values along the main diagonal for the case of equal variances among features.  
The broadest model encompassed by Equation (1) framework is the _MclustDA_ model [2], which employs a finite mixture of Gaussian distributions within each class, allowing for flexibility in the number of components and covariance matrices, with the latter parameterized using the eigen-decomposition shown in Equation (2). Specifically, each class may exhibit a distinct Gaussian mixture, and the covariance matrix characteristics can vary across classes. This enables the model to capture a wide range of data distributions and provides a powerful tool for classification tasks that involve complex, heterogeneous datasets.