# Entropy-Based Anomaly Detection for Gaussian Mixture Modeling
## 3 Results
### Wisconsin Diagnostic Breast Cancer Data  
This dataset, available at the UCI Machine Learning Repository [32], contains measurements for 569 patients on 30 features of the cell nuclei obtained from a digitized image of a fine needle aspirate (FNA) of a breast mass [PERSON] et al. [33]. Each patient's breast mass was later analyzed and classified as either malignant (212 cases) or benign (357 cases). In accordance with [PERSON] et al. [33] and [PERSON] [15], we rely on three features for clustering: extreme area, extreme smoothness, and mean texture. Furthermore, the following analysis does not assume knowledge of either the covariance eigen-decomposition or the number of mixture components. We start our analysis by fitting several GMMs with all the available eigen-decomposition of covariance matrices and with number of mixture components from 1 to 9. The \"optimal\" model is selected using ICL, as described in Section 2.2, with ICL traces shown in Figure 7. Based on this criterion, we selected the VVE model with 2 clusters. Table 1 provides a summary and clustering results for the estimated model. Figure 6: BIC traces for the selection of the optimal GMM with the noise component on the second synthetic data example (**a**). Scatterplot of data points marked by colors and symbols according to the classification (identified anomalies are represented with the + symbol) from the estimated GMM with the noise component, ellipse corresponding to estimated cluster covariance, dashed lines referring to principal axes, and \(*\) to the cluster means (**b**). Plot of entropy contribution values \(h_{i}\) for the model without and with the noise component (**c**). Gaussian finite mixture model fitted by EM algorithm  
Mclust VVE (ellipsoidal, equal orientation) model with 2 components:  
log-likelihood n df BIC ICL Entropy  
-4449.632 569 16 -9000.766 -9099.815 7.820091  
Clustering table:  
1 2
240 329  
Confusion matrix:  
Cluster  
Diagnosis 1 2  
B 40 317  
M 200 12  
Figure 8 shows the scatterplots between pairs of features with data points marked according to the estimated clusters from model (VVE,2) and corresponding ellipses representing estimated cluster covariances. There is clearly a strong overlap between the two groups, which roughly represent the two types of diagnoses. Note that some data points from one group are located around the majority of data points from the other group. This is the consequence of an inflated covariance matrix. One further characteristic that emerges from the plots in Figure 8 is the presence of many data points that are distant from the bulk of the data, particularly for the malignant cases. \begin{table}
\begin{tabular}{|c|} \hline Gaussian finite mixture model fitted by EM algorithm \\ \hline \multirow{2}{*}{Mclust VVE (ellipsoidal, equal orientation) model with 2 components:} \\  & \\  & \\  & \\  & \\  & \\ \end{tabular}
\end{table} 
Table 1: Summary and clustering results provided by mclust package for the model GMM(VVE,2) estimated on the Wisconsin diagnostic breast cancer data. Figure 7: ICL traces from fitting GMMs without the noise component to the Wisconsin diagnostic breast cancer data. Figure 9a shows the contribution of each case to the overall entropy, with the reference dashed line representing the entropy contribution from the uniform noise. Using the observations above the uniform-noise reference line for initialization of the noise component, and the default estimate of the volume of the data, GMMs with an additional component for the noise can be estimated. A summary of the \"optimal\" model according to ICL is reported in Table 2, while Figure 10 shows the ICL traces that lead to the selection of model GMM(EVI,2). Figure 8: Scatterplots of selected features for the Wisconsin diagnostic breast cancer data with data points marked according to the GMM(VVE,2) classification and ellipses representing estimated cluster covariances.