# Entropy-Based Anomaly Detection for Gaussian Mixture Modeling
## 3 Results
### Three-Component Gaussian Mixture with Uniform Random Noise  
In this second synthetic data example, a bivariate dataset is simulated from a three-component Gaussian mixture with equal mixing weights, different covariance matrices across components, and several noisy data points added from a uniform distribution over a square. The resulting dataset is shown in Figure 4a. The optimal GMM selected by BIC is a four-component Gaussian mixture with covariance matrices having varying volume, shape, and orientation (VVV,4; see Figure 4c). Figure 4b shows the scatterplot with data points marked according to the GMM classification and the ellipses corresponding to the estimated cluster covariances. A component with a large covariance is also necessary in this scenario to account for the noise present in the data.  
Figure 5a contains the graph employed for the identification of the observations to be used in the initialization of the noise component, while the scatterplot in Figure 5b shows their distribution in the features space.  
The optimal GMM with an additional component to account for anomalies is the three-component Gaussian mixture with covariance matrices having equal shape but varying volume and orientation (VEV,3; see Figure 6a). Figure 6b shows the scatterplot with data points marked according to the GMM classification (including the noise component). As a result, most of the simulated outliers are identified by the final model. Finally, a comparison of the entropy contribution from the model without and the model with a noise component can be seen in Figure 6c. From this graph it can be seen that the contribution to the overall entropy is mostly reduced for the observations assigned to the noise component.  
Finally, we note that for this dataset the anomaly detection procedure has sensitivity equal to 0.84 and specificity 0.99. Thus, the procedure appears to be quite good at distinguishing normal data from anomalous data without mistakenly flagging normal data as anomalous.  
Figure 3: BIC traces for the selection of the optimal GMM with the noise component on the first synthetic data example (**a**). Scatterplot of data points marked by colors and symbols according to the classification (identified anomalies are represented with the + symbol) from the estimated GMM with the noise component, ellipse corresponding to estimated cluster covariance, dashed lines referring to principal axes, and \(*\) to the cluster means (**b**). Plot of entropy contribution values \(h_{i}\) for the model without and with the noise component (**c**).  
Figure 4: Scatterplot of second synthetic data example with data points generated from a three-component Gaussian mixture with equal mixing weights and different covariance matrices across components (represented with the \(\bullet\) symbol), and with several noise data points added from a uniform distribution over a square (represented with the \(\circ\) symbol) (**a**). Estimated GMM with points marked by colors and symbols according to the GMM classification, ellipses corresponding to estimated cluster covariances, dashed lines referring to principal axes, and \(*\) to the cluster means (**b**). BIC traces for the selection of the optimal GMM (**c**).  
Figure 5: Entropy contribution of each data point in the second synthetic dataset, with dashed horizontal line from uniform noise for reference (**a**). Scatterplot of data points marked according to the initial classification of noise and with size proportional to the entropy contribution (**b**).