# Gaussian Mixture Modeling for Detecting Integrity Attacks in Smart Grids
## 3 Probabilistic Modeling
### Gaussian Mixture Modeling of LTI Coefficients  
The various normal system states are represented by Gaussian mixture models, an important characteristic of which is their ability to approximate with high accuracy every possible distribution as long as a sufficient amount of data is available [23]. A weighted sum of Gaussian functions is fit to the available data while the associated parameters are learned using the Expectation-Maximization (EM) algorithm.  
A GMM composed of \(M\) Gaussian components is given by the next equation: \(p(x|\lambda)=\sum\limits_{i=1}^{M}w_{i}g(x|\mu_{i},\sigma_{i})\), where \(x\) is a continuous-valued data vector, \(w_{i}\) the weight of the \(i\)-th component (\(\sum\limits_{i=1}^{M}w_{i}=1\)), and \(g\) denotes the Gaussian component. In our case, \(x\) represents the coefficients of linear time-invariant models approximating the relationship between the datastreams of the power grid nodes.  
The LTI model is described by the general _discrete-time linear Multiple-Input Single-Output_ structure [24]:  
\[A(z)X_{i}(t)=\sum\limits_{j=1}^{m}\frac{B(z)}{F(z)}X_{j}(t)+\frac{C(z)}{D(z)}d( t), \tag{1}\]  
where \(d(t)\) is an independent and identically distributed random variable accounting for the noise, \(m\) is the number of inputs, \(z\) is the time-shift operator, while \(A(z),B(z),C(z),D(z)\) and \(F(z)\) represent z-transfer functions, whose parameter vectors are \(\theta_{A},\theta_{B},\theta_{C},\theta_{D}\) and \(\theta_{F}\), respectively. Consequently an element \(f_{\theta}\) in the approximating model family \(\mathcal{M}(\subseteq)\) is fully described with a \(\theta\in\mathbb{R}^{p}\), which comprises the above parameter vectors, i.e., \(x=\theta\). Following the logic of [25], we create an ensemble of dynamic models (e.g., Autoregressive with Exogenous Input, Autoregressive Moving Average with Exogenous Input, Output Error, etc.) with various orders and select the one which best fits the datastreams (i.e., lowest reconstruction error) while low-order models are preferred. The model search algorithm minimizes a robustified quadratic prediction error criterion [26]. It should be mentioned that the methodology is independent of the model type selection and can be applied using another type without any modifications.  
Moving on, it should be mentioned that each density has the following form:  
\[g(x|\mu_{i},\sigma_{i})=\frac{1}{(2\pi)^{D/2}}\exp^{-1/2(x-\mu_{i})\sigma_{i}^ {-1}(x-\mu_{i})}, \tag{2}\]  
where \(D\) denotes the size of \(x\), \(\mu_{i}\) the mean vector and \(\sigma_{i}\) the covariance matrix.  
A GMM may be completely defined by the mean vectors \(\mu\), the covariance matrices \(\sigma\) and the mixture weights \(w\) with respect to every component density. These parameters are typically assembled as \(\lambda=\{\mu_{i},\sigma_{i},w_{i}\},i=1\ldots M\) while the covariance matrices can belong to a variety of forms, e.g., full, constrained, etc. This choice depends on the requirements of the application and here we employ Gaussian functions of diagonal covariance because they can potentially be equally effective to the full ones [27; 28] at a much lower computational cost due to their simplicity. Moreover, by combining Gaussians with diagonal covariance bases, one is able to capture the correlations existing between the elements of the feature vector.