# Gaussian Mixture Modeling for Detecting Integrity Attacks in Smart Grids
## 3 Probabilistic Modeling
### k-Nearest Neighbor Clustering Using Kullback-Leibler Divergence  
This subsection details the way the LTI coefficients are represented in the stochastic plane for achieving clustering. Here, the aim is to assess the similarity, denoted as \(\mathcal{S}\), between the data under investigation and ones coming from the nominal state which were processed during the training of the system. This work proposes that \(\mathcal{S}\) can be estimated via Kullback-Leibler (KL) distances computed in the probabilistic space (see Figure 1). Thus, the normal state is decomposed into a group of GMMs rather than using only one universal model. More precisely, each data sequence is transformed to LTI parameters, the distribution of which is then estimated by means of a GMM, i.e.,  
\[(X_{i,T_{0}},X_{j,T_{0}})\to x_{i}\to p(x_{i}|\mu_{i},\sigma_{i},w_{i}), \tag{3}\]  
where \(X_{i}\) denotes the voltage datastream of the \(i\)-th bus up to time \(T_{0}\), \(x\) the respective sequence of LTI parameters and \(p\) its distribution in a Gaussian form. The next step of the proposed methodology is the computation of the pairwise distances between every GMM pair in the set. For the computation of each distance, a Monte Carlo approximation of the KL divergence is employed, the outcome of which is inversely analogous to the proximity among the involved distributions. For two distributions denoted as \(p(x_{i}|\mu_{i},\sigma_{i},w_{i})\) and \(p(x_{j}|\mu_{j},\sigma_{j},w_{j})\), the KL distance is defined as follows:  
\[KL(M||N)=\int p(x_{i}|\mu_{i},\sigma_{i},w_{i})\log\frac{p(x_{j}|\mu_{j},\sigma _{j},w_{j})}{p(x_{i}|\mu_{i},\sigma_{i},w_{i})}dF_{n}. \tag{4}\]  
Due to the absence of a closed-form solution, the above formula is approximated by the empirical mean [29]:  
\[KL(M||N)\approx\frac{1}{t}\sum_{k=1}^{t}\log\frac{p(p(x_{j}|\mu_{j},\sigma_{j},w_{j}))}{p(x_{i}|\mu_{i},\sigma_{i},w_{i})}. \tag{5}\]  
This metric represents the distance quite accurately given that \(t\) is sufficiently large. In the experiments included in this work, the number of Monte Carlo draws is \(t=2000\). An important detail here is the fact that \(KL\) divergence is used as a distance metric, and thus symmetricity is required. However, in general, the quantity defined above is not symmetric, i.e., the distance \(KL(M||N)\) may be different than \(KL(N||M)\). In order to satisfy this requirement, the following symmetrized form was inferred: \(KL(M||N)\to KL(N||M)\to KL(M||N)+KL(N||M)\). Thus, the distance matrix \(\mathcal{D}\) is derived with respect to every nominal system state as depicted in Figure 1. This scheme can be parallelized to a \(k-NN\) logic where the distance metric is the KL divergence. \(\mathcal{D}\) is summed column-wise and the maximum value comprises the threshold \(\mathcal{T}\) which discriminates between the normal vs. the rest of the data samples. Algorithm 1 demonstrates the process leading to integrity attack detection while a representative paradigm is depicted in Figure 2. The following section describes the experimental set-up and analyses the results. Figure 1: The process leading to the matrix which depicts the distribution of the LTI parameters of the dataset in the Kullback–Leibler sense. ```
1. Build the GMMs representing the nominal class, \(G_{N}=\{\mu_{N},\sigma_{N},w_{N}\}\) from the vectors of parameters \(\theta_{1}...\theta_{d}\) each of which associated with a linear dynamic model applied to the training data \(O_{i,T_{0},i=1,...,d}\) windowized using length \(M\) overlapping by \(M-1\);
2. Compute the pairwise distances with respect to all GMMs using the KL divergence \(KL(M||N)=\int p(x_{i}|\mu_{i},\sigma_{i},w_{i})\log\frac{p(x_{i}|\mu_{i}, \sigma_{i},w_{i})}{p(x_{i}|\mu_{i},\sigma_{i},w_{i})}dF_{n}\) and derive distance matrix \(\mathcal{D}\);
3. Sum \(\mathcal{D}\) column-wise and determine the threshold for nominal data as \(T_{h}=\max(sum(\mathcal{D}))\) ;
4. Windowize the incoming novel data as above, which results in windows \(W=W_{1}...W_{x}\); repeat 5. j = 1; 6.