# Multivariate Statistical Process Control Using Enhanced Bottleneck Neural Network
## 2 Materials and Methods  
### Multivariate Statistical Process Control  
#### 2.1.1 Gaussian Mixture Model  
The use of Gaussian mixture models for modelling data distribution is motivated by the interpretation that the Gaussian components represent some general underlying physical phenomena and the capability of Gaussian mixtures to arbitrary shaped densities model densities. This subsection describes the form of the Gaussian mixture model (GMM) and motivates its use as a representation of historical data for advanced statistical process monitoring (SPM). The use of Gaussian mixture density for process monitoring is motivated by two interpretations. First, the individual component Gaussians are interpreted to represent some operating modes or classes. These classes reflect some underlying phenomena having generated data; for example, hydrological or biological in water treatment plants. Second, a Gaussian mixture density is shown to provide a smooth approximation of the underlying long-term sample distribution of observations collected from industrial plants. The GMM parametrization and the maximum-likelihood parameter estimation are described. Multiple operating modes in normal process conditions can be characterized by different Gaussian components in GMM, and the prior probability of each Gaussian component represents the percentage of total operation when the process runs at the particular mode [19; 20]. The probability density function of Gaussian mixture model is equivalent to the weighted sum of the density functions of all Gaussian components as given below:  
\[p(x|\wedge)=\sum_{j=1}^{K}\beta_{j}g(\mathrm{x_{i}}|\lambda_{j}) \tag{1}\]  
where \(\mathrm{x}\in R^{l}\), \(\beta_{j}\) denotes the prior probability of the \(j\)th Gaussian component and \(g(x|\lambda_{j})\) is the multivariate Gaussian density function of the \(j\)th component. For each component, the model parameters to be estimated are \(\beta_{j}\) and \(\lambda_{j}=\left\{\mu_{j},\sigma_{j}\right\}\), the latter of which include the mean vector \(\mu_{j}\) and the covariance matrix \(\sigma_{j}\)([PERSON] et al., 2001). During the model learning, the following log-likelihood function is used as objective function to estimate the parameter values  
\[logL(x|\wedge)=\sum_{i=1}^{N}log\left(\sum_{j=1}^{K}\beta_{j}g(\mathrm{x_{i}}| \lambda_{j})\right) \tag{2}\]  
where \(\mathrm{x_{i}}\) is the \(i\)th training sample among the total \(N\) measurements.  
The Gaussian mixture model can be estimated by Expectation-Maximization algorithm through the following iterative procedure:  
* E-Step: compute the posterior probability of the \(i\)th training sample \(\mathrm{x_{i}}\) at the \(\mathrm{sth}\) iteration \[P^{(\mathrm{s})}(m_{j}|\mathrm{x_{i}})=\frac{\mathrm{\alpha}_{j}^{(\mathrm{s} )}g(\mathrm{x_{i}}|\mathrm{\alpha}_{j}^{(\mathrm{s})})}{\sum_{l=1}^{K}\beta_{ l}^{(\mathrm{s})}g(\mathrm{x_{i}}|\mathrm{\alpha}_{l}^{(\mathrm{s})})}\] (3) where \(m_{j}\) denotes the \(j\)th Gaussian component.
* M-Step: update the model parameters at the \((s+1)\)th iteration \[\mu_{j}^{(s+1)}=\frac{\sum_{i=1}^{N}P^{(\mathrm{s})}(m_{j}|\mathrm{x_{i}}) \mathrm{x_{i}}}{\sum_{i}^{N}P^{(\mathrm{s})}(m_{j}|\mathrm{x_{i}})}\] (4) \[\sigma_{j}^{(s+1)}=\frac{\sum_{i=1}^{N}P^{(\mathrm{s})}(m_{j}| \mathrm{x_{i}})(\mathrm{x_{i}}-\mu_{j}^{(s+1)})(\mathrm{x_{i}}-\mu_{j}^{(s\[\beta_{j}^{(s+1)}=\frac{\sum_{i=1}^{N}P^{(s)}(m_{j}|\mathrm{x_{i}})}{N} \tag{6}\]  
It should be noted that the number of Gaussian components corresponds to the number of operating modes in normal conditions [21]. The next phase after extracting the mixture operating modes consists to design the EBNN model.