# Multimode Process Monitoring Method Based on Multiblock Projection Nonnegative Matrix Factorization

Authors: Yan Wang; Yu-Bo Zhao; Chuang Li; Chuan-Qian Zhu; Shuai-shuai Han; Xiao-Guang Gu
Journal: Advances in Mathematical Physics
 (2020)

---

## Abstract

A multimode process monitoring method based on multiblock projection nonnegative matrix factorization (MPNMF) is proposed for traditional process monitoring methods which often adopt global model of data and ignore local information of data. Firstly, the training data set of each mode is partitioned by the complete link algorithm and the multivariate data space is divided into several subblocks. Then, the projection nonnegative matrix factorization (PNMF) algorithm is used to model each subspace of each mode separately. A joint probabilistic statistic index is defined to identify the running modes of the process data. Finally, the Bayesian information criterion (BIC) is used to synthesize the statistics of each subblock and construct a new statistic for process monitoring. The proposed process monitoring method is applied to the TE process to verify its effectiveness.


---

## Body

## 1. Introduction

With the rapid development of computer technology, the chemical process has become more automatic and intelligent. In recent years, accidents have occurred frequently in the chemical process, safety in the production process has become crucial, and process monitoring has received more and more attention [1]. Multivariate statistical process monitoring (MSPM) methods, such as principal component analysis (PCA) and partial least squares (PLS), are widely used in industrial processes due to their ability to extract effective characteristic information from process data and conduct process monitoring [2–4].Traditional MSPM methods need to assume that the measurement data come from a single stable mode, while modern chemical enterprises have a variety of operating modes due to changes in product characteristics, set values, or raw material components. Therefore, it is assumed that the MSPM method under a single working condition cannot meet the requirements of multimode monitoring. This not only weakens the statistical characteristics of the process in different modes but also leads to inaccurate process performance analysis. Xu et al. proposed a PCA mixture model based on scale-incremental EM algorithm to realize multimode process monitoring [5]. Feital et al. proposed a multimodal modeling and monitoring method based on maximum likelihood principal component analysis and carried out modal identification [6]. Jiang et al. defined a joint probability statistical indictor to identify the running mode of the process [7]. Zhang et al. proposed a fully automatic offline pattern recognition method based on sliding window and k-means clustering [8]. Xiong et al. proposed a sliding window and differential algorithm to realize multicondition identification of industrial processes [9].In addition to multimode processes, the modern chemical industry also contains a lot of non-Gaussian data. Therefore, the MSPM method assuming Gaussian conditions cannot meet monitoring requirements. Nonnegative matrix decomposition (NMF) is a new dimension reduction method. The advantage of this method is that the requirements for measurement data do not exceed nonnegative requirements so it is wider [10]. Yuan et al. proposed a projection nonnegative matrix factorization (PNMF) algorithm. The PNMF algorithm adds constraints to the NMF algorithm, reduces an unknown variable, and increases the sparsity of the matrix [11]. Ahmed et al. proposed an approximately optimal nonnegative matrix factor two-dimensional deconvolution (NMF2D) algorithm for separating underdetermined convolution integrals [12]. After that, they proposed a mono audio separation method based on multicomponent nonnegative matrix factor two-dimensional deconvolution (NMF2D) [13]. Woo et al. proposed an optimized complex nonnegative tensor factor two-dimensional deconvolution (CNTF2D) to separate sound sources mixed in an uncertain reverberation environment [14]. In recent decades, NMF-based methods have been used more and more widely. Gao et al. proposed a variational Bayesian subgroup adaptive sparse component extraction algorithm for thermal NDT & E imaging diagnosis [15]. They conducted experimental tests on artificial and natural defects of the robot and verified the effectiveness of the proposed method. Ahmed et al. proposed a thermal imaging CFRP defect detection method based on wavelet integral alternating sparse dictionary matrix factorization [16]. The above papers mainly study the thermal imaging diagnostic system for detecting product defects in the production process. Currently, there are few articles on NMF methods for industrial process fault monitoring.Yang et al. proposed a method of fault diagnosis based on NMF and SVM [17]. They built an online monitoring model through NMF and then used SVM to build a fault classifier for further fault identification. Considering the imperfection of process data, Li et al. proposed a method based on robust nonnegative matrix projection (RNMP) to detect and diagnose faults in industrial processes [18]. Li et al. proposed an improved NMF (MNMF) method to extract potential variables in the process and combine it with process monitoring technology for fault detection [19].However, the above process monitoring methods based on NMF only consider the global model of process data and ignore local information of the data, resulting in inaccurate monitoring results. To solve these problems, process data are usually divided into several subblocks, and then a monitoring model for each subblock is established separately. Jiang and Yan developed a multiblock principal component analysis method based on mutual information for fault detection and isolation [20]. Tong et al. introduced an improved multiblock principal component analysis (MBPCA) algorithm to extract the specificity of each block and the block fraction of correlation between different blocks [21]. Wang and Deng considered the local variable behavior of process data and proposed a multiblock PCA process monitoring method based on variable weight information [22]. Wang et al. proposed an adaptive partitioned nonnegative matrix factorization algorithm based on nonfixed subblock NMF model for fault monitoring in chemical processes [23]. Although they considered the local information of the data, they did not consider the mode identification of multimode processes. These multiblock monitoring methods have proved to be generally effective. However, multimode and non-Gaussian characteristics of the data should also be considered in process monitoring.In this study, a multiblock projection nonnegative matrix factorization (MPNMF) algorithm for complex chemical processes in a multimode state is proposed. Firstly, the training data of each mode is divided into several subblocks by a complete link algorithm. Secondly, a PNMF model for each subblock is established for each mode, and a joint probability index is constructed to judge the current running mode. Finally, the statistics of each block in the recognition mode is combined by Bayesian information criterion to monitor whether abnormal events occur. The proposed method takes into account the multimode and non-Gaussian characteristics of process data, combines the advantages of partitioning idea and PNMF method, makes full use of local and global information of the data, enhances the sparsity of the data, and has good process monitoring performance.The main contributions are as follows: (1) a multimode identification method based on joint probability is proposed, which can realize the identification of process operation mode. (2) A fault monitoring model based on multiblock PNMF algorithm is proposed. This paper uses the PNMF method to extract the main characteristics of the data and establish a monitoring model, which expands the application of PNMF in the field of process monitoring.


## 2. Preliminaries

This section briefly reviews the NMF and the PNMF algorithms, the process monitoring methods based on the PNMF method.


### 2.1. Nonnegative Matrix Factorization (NMF)

As a new matrix factorization algorithm, in addition to nonnegative, the NMF method has no other requirements on the original data [10]. Since the decomposed matrix elements are only added and calculated, and the decomposed base matrix is sparse, NMF can learn the local features of the data and has better explanatory performance.SupposingX=x1,x2,⋯,xm∈Rm×n is a nonnegative matrix, the core of NMF is to find W∈Rm×k and H∈Rk×n (belonging to a suitable set of nonnegative matrices) to satisfy the following formula:
(1)X≈WH,where m is the number of variables, n is the number of samples, k is the dimension of the low-dimensional space after dimensionality reduction, W is the base matrix, and H is the coefficient matrix. In addition, Eq. (1) should satisfy m+nk<mn.The solution of NMF can be summed up as a nonlinear optimization process. That is, an objective function is defined to describe the approximation of X and WH, and an appropriate set of iterative rules is found to solve the optimization problem. Lee and Seung proposed two kinds of objective functions to solve the optimization problem and gave iterative rules [10]. The first objective function “Euclidean distance” was selected in this paper.The objective function can be expressed as follows:(2)argminW,HE=12X−WHF2,(3)s.t.∀ij,Wij≥0,Hij≥0.The iterative rules of the above objective function are as follows:(4)Wij⟵WijXHTijWHHTij,Hij⟵HijWTXijWTWHij.Under the above iterative rule,W retains the behavior information of the original matrix X, H is the low-dimensional approximation matrix of X.


### 2.2. Projection Nonnegative Matrix Factorization (PNMF)

In 2005, Yuan and Oja in [11] proposed the PNMF algorithm. The basic idea of the algorithm is to replace the coefficient matrix H with WTX, which increases the sparsity of the matrix. PNMF improves the framework of NPF and reduces an unknown variable, which makes the solution complexity lower.LetWTX=H, Eq. (1) can be expressed as follows:
(5)X=WWTX,where the value of WWTshould be infinitely close to the identity matrix.Similarly, Euclidean distance in Eq. (2) is used to measure the proximity of both sides of Eq. (5). The objective function of the PNMF algorithm is as follows:
(6)argminWF=12X−WWTXF2,s.t.∀ij,Wij≥0.The iterative formula for the objective function is as follows [11]:
(7)Wij⟵Wij2XXTWijWWTXXTWij+XXTWWTWij.At present, the initialization schemes often used are random value, singular value decomposition (SVD), and principal component analysis (PCA) [13, 14, 18]. Wang et al. pointed out that using the PCA’s load matrix as the initial value of W will make NMF converge faster and minimize errors [18]. Therefore, this paper uses the method proposed by Wang et al. to obtain the initial value of NMF.The fault monitoring model of the PNMF is as follows:(8)X=X^+X~=WH^+X~,where X^ is the estimation matrix of X, which contains the main feature information of matrix X. X~ is a residual matrix. H^=WTX is a low-order approximation matrix.In general, the monitoring statistical indicators of PNMF areN2 and SPE as follows [19]:
(9)N2=h∧Th^,=WTxTWTx,=xTWWTx≤Nlim2,(10)SPE=x−x∧2,=x−WWTx2,=xTI−WWTTI−WWTx≤SPElim,with h^, x, x^, and x~ as the column vectors of the matrices H^, X, X^, and X~, respectively. The N2 statistic is the monitoring indicator of the X^ space, and the SPE statistic is the monitoring indicator of the X~ space. Some fault information are reflected in the X^ space, and some fault information are reflected in the X~ space. In order to monitor the fault more accurately, two monitoring statistics are usually monitored at the same time in many literatures [18–23].Nlim2 and SPElim are confidence limits of statistics N2 and SPE. If statistic N2 or SPE of the measurement sample exceeds the confidence limit, an abnormality is considered to have occurred and the system issues an alarm. Because the PNMF method does not assume that the data follow a certain fixed distribution, the kernel density estimation (KDE) can be used to calculate the confidence limits Nlim2 and SPElim of statistics N2 and SPE. The specific KDE algorithm can be referred to in [19].


## 2.1. Nonnegative Matrix Factorization (NMF)

As a new matrix factorization algorithm, in addition to nonnegative, the NMF method has no other requirements on the original data [10]. Since the decomposed matrix elements are only added and calculated, and the decomposed base matrix is sparse, NMF can learn the local features of the data and has better explanatory performance.SupposingX=x1,x2,⋯,xm∈Rm×n is a nonnegative matrix, the core of NMF is to find W∈Rm×k and H∈Rk×n (belonging to a suitable set of nonnegative matrices) to satisfy the following formula:
(1)X≈WH,where m is the number of variables, n is the number of samples, k is the dimension of the low-dimensional space after dimensionality reduction, W is the base matrix, and H is the coefficient matrix. In addition, Eq. (1) should satisfy m+nk<mn.The solution of NMF can be summed up as a nonlinear optimization process. That is, an objective function is defined to describe the approximation of X and WH, and an appropriate set of iterative rules is found to solve the optimization problem. Lee and Seung proposed two kinds of objective functions to solve the optimization problem and gave iterative rules [10]. The first objective function “Euclidean distance” was selected in this paper.The objective function can be expressed as follows:(2)argminW,HE=12X−WHF2,(3)s.t.∀ij,Wij≥0,Hij≥0.The iterative rules of the above objective function are as follows:(4)Wij⟵WijXHTijWHHTij,Hij⟵HijWTXijWTWHij.Under the above iterative rule,W retains the behavior information of the original matrix X, H is the low-dimensional approximation matrix of X.


## 2.2. Projection Nonnegative Matrix Factorization (PNMF)

In 2005, Yuan and Oja in [11] proposed the PNMF algorithm. The basic idea of the algorithm is to replace the coefficient matrix H with WTX, which increases the sparsity of the matrix. PNMF improves the framework of NPF and reduces an unknown variable, which makes the solution complexity lower.LetWTX=H, Eq. (1) can be expressed as follows:
(5)X=WWTX,where the value of WWTshould be infinitely close to the identity matrix.Similarly, Euclidean distance in Eq. (2) is used to measure the proximity of both sides of Eq. (5). The objective function of the PNMF algorithm is as follows:
(6)argminWF=12X−WWTXF2,s.t.∀ij,Wij≥0.The iterative formula for the objective function is as follows [11]:
(7)Wij⟵Wij2XXTWijWWTXXTWij+XXTWWTWij.At present, the initialization schemes often used are random value, singular value decomposition (SVD), and principal component analysis (PCA) [13, 14, 18]. Wang et al. pointed out that using the PCA’s load matrix as the initial value of W will make NMF converge faster and minimize errors [18]. Therefore, this paper uses the method proposed by Wang et al. to obtain the initial value of NMF.The fault monitoring model of the PNMF is as follows:(8)X=X^+X~=WH^+X~,where X^ is the estimation matrix of X, which contains the main feature information of matrix X. X~ is a residual matrix. H^=WTX is a low-order approximation matrix.In general, the monitoring statistical indicators of PNMF areN2 and SPE as follows [19]:
(9)N2=h∧Th^,=WTxTWTx,=xTWWTx≤Nlim2,(10)SPE=x−x∧2,=x−WWTx2,=xTI−WWTTI−WWTx≤SPElim,with h^, x, x^, and x~ as the column vectors of the matrices H^, X, X^, and X~, respectively. The N2 statistic is the monitoring indicator of the X^ space, and the SPE statistic is the monitoring indicator of the X~ space. Some fault information are reflected in the X^ space, and some fault information are reflected in the X~ space. In order to monitor the fault more accurately, two monitoring statistics are usually monitored at the same time in many literatures [18–23].Nlim2 and SPElim are confidence limits of statistics N2 and SPE. If statistic N2 or SPE of the measurement sample exceeds the confidence limit, an abnormality is considered to have occurred and the system issues an alarm. Because the PNMF method does not assume that the data follow a certain fixed distribution, the kernel density estimation (KDE) can be used to calculate the confidence limits Nlim2 and SPElim of statistics N2 and SPE. The specific KDE algorithm can be referred to in [19].


## 3. Methodology

In this section, the proposed multimode process monitoring method based on MPNMF model will be introduced in detail. In this method, a complete link algorithm is used first to block the training data, then multiple PNMF models are established, and mode identification is performed using joint probability. Finally, two monitoring indicators are constructed using BIC to achieve fault detection.


### 3.1. Complete Link Algorithm

The complete link algorithm is a very popular method for calculating hierarchical clustering. Given a setX=x1,x2,⋯,xm∈Rm×n of variables, its core idea is to treat each set of variable data xii=1,2,⋯,m as a separate cluster. Then, merge the two closest clusters based on some distance metric to get the next cluster and repeat the merging process until only one cluster remains. In addition, the complete linking algorithm uses the distance between the most distant sample pairs in two clusters as the distance between the two clusters [24].The specific process of dividing subblocks based on the complete link algorithm is as follows:(1)
Taking each variable data as a cluster, the matrixX is divided into m clusters
(2)
Calculating the distancedxi,xj between two different clusters xii=1,2,⋯,m and xjj=1,2,⋯,m. Euclidean distance dxi,xj=∑k=1nxik−xjk21/2is a common distance function, which is selected as the distance function of the complete link algorithm
(3)
Conducting distance matrix as following:
(11)D=0dx1,x2⋯dx1,xmdx2,x10⋯dx2,xm⋮⋮⋱⋮dxm,x1dxm,x2⋯0∈Rm×m.Merging the two clusters with the smallest distance into the new cluster xi,xj and calculating the distance between xi,xj and the remaining clusters as follows:
(12)d¯xi,xj,xk=maxdxi,xk,dxj,xk.
(4)
Updating the distance matrixD and repeat Step (3) until the desired number of clusters is obtained


### 3.2. MPNMF Models

Supposing a set of historical normal data matricesX∈Rm×n, where m and n are the number of variables and sample, and X can be divided into several subblocks using complete link algorithm. Therefore, the matrix X can be rewritten as:
(13)X=X1,X2,⋯,XB,b=1,2,⋯,B,with B being the number of subblocks, Xb∈Rmb×n, and mb the number of variables in the bth subblock. Then, the PNMF model of each subblock is established as follows:
(14)Xb=WbWbTXb,with Wb∈Rmb×kb being the basis matrix of the bth subblock matrix decomposition, kb the dimension of the low-dimensional space after the dimensionality reduction of the b subblock, and the value of WbWbT is infinitely close to the identity matrix.According to Eqs. (9) and (10), the monitoring statistical indicators Nb2 and SPEb of each PNMF model can be obtained as follows:
(15)Nb2=WbTxbTWbTxb=xbTWbWbTxb≤Nb,lim2,(16)SPEb=xbTI−WbWbTI−WbWbTTxb≤SPEb,lim,with xb∈Rmb×1 as an measurement vector, Nb,lim2 and SPEb,lim are confidence limits of statistics Nb2 and SPEb.


### 3.3. Joint Probability for Mode Identification

Jiang and Yan used monitoring statistical indicatorsT2 to identify the current sample mode and achieved good mode identification results [7]. Inspired by them, N2 statistics with a large number of data features are used for mode identification in this paper.For a current observation samplexnew∈Rm×1, it is divided into B subblocks according to the partitioning results of the previous historical data, and get xnew=x1,new,x2,new,⋯,xB,new. Then, Nb,new2 is calculated for each subblock b in each corresponding mode. In order to identify the mode of the current subblock sample xb,new, the mode probability p is defined as follows [7]:
(17)pxb,new∈Mi=e−Ni,b2,with Mi as the ith mode, and Ni,b2 as the Nb,new2 in the ith mode.Sincexnew∈Mi is equal to that of x1,new∈Mi∩x2,new∈Mi∩⋯∩xB,new∈Mi, the probability of event x∈Mi is calculated by the joint probability of each block as follows:
(18)pxnew∈Mi=px1,new∈Mipx2,new∈Mi⋯pxB,new∈Mi,=e−Ni,12⋅e−Ni,22⋅⋯⋅e−Ni,B2,=e−Ni,12+Ni,22+⋯+Ni,B2.If the observation samplexnew∈Rm×1 is assigned to a mismatch mode, the joint probability p is close to zero according to Jiang and Yan’s study [7]. Therefore, it can be determined that the mode with the highest joint probability is the running mode of the current process.


### 3.4. Fault Monitoring Index Based on Bayesian Information Criterion

After determining the mode of the current measurement samplexnew, it is necessary to monitor it. Different subblocks have different monitoring statistics. To provide intuitive monitoring results, Bayesian information criterion (BIC) is often used to combine results with probabilistic methods [25]. The fault probability of xb,new corresponding to N2 statistics is defined as follows:
(19)PN2F∣xb,new=PN2xb,new∣FPN2FPN2xb,new,PN2xb,new=PN2xb,new∣NPN2N+PN2xb,new∣FPN2F,with F as the fault condition and N as the normal condition. The confidence level of pN2N is α0≤α≤1 and the confidence level of pN2F is 1−α. Two conditional probabilities pN2xb,newN and pN2xb,newF are defined as:
(20)pN2xbN=e−Nb,new2/Nb,lim2,pN2xbF=e−Nb,lim2/Nb,new2.Using BIC to combineN2 statistics in all subblocks to get the final statistic, as shown below:
(21)BICN2=∑b=1BpN2xnew,bFpN2Fxnew,b∑b=1BpN2xnew,bF.Similarly, using BIC to combineSPE statistic to get the final statistic, as shown below:
(22)BICSPE=∑b=1BpSPExnew,bFpSPEFxnew,b∑b=1BpSPExnew,bF.The final statistical value within the confidence levelα can be regarded as normal operation data.


### 3.5. The Proposed Method Implementation

In this paper, a multimode process monitoring method based on MPNMF is proposed. The method mainly includes offline modeling and online monitoring, as shown in Figures1 and 2. The implementation of these two parts is as follows:Figure 1
Offline modeling flow chart.Figure 2
Online monitoring flowchart.Part 1: offline modeling(1)
Collecting historical normal dataX∈Rm×n for different modes
(2)
Dividing the historical data of each mode intoB subblocks by the complete link algorithm
(3)
Establishing the PNMF model for each subblock in different modes according to Eq. (14)
(4)
Calculating the monitoring statisticsNb2 and SPEb of each subblock according to Eqs. (15) and (16), and the corresponding control limits Nb,lim2 and SPEb,limPart II: online monitoring(1)
Sampling measurement dataxnew∈Rm×1 and dividing the current data into corresponding subblocks xnew=xnew1,xnew2,⋯,xnewB
(2)
Identifying the mode ofxnew by joint probability
(3)
Calculating the monitoring statisticsNb,new2 and SPEb,newof each subblock according to Eqs. (15) and (16)
(4)
Calculating statisticsBICN2 and BICSPE according to equations (21) and (22)
(5)
Comparing statisticsBICN2 and BICSPE with the confidence level α. If BICN2≥α or BICSPE≥α, a fault is considered to have occurred


## 3.1. Complete Link Algorithm

The complete link algorithm is a very popular method for calculating hierarchical clustering. Given a setX=x1,x2,⋯,xm∈Rm×n of variables, its core idea is to treat each set of variable data xii=1,2,⋯,m as a separate cluster. Then, merge the two closest clusters based on some distance metric to get the next cluster and repeat the merging process until only one cluster remains. In addition, the complete linking algorithm uses the distance between the most distant sample pairs in two clusters as the distance between the two clusters [24].The specific process of dividing subblocks based on the complete link algorithm is as follows:(1)
Taking each variable data as a cluster, the matrixX is divided into m clusters
(2)
Calculating the distancedxi,xj between two different clusters xii=1,2,⋯,m and xjj=1,2,⋯,m. Euclidean distance dxi,xj=∑k=1nxik−xjk21/2is a common distance function, which is selected as the distance function of the complete link algorithm
(3)
Conducting distance matrix as following:
(11)D=0dx1,x2⋯dx1,xmdx2,x10⋯dx2,xm⋮⋮⋱⋮dxm,x1dxm,x2⋯0∈Rm×m.Merging the two clusters with the smallest distance into the new cluster xi,xj and calculating the distance between xi,xj and the remaining clusters as follows:
(12)d¯xi,xj,xk=maxdxi,xk,dxj,xk.
(4)
Updating the distance matrixD and repeat Step (3) until the desired number of clusters is obtained


## 3.2. MPNMF Models

Supposing a set of historical normal data matricesX∈Rm×n, where m and n are the number of variables and sample, and X can be divided into several subblocks using complete link algorithm. Therefore, the matrix X can be rewritten as:
(13)X=X1,X2,⋯,XB,b=1,2,⋯,B,with B being the number of subblocks, Xb∈Rmb×n, and mb the number of variables in the bth subblock. Then, the PNMF model of each subblock is established as follows:
(14)Xb=WbWbTXb,with Wb∈Rmb×kb being the basis matrix of the bth subblock matrix decomposition, kb the dimension of the low-dimensional space after the dimensionality reduction of the b subblock, and the value of WbWbT is infinitely close to the identity matrix.According to Eqs. (9) and (10), the monitoring statistical indicators Nb2 and SPEb of each PNMF model can be obtained as follows:
(15)Nb2=WbTxbTWbTxb=xbTWbWbTxb≤Nb,lim2,(16)SPEb=xbTI−WbWbTI−WbWbTTxb≤SPEb,lim,with xb∈Rmb×1 as an measurement vector, Nb,lim2 and SPEb,lim are confidence limits of statistics Nb2 and SPEb.


## 3.3. Joint Probability for Mode Identification

Jiang and Yan used monitoring statistical indicatorsT2 to identify the current sample mode and achieved good mode identification results [7]. Inspired by them, N2 statistics with a large number of data features are used for mode identification in this paper.For a current observation samplexnew∈Rm×1, it is divided into B subblocks according to the partitioning results of the previous historical data, and get xnew=x1,new,x2,new,⋯,xB,new. Then, Nb,new2 is calculated for each subblock b in each corresponding mode. In order to identify the mode of the current subblock sample xb,new, the mode probability p is defined as follows [7]:
(17)pxb,new∈Mi=e−Ni,b2,with Mi as the ith mode, and Ni,b2 as the Nb,new2 in the ith mode.Sincexnew∈Mi is equal to that of x1,new∈Mi∩x2,new∈Mi∩⋯∩xB,new∈Mi, the probability of event x∈Mi is calculated by the joint probability of each block as follows:
(18)pxnew∈Mi=px1,new∈Mipx2,new∈Mi⋯pxB,new∈Mi,=e−Ni,12⋅e−Ni,22⋅⋯⋅e−Ni,B2,=e−Ni,12+Ni,22+⋯+Ni,B2.If the observation samplexnew∈Rm×1 is assigned to a mismatch mode, the joint probability p is close to zero according to Jiang and Yan’s study [7]. Therefore, it can be determined that the mode with the highest joint probability is the running mode of the current process.


## 3.4. Fault Monitoring Index Based on Bayesian Information Criterion

After determining the mode of the current measurement samplexnew, it is necessary to monitor it. Different subblocks have different monitoring statistics. To provide intuitive monitoring results, Bayesian information criterion (BIC) is often used to combine results with probabilistic methods [25]. The fault probability of xb,new corresponding to N2 statistics is defined as follows:
(19)PN2F∣xb,new=PN2xb,new∣FPN2FPN2xb,new,PN2xb,new=PN2xb,new∣NPN2N+PN2xb,new∣FPN2F,with F as the fault condition and N as the normal condition. The confidence level of pN2N is α0≤α≤1 and the confidence level of pN2F is 1−α. Two conditional probabilities pN2xb,newN and pN2xb,newF are defined as:
(20)pN2xbN=e−Nb,new2/Nb,lim2,pN2xbF=e−Nb,lim2/Nb,new2.Using BIC to combineN2 statistics in all subblocks to get the final statistic, as shown below:
(21)BICN2=∑b=1BpN2xnew,bFpN2Fxnew,b∑b=1BpN2xnew,bF.Similarly, using BIC to combineSPE statistic to get the final statistic, as shown below:
(22)BICSPE=∑b=1BpSPExnew,bFpSPEFxnew,b∑b=1BpSPExnew,bF.The final statistical value within the confidence levelα can be regarded as normal operation data.


## 3.5. The Proposed Method Implementation

In this paper, a multimode process monitoring method based on MPNMF is proposed. The method mainly includes offline modeling and online monitoring, as shown in Figures1 and 2. The implementation of these two parts is as follows:Figure 1
Offline modeling flow chart.Figure 2
Online monitoring flowchart.Part 1: offline modeling(1)
Collecting historical normal dataX∈Rm×n for different modes
(2)
Dividing the historical data of each mode intoB subblocks by the complete link algorithm
(3)
Establishing the PNMF model for each subblock in different modes according to Eq. (14)
(4)
Calculating the monitoring statisticsNb2 and SPEb of each subblock according to Eqs. (15) and (16), and the corresponding control limits Nb,lim2 and SPEb,limPart II: online monitoring(1)
Sampling measurement dataxnew∈Rm×1 and dividing the current data into corresponding subblocks xnew=xnew1,xnew2,⋯,xnewB
(2)
Identifying the mode ofxnew by joint probability
(3)
Calculating the monitoring statisticsNb,new2 and SPEb,newof each subblock according to Eqs. (15) and (16)
(4)
Calculating statisticsBICN2 and BICSPE according to equations (21) and (22)
(5)
Comparing statisticsBICN2 and BICSPE with the confidence level α. If BICN2≥α or BICSPE≥α, a fault is considered to have occurred


## 4. Simulation Study of TE Process

The Tennessee Eastman (TE) process simulation platform is developed by Eastman Chemical Company based on actual chemical processes. It can not only simulate the chemical production process and generate a large amount of normal data but also can obtain multiple fault data by setting multiple fault modes. Therefore, the TE process is often used to evaluate the effectiveness of various fault detection methods. As shown in Figure3, the TE process is mainly composed of five operating units: a reactor, a condenser, a gas-liquid separator, a stripper, and a circulating compressor. A variety of control strategies have been proposed for the research of TE process. Ricker [26] proposed a decentralized control structure and gave six different operation modes according to the G/H ratio, as shown in Table 1, and the simulation data can be downloaded from http://depts.washington.edu/control/LARRY/TE/download.html. Lyman and Georgakis [27] proposed a plant-wide control scheme, and simulation data can be downloaded from http://web.mit.edu/braatzgroup/links.html.Figure 3
The diagram of the TE process.Table 1
Operation modes of the TE process.
Operating mode
G/H
Production rate (kg/h)
1
50/50
14,076
2
10/90
14,076
3
90/10
11,111
4
50/50
Maximum
5
10/90
Maximum
6
90/10
MaximumSelect mode 1 and mode 2 proposed by Richer and mode 3 proposed by Lyman and Georgakis to simulate multimodal processes. 33 variables (11 control variables, 22 measured variables) are selected for fault detection and diagnosis. The specific details of the variables can be found in [26].Since the NMF-based method is very sensitive to the initial values ofWij and Hij, the selection of the initial values of Wij and Hij is important. At present, the initialization schemes often used are random value, singular value decomposition (SVD), and principal component analysis (PCA) [18, 19, 23]. Wang et al. pointed out that using the PCA’s load matrix as the initial value of Wij will make NMF converge faster and minimize errors [23]. Therefore, this paper uses the method proposed by Wang et al. to obtain the initial value of NMF. 500 normal history samples were collected in each mode to build a monitoring model for each mode. In each mode, 21 faults are simulated, and the fault names are referenced in [27]. Each set of measurement data contains 960 samples, of which the first 160 samples are normal and the fault starts from the 161st sample and continues to the end of the process. The article uses 85% of the contribution of PCA to select the number of principal elements. The number of principal elements is used as the number of dimensionality reductions of all monitoring algorithms. The number of iterations of the NMF-based algorithm is 1000. All monitoring algorithms use kernel density estimation to calculate the control limits of statistical indicators, with a confidence level of 99%.In order to verify the validity of mode identification, two data sets were collected for testing, each of which contains 960 groups of samples, as shown below.Case 1.
Fault 0 in TE process represents a normal process. Normal data in different modes are used to test the performance of mode identification. In the system, samples range from 1 to 400 runs in mode 1, range from 401 to 560 runs in mode 2, and range from 561 to 960 runs in mode 3.Case 2.
Fault 1 in TE process is a step change in the A/C feed ratio, and the fault continues from the 161st point to the end of the process. The system runs from the 1st sample point to the 960th point in mode 2.The joint probability of the three modes in Case1. is shown in Figure 4(a). In Figure 4(a), it can be seen that as the operation mode changes, the joint probability will change. From point 1 to point 100, the joint probability of mode 1 is the highest, indicating that the system is operating in mode 1, followed by modes 2 and 3. The joint probability can be used to derive the operating mode of the system. The joint probability of the three modes in s2. is shown in Figure 4(b). Therefore, Figure 4(b) shows that the system is operating in mode 2.Figure 4
Joint probability: (a) case1.; (b) case 2..
(a)
(b)Once the mode of the measurement sample is identified, it is necessary to monitor it. PCA method, NMF method, and PNMF method are selected as reference objects to verify the effectiveness of MPNMF method in fault monitoring. Fault detection accuracy refers to the number of samples detected by the algorithm and the percentage of total fault samples. The fault detection results of the main methods (PCA, NMF, PNMF, and MPNMF) which are mentioned in the introduction are shown in Table2. The NMF-based algorithm used in Table 2 refers to the NMF software package published online by Professor Chih-Jen Lin of Tsinghua University in Taiwan (website: https://www.csie.ntu.edu.tw/cjlin/nmf).Table 2
Comparison of fault detection accuracy (%).
Fault no.
PCA
NMF
PNMF
MPNMF
T2
SPE
N2
SPE
N2
SPE
BICN2
BICSPE
1
99.3
100
99.6
97.4
99.6
91.6
99.9
91.2
2
98.3
99
98.4
96.8
98.4
96.0
98.6
95.6
3
6.3
4.9
10.4
8.8
15.4
9.1
21.4
12.1
4
31.1
100
77.5
99.2
57.7
99.0
100
11.6
5
27.8
26.9
33
36.1
35.9
34.3
36.8
98.9
6
99.4
100
100
99.9
100
99.6
100
100
7
100
100
97.8
100
100
100
100
100
8
97.4
94.9
98
97.3
98.6
97.8
98.9
97.1
9
5.3
4.6
7.8
16.5
10.3
15.4
13.3
11.3
10
45.6
49.6
55.4
57.4
57.0
57.8
63.6
59.5
11
48.1
80.5
69
68.3
67.1
60.9
78.1
17.7
12
98.5
94.6
99.4
98.3
99.5
98.6
99.5
100
13
94.3
95.3
94.8
93.8
94.9
94.1
95.0
94.7
14
99.5
100
100
99.9
100
91.4
100
87.7
15
8.5
8.3
19
10.8
20.5
10.9
23.8
18.0
16
30
47.6
43.9
55.8
49.0
54.5
55.8
52.4
17
80
96
89.3
95.3
87.8
95.4
91.7
88.2
18
89.9
90.5
90.5
91.4
90.5
91.1
92.2
90.2
19
14.5
28.8
10.3
37.3
8.4
39.0
27.4
31.0
20
42.5
60
55.5
67
57.5
67.3
62.8
81.2
21
40.6
56.1
46
47.5
46.6
51.6
57. 6
62.1
The significant test results are shown in bold.Compared with the results of PCA method, NMF method, and PNMF method, MPNMF method has the highest fault detection rate, especially in failure modes 3, 5, 10, 15, 20, and 21. This paper takes faults 5 and 10 as examples to illustrate the monitoring results of different methods.Fault 5 is the step change of cooling water inlet temperature of condenser. Figure5 shows the detection results of NMF, PNMF, and MPNMF methods. In Figure 5(a), it is easy to see that the fault detection rate of global NMF is lower. The detection rates of N2 statistics and SPE statistics are 33% and 36.1%, respectively. Figure 5(b) shows the detection results of PNMF with a detection rate of N2 statistics is 35.9% and SPE statistics is 34.3%. In Figure 5(c), the detection results of MPNMF with the two detection rate of N2 statistics and SPE statistics are 36.8% and 98.9%, respectively. Therefore, the detection performance of the proposed MPNMF method is superior to NMF and PNMF, and the SPE detection effect is more obvious, which proves that the method is effective.Figure 5
Fault detection results: (a) NMF method; (b) PNMF method; and (c) MPNMF method.
(a)
(b)
(c)Fault 10: the fault is caused by the temperature variation of C feed. The faults are introduced from the 161th to the 960th sample points, and the fault type is random fault. The detection results of the NMF, PNMF, and MPNMF methods are shown in Figure6. Figure 6(a) shows the detection results of NMF with the detection rates of N2 statistic and SPE statistic are 55.4% and 57.4%, respectively. Figure 6(b) is the PNMF detection result that the detection rates of N2 statistic and SPE statistic are 57% and 57.8%, respectively, and it is obvious that the detection effect is better than that of NMF. Figure 6(c) is the detection result of MPNMF, and the detection rates of N2 statistic and SPE statistic are 63.6% and 59.5%, respectively. The overall detection effect of MPNMF is better than NMF and PNMF. In summary, in the monitoring of the TE process, the MPNMF method proposed in this paper has greater advantages than the traditional NMF method and PNMF method.Figure 6
Fault detection results: (a) NMF method; (b) PNMF method; and (c) MPNMF method.
(a)
(b)
(c)To compare the computational complexity of the various methods, the dataset was run on a PC with a 2.2 GHz dual-core processor (i5 Intel processor), 4GB RAM, and 1 TB HDD.Table3 shows the computational time of each fault monitoring program. Computational time is in seconds. Since the fault monitoring process includes offline modeling and online monitoring, Table 3 gives the computational time of offline modeling and online monitoring. Although the proposed method has the longest offline modeling time, the online monitoring time is shorter. Industrial process monitoring is expected to have higher accuracy and lower monitoring time. As can be seen from Tables 2 and 3, the proposed method has better monitoring results.Table 3
Computational time.
Method
Computational time (s)
Offline modeling time
Online monitoring time
PCA
0.35
0.0015
NMF
0.78
0.0021
PNMF
2.06
0.0004
MPNMF
10.9
0.0011


## 5. Conclusion

Traditional process monitoring methods usually adopt a global model of data and ignore local information about the data. Compared with the NMF algorithm, the PNMF algorithm converges faster and has more advantages in data reduction and decomposition. Combining the advantages of block modeling and PNMF algorithm, a multimode process monitoring method based on MPNMF is proposed. In order to make full use of the local information of the process data, the entire process is divided into several subblocks by a complete link algorithm, a PNMF model of each subblock is established, and then a joint probability indicator is constructed to identify the current running mode of the process. Finally, BIC is adapted to synthesize the statistics of each subblock to realize process monitoring. The proposed process monitoring method is applied to the TE process to verify its effectiveness, and compared with the traditional PCA algorithm, NMF algorithm, and PNMF algorithm, the results show that the method can monitor the fault timely and effectively.In future work, a more flexible model can be used instead of the PNMF model. With the help of historical fault data information, building a more optimized multiblock distributed monitoring model will become the focus of future research.



---

*Source: 4610493-2020-04-24.xml*