# Comprehensive Monitoring of Complex Industrial Processes with Multiple Characteristics

Authors: Chenxing Xu; Jiarula Yasenjiang; Pengfei Cui; Shengpeng Zhang; Xin Zhang
Journal: International Journal of Chemical Engineering
 (2022)

---

## Abstract

Traditional onefold data-driven methods for fault detection in complex process industrial systems with high-dimensional, linear, nonlinear, Gaussian, and non-Gaussian coexistence often have less than satisfactory monitoring performance because only a single distribution of process variables is considered. To address this problem, a hybrid fault detection model based on PCA-KPCA-ICA-KICA-BI (Bayesian inference) is proposed, taking into account the advantages of principal component analysis (PCA), kernel principal component analysis (KPCA), independent component analysis (ICA), and kernel independent component analysis (KICA) in terms of dimensionality reduction and feature extraction. Foremost, this paper proposed a nonlinear evaluation method and divided the feature variables into Gaussian linear blocks, Gaussian nonlinear blocks, non-Gaussian linear blocks, and non-Gaussian nonlinear blocks by using the Jarque–Bera (JB) test and nonlinear discrimination method. Each division was monitored by the PCA-KPCA-ICA-KICA model, and finally the Bayesian fusion strategy proposed in this study is used to synthesize the detection results for each block. The hybrid model helps in evaluating variable features and bettering detection performance. Ultimately, the superiority of this hybrid model was verified through the Tennessee Eastman (TE) process and the Continuous Stirred Tank Reactor (CSTR) process, and the fault monitoring results showed an average accuracy of 85.91% for this hybrid model.


---

## Body

## 1. Introduction

People now set a higher threshold for the production quality, system performance, economical efficiency, and increasingly complicated grow process industry systems at the structure and automation level, amid the rapid advance of artificial intelligence and sensor detection technology. Reliability and safety of complex industrial processes now jostle for increasing attention and have to be safeguarded urgently. Industrial process fault detection shines in effectively improving product quality, safe operation, and continuous production. Condition monitoring can be performed by knowledge- and model-based or data-driven approaches [1–5]. The first two kinds face great challenges in dealing with complex process industrial system as they require a large number of model parameters and prior knowledge. Operation data can be recorded, transmitted, and stored; these features lay the foundation for data-driven fault detection techniques [6–8].Data-driven fault detection methods have been intensively studied over the past years. Great progress was seen in the Multivariate Statistical Process Monitoring (MSPM). For example, principal component analysis (PCA), partial least squares (PLS), canonical correlation analysis (CCA), independent component analysis (ICA), and other methods have been widely studied and applied [3, 9–12]. MSPM dimensionally reduces high-dimensional process variables to low-dimensional space by projection under normal circumstances, thus outputting a process monitoring model. To address the issue of early fault detection, because the memory monitoring charts are sensitive to incipient anomalies in the process mean, the advantages of PCA and multivariate memory monitoring schemes are used for early abnormality detection, and the results showed that the scheme could detect early anomalies in multivariate data [13]. Given complex industrial process systems, varying basic MSPM improvement schemes have been proposed. For example, to reduce computational cost and time while ensuring that important information is not lost or deleted, a dynamic simplification method which enhances the nonlinear process monitoring capability of KPLS is proposed [14]. In addition, an iterative robust kernel principal component analysis is brought forth to improve the robustness of the fault detection model by iteratively optimizing the function index and the kernel method [15]. The KPCA was revised to deal with nonlinear optimization problems. The newly improved approach greatly enhanced the fault detection performance of the original kernel principal component analysis [16]. Facts prove that not all variables of complex industrial processes completely obey the Gaussian distribution. ICA outperforms PLS as it saves the trouble of determining the input/output relationship of the system. Therefore, a new kernel independent component analysis method was driven by the aforementioned non-Gaussian nonlinear distribution. The proposed method effectively captures the nonlinear relationship in the process variables [17]. The above methods have improved the fault detection performance and model stability to a certain extent. However, the above methods all have certain assumptions, such as variables obeying a Gaussian or non-Gaussian distribution and linear or nonlinear correlation between variables. Such assumptions are unrealistic in complex industrial processes. Therefore, complex industrial processes require further research.Given the characteristic that the process variables do not fully obey the Gaussian distribution, [18] an approach integrating ICA and PCA was proposed, which monitors non-Gaussian distributed variables and Gaussian distributed variables making use of ICA and PCA, respectively. Reference [19] proposed Gaussian and non-Gaussian dual subspace statistical process monitoring. Foremost, the D test identified the normality of the process variables, then divided the process variables into Gaussian and non-Gaussian subspaces, and finally classified the PCA and ICA models for fault detection in Gaussian and non-Gaussian subspaces, respectively. Reference [20] proposed a dynamic non-Gaussian mixture serial modeling method for industrial process monitoring, using the multivariate non-Gaussian evaluation method to divide the industrial process variables into Gaussian variable subspace and non-Gaussian variable subspace. Then, using DICA and DPCA, the two subspaces were monitored, and the monitoring performance was improved to a certain extent. Aiming at the characteristics of incomplete linear correlation of process variables, [21] proposed a new hybrid linear-nonlinear statistical model (SPCA) for nonlinear process monitoring, using PCA to extract the characteristics of linear features to monitor the linear subspace. Using KPCA can extract the characteristics of nonlinear features to monitor the nonlinear subspace, which makes better use of the underlying process to improve the monitoring performance. Reference [22] proposed a parallel PCA-KPCA (P-PCA-KPCA) model and monitoring scheme combining stochastic algorithm (RA) and genetic algorithm (GA). The GA-based optimization method was used to determine whether the parallel principal component analysis model (P-PCA) and parallel KPCA models (P-KPCA) contain variables, and the proposed method can effectively handle nonlinear processes. In addition, some studies have adopted layering, subspace division, and Bayesian decision fusion mechanisms for process monitoring, and the monitoring performance has been improved to a certain extent [23–25].Although the aforementioned research has been successful, the detection of faults in complex industrial processes where multiple features coexist remains a challenge. Complex industrial process variables always present high-dimensional, linear, nonlinear, Gaussian, and non-Gaussian coexistence. The proposed hybrid model provides a new direction for complex industrial process fault detection; however, the lack of in-depth research on nonlinear evaluation methods for process variables has led to the poor performance of existing hybrid models for complex industrial process fault detection with linear, nonlinear, Gaussian, and non-Gaussian feature coexistence. For example, the PCA-ICA model has good performance in industrial process monitoring where linear, Gaussian, and non-Gaussian features coexist, but there are always false positives and false negatives in nonlinear industrial processes. Therefore, it is important for theoretical research and engineering practice to carry out nonlinear evaluation of process variables and to build models for monitoring complex industrial processes.In this study, a PCA-KPCA-ICA-KICA-BI mixed model is proposed for the coexistence of high-dimensional, linear, nonlinear, Gaussian, and non-Gaussian variables in complex industrial processes. First, a multivariate feature evaluation method is proposed. The original variables are classified using the Jarque–Bera test and the nonlinear discriminant method. This method calculates the kurtosis and skewness statistics of the variables. According to the normality test results, the process variables are divided into Gaussian blocks and non-Gaussian blocks; then we use Pearson correlation coefficient, maximum mutual information coefficient, and nonlinear evaluation function to divide Gaussian blocks and non-Gaussian blocks into Gaussian linear blocks, Gaussian nonlinear blocks, non-Gaussian linear blocks, and non-Gaussian blocks nonlinear blocks; then we apply the PCA-KPCA-ICA-KICA model to monitor the blocking process; and we finally propose a Bayesian inference fusion strategy to comprehensively decide the detection results of each block.Section 2 briefly reviews PCA, ICA, kernel methods, and Jarque–Bera tests. Section 3 proposes nonlinear discriminant methods and Bayesian inference fusion strategies. Section 4 describes in detail the PCA-KPCA-ICA-KICA-BI method and its construction. Section 5 presents the application of PCA-KPCA-ICA-KICA-BI in the TE process and CSTR process and analyzes the corresponding monitoring performance. Section 6 concludes the paper.


## 2. Preliminaries

### 2.1. Principal Component Analysis and Kernel Principal Component Analysis

Principal component analysis constitutes one of the common data dimensionality reduction methods in linear correlation, Gaussian, and high-dimensional process monitoring. It maps industrial process data to a low-dimensional space and then performs the analysis according to the data characteristics of the decomposed main subspace and residual subspace. For a given process datasetX=x1,x2,…,xm∈Rn×m, n represents the number of samples and m represents the number of variables. After mapping, X is decomposed into(1)X=T^P^T+E=X^+E,in which X is the normalized data matrix, P∈Rm×k is the loading matrix obtained by eigenvalue decomposition of the covariance matrix C, T∈Rn×k is the score matrix, E is the residual subspace, and k is the number of the principal components obtained from the cumulative percent variance (CPV):(2)∑i=1kλi∑i=1mλi×100%≥85%,in which λi is the eigenvalues of the covariance matrix C.When a new sampleXnew=x1,x2,x3,…,xmT is obtained, according to the aforementioned decomposition model, the score vector tnew and residual subspace vector e of the new sample are as follows:(3)tnew=xnewTP^,e=xnewTI−PPT.In process monitoring, PCA algorithms typically adopt residual squared prediction error (SPE) andT2 statistic to monitor operational status. T2 statistic measures the PCA main subspace, and the SPE statistic considers the residual subspace. The loading matrix P^ contains a lot of variance variation information. Since T2 statistic is modeled based on a loading matrix P^ with large singular values, it is sensitive to low singular value inaccuracies, which can be addressed by the SPE statistic [26, 27]. Similarly, the calculation formulas of the T2 statistics and SPE statistics of the new sample xnew are as follows [26]:(4)TPCA2=tnewT∧−1tnew,SPEPCA=eTe,in which ∧ is the diagonal matrix containing the eigenvalues of the data variance matrix.T2 and SPE control limits are calculated from normal operating data. Since the T2 statistic is in line with the F distribution, given a certain degree of confidence α (usually 99%), the statistical control limits Tα2 and SPEα are as follows:(5)TαT=km2−1mm−kFk,m−k,α,SPEα=θ1ηα2θ2h02θ1+1+θ2h0h0−1θ11/h0,in which the freedom of Fk,m−k,α is m−k; θi is the sum of the residual eigenvalues to the power of 1, 2, and 3; and h0 is the intermediate outcome variable, h0=1−2θ1θ3/3θ22.


### 2.2. Independent Component Analysis

Unlike PCA, ICA can monitor the non-Gaussian processes by extracting independent non-Gaussian features from industrial process variables. The given process data vectorX=x1,x2,…,xm∈R1×m can be represented as a linear combination of statistically independent d non-Gaussian sources s=s1,s2,s3,…,sd, in which d≤m. After normalization and preprocessing, the relationship between the original industrial process data and the independent components ICs is expressed as follows:(6)x=As+e,in which A∈Rm×d is the mixture matrix and e∈Rm×1 is the residual matrix. The basic problem of ICA is to estimate the mixture matrix A and the independent component vector s. Because the FastICA algorithm is characterized in fast calculation speed, this study uses the FastICA method [28] to calculate the decomposition matrix W and reconstruct s^ as follows:(7)s^=Wx.To eliminate the correlation between the process data and facilitate the calculation, normalization and whitening are required. This paper adopts the commonly used PCA whitening method to whiten the process data. Through singular value decomposition, the obtained covariance matrix and whitened data are as follows:(8)C=ExxT=U∧UT,a=∧−1/2UT,z=Qx=QAs,in which E represents the expectation, C is the covariance matrix, Q is the whitening matrix, and zis the whitened data. Orthogonal matrix B can be calculated by the following formula:(9)B=QA.From the above formula,s^ can be expressed as follows:(10)s^=BTz=BTQx.The relationship between the decomposition matrixW and B can be expressed by the following formula:(11)w=BTQ.WhenICs is determined, process monitoring can be performed by establishing I2 and statistical magnitude SPE. The specific process is as follows:(12)I2=s^Ts^,SPE=eTe.The kernel density estimation (KDE) is used to calculate the thresholds of the above two statistics [28].


### 2.3. Kernel Method

KPCA and KICA models map nonlinear process variables into high-dimensional space for processing through kernel methods. This section introduces the kernel method at length since the monitoring schemes of PCA and ICA models have been explicated in Sections 2.1 and 2.2.Assume the process dataX=x1,x2,…,xm∈Rn×m is a normalized 0-mean dataset. Through nonlinear mapping Φ⋅, the covariance matrix CF of the mapped data in the feature space is as follows [28]:(13)CF=1N∑j=1N∅x∅xT.CF can be diagonalized by eigenvalue decomposition as follows:(14)λv=CFv,in which λ represents the eigenvalue and satisfies λ≥0, and v represents the eigenvector. Formulas (13) and (14) suggest the following:(15)CFv=1N∑j=1n∅xj∅xjTv=1N∑∅xj,v∅xj.Since the eigenvectors of the covariance matrix are spanned by samples, there must be a coefficientαi (i=1,2,3,…,n):(16)v=∑j=1nαi∅xi.Formulas (15) and (16) suggest the following:(17)λ∑i=1nαi∅xk,∅xi=1n∑i=1nαi∅xk,∑j=1n∅xi∅xj,∅xi.The inner product of the reconstructed variables in the feature space can be described by a kernel function. This study adopts a Gaussian kernel function, which is defined as follows:(18)kx,y=∅x,∅y=expx−y22×σ2.The kernel matrixk is kij=kij=Φxi,Φxj [28], and (17) can be further simplified as follows:(19)λnkα=k2α,in which α and λ are the eigenvalues and eigenvectors of the kernel matrix k. According to the above formula, λ and v can be calculated.


### 2.4. Jarque–Bera Test

MSPM method usually performs parameter evaluation and feature extraction given that variables obey normal distribution. Although it is reasonable to assume that some industrial processes obey normality, this assumption is often questionable when faced with complex industrial processes. Therefore, the Jarque–Bera (JB) test for normality distribution of complex process data is proposed. In statistics, the Jarque–Bera test examines whether the sample data shows the skewness and kurtosis of a normal distribution and goodness.AssumeX=x1,x2,x3,…,xn to be a dataset of n independent random variables; the sample skewness and kurtosis of x1,x2,x3,…,xn are calculated as follows:(20)b1=1/n∑i=1nxi−x¯31/n∑i=1nxi−x¯23/2,b2=1/n∑i=1nxi−x¯41/n∑i=1nxi−x¯22,in which x¯=1/n∑i=1nxi; if the process variable follows a Gaussian distribution, the skewness and kurtosis of X are close to 0 and 3, respectively. The normality of the sample can be tested by the skewness and kurtosis deviation of the expected value. The formula for calculating the JB statistic is as follows:(21)JB=n6b12+b2−324.The thresholdJBα of the JB statistical magnitude is calculated from the significance level α and the number of samples n. If JB<JBα, this means that the process variable obeys the normality assumption.


## 2.1. Principal Component Analysis and Kernel Principal Component Analysis

Principal component analysis constitutes one of the common data dimensionality reduction methods in linear correlation, Gaussian, and high-dimensional process monitoring. It maps industrial process data to a low-dimensional space and then performs the analysis according to the data characteristics of the decomposed main subspace and residual subspace. For a given process datasetX=x1,x2,…,xm∈Rn×m, n represents the number of samples and m represents the number of variables. After mapping, X is decomposed into(1)X=T^P^T+E=X^+E,in which X is the normalized data matrix, P∈Rm×k is the loading matrix obtained by eigenvalue decomposition of the covariance matrix C, T∈Rn×k is the score matrix, E is the residual subspace, and k is the number of the principal components obtained from the cumulative percent variance (CPV):(2)∑i=1kλi∑i=1mλi×100%≥85%,in which λi is the eigenvalues of the covariance matrix C.When a new sampleXnew=x1,x2,x3,…,xmT is obtained, according to the aforementioned decomposition model, the score vector tnew and residual subspace vector e of the new sample are as follows:(3)tnew=xnewTP^,e=xnewTI−PPT.In process monitoring, PCA algorithms typically adopt residual squared prediction error (SPE) andT2 statistic to monitor operational status. T2 statistic measures the PCA main subspace, and the SPE statistic considers the residual subspace. The loading matrix P^ contains a lot of variance variation information. Since T2 statistic is modeled based on a loading matrix P^ with large singular values, it is sensitive to low singular value inaccuracies, which can be addressed by the SPE statistic [26, 27]. Similarly, the calculation formulas of the T2 statistics and SPE statistics of the new sample xnew are as follows [26]:(4)TPCA2=tnewT∧−1tnew,SPEPCA=eTe,in which ∧ is the diagonal matrix containing the eigenvalues of the data variance matrix.T2 and SPE control limits are calculated from normal operating data. Since the T2 statistic is in line with the F distribution, given a certain degree of confidence α (usually 99%), the statistical control limits Tα2 and SPEα are as follows:(5)TαT=km2−1mm−kFk,m−k,α,SPEα=θ1ηα2θ2h02θ1+1+θ2h0h0−1θ11/h0,in which the freedom of Fk,m−k,α is m−k; θi is the sum of the residual eigenvalues to the power of 1, 2, and 3; and h0 is the intermediate outcome variable, h0=1−2θ1θ3/3θ22.


## 2.2. Independent Component Analysis

Unlike PCA, ICA can monitor the non-Gaussian processes by extracting independent non-Gaussian features from industrial process variables. The given process data vectorX=x1,x2,…,xm∈R1×m can be represented as a linear combination of statistically independent d non-Gaussian sources s=s1,s2,s3,…,sd, in which d≤m. After normalization and preprocessing, the relationship between the original industrial process data and the independent components ICs is expressed as follows:(6)x=As+e,in which A∈Rm×d is the mixture matrix and e∈Rm×1 is the residual matrix. The basic problem of ICA is to estimate the mixture matrix A and the independent component vector s. Because the FastICA algorithm is characterized in fast calculation speed, this study uses the FastICA method [28] to calculate the decomposition matrix W and reconstruct s^ as follows:(7)s^=Wx.To eliminate the correlation between the process data and facilitate the calculation, normalization and whitening are required. This paper adopts the commonly used PCA whitening method to whiten the process data. Through singular value decomposition, the obtained covariance matrix and whitened data are as follows:(8)C=ExxT=U∧UT,a=∧−1/2UT,z=Qx=QAs,in which E represents the expectation, C is the covariance matrix, Q is the whitening matrix, and zis the whitened data. Orthogonal matrix B can be calculated by the following formula:(9)B=QA.From the above formula,s^ can be expressed as follows:(10)s^=BTz=BTQx.The relationship between the decomposition matrixW and B can be expressed by the following formula:(11)w=BTQ.WhenICs is determined, process monitoring can be performed by establishing I2 and statistical magnitude SPE. The specific process is as follows:(12)I2=s^Ts^,SPE=eTe.The kernel density estimation (KDE) is used to calculate the thresholds of the above two statistics [28].


## 2.3. Kernel Method

KPCA and KICA models map nonlinear process variables into high-dimensional space for processing through kernel methods. This section introduces the kernel method at length since the monitoring schemes of PCA and ICA models have been explicated in Sections 2.1 and 2.2.Assume the process dataX=x1,x2,…,xm∈Rn×m is a normalized 0-mean dataset. Through nonlinear mapping Φ⋅, the covariance matrix CF of the mapped data in the feature space is as follows [28]:(13)CF=1N∑j=1N∅x∅xT.CF can be diagonalized by eigenvalue decomposition as follows:(14)λv=CFv,in which λ represents the eigenvalue and satisfies λ≥0, and v represents the eigenvector. Formulas (13) and (14) suggest the following:(15)CFv=1N∑j=1n∅xj∅xjTv=1N∑∅xj,v∅xj.Since the eigenvectors of the covariance matrix are spanned by samples, there must be a coefficientαi (i=1,2,3,…,n):(16)v=∑j=1nαi∅xi.Formulas (15) and (16) suggest the following:(17)λ∑i=1nαi∅xk,∅xi=1n∑i=1nαi∅xk,∑j=1n∅xi∅xj,∅xi.The inner product of the reconstructed variables in the feature space can be described by a kernel function. This study adopts a Gaussian kernel function, which is defined as follows:(18)kx,y=∅x,∅y=expx−y22×σ2.The kernel matrixk is kij=kij=Φxi,Φxj [28], and (17) can be further simplified as follows:(19)λnkα=k2α,in which α and λ are the eigenvalues and eigenvectors of the kernel matrix k. According to the above formula, λ and v can be calculated.


## 2.4. Jarque–Bera Test

MSPM method usually performs parameter evaluation and feature extraction given that variables obey normal distribution. Although it is reasonable to assume that some industrial processes obey normality, this assumption is often questionable when faced with complex industrial processes. Therefore, the Jarque–Bera (JB) test for normality distribution of complex process data is proposed. In statistics, the Jarque–Bera test examines whether the sample data shows the skewness and kurtosis of a normal distribution and goodness.AssumeX=x1,x2,x3,…,xn to be a dataset of n independent random variables; the sample skewness and kurtosis of x1,x2,x3,…,xn are calculated as follows:(20)b1=1/n∑i=1nxi−x¯31/n∑i=1nxi−x¯23/2,b2=1/n∑i=1nxi−x¯41/n∑i=1nxi−x¯22,in which x¯=1/n∑i=1nxi; if the process variable follows a Gaussian distribution, the skewness and kurtosis of X are close to 0 and 3, respectively. The normality of the sample can be tested by the skewness and kurtosis deviation of the expected value. The formula for calculating the JB statistic is as follows:(21)JB=n6b12+b2−324.The thresholdJBα of the JB statistical magnitude is calculated from the significance level α and the number of samples n. If JB<JBα, this means that the process variable obeys the normality assumption.


## 3. Nonlinear Discriminant Method and Bayesian Inference Fusion Strategy

### 3.1. Nonlinear Discriminant Method

It is of great significance to divide the linearly correlated variables and the nonlinearly correlated variables into two subblocks for separate monitoring since the complex industrial process variables may have both linear and nonlinear correlations. This section proposes a nonlinear discriminant method for this problem, which is described at length as follows.Mutual information (MI), a nonlinear evaluation method based on communication entropy theory, quantitatively describes the nonlinear correlation between two random variables [29, 30]. If there is a strong correlation between these two variables, their MI values will become relatively large. On the contrary, if the two variables are approximately independent, the MI value will become very small. The proposal of mutual information provides great convenience for characterizing the relationship between two variables [30], but the computational complexity of the joint probability in mutual information is relatively large. Therefore, this study selects the maximum mutual information coefficient (MIC) with low computational complexity to indicate the degree of correlation between variables.Although the maximum mutual information coefficient has a strong ability to represent linear and nonlinear correlations, it cannot clearly indicate whether the variables are linear or nonlinear. The Pearson correlation coefficient can just make up for the lack of MIC. It is obtained by dividing the covariance of the two variables by the product of the standard deviation. The Pearson correlation coefficient value is located in [−1, 1]: a value equal to 1 indicates that the two variables are perfectly linearly correlated, a value equal to −1 indicates that the two variables are perfectly linearly negatively correlated, and a value equal to 0 indicates that there is no linear relationship between the two variables. The calculation process of the MIC and Pearson correlation coefficient is as follows:(22)Ix,y=∫px,ylog2px,ypxpydxdy,MICx,y=maxa×b<BIx,ylog2mina,b,Px,y=covx,yvarxvary=Exy−ExEyEx2−E2xEy2−E2y,in which MICx,y represents the maximum mutual information coefficient of the two variables, and Px,y represents the Pearson correlation coefficient between the two variables. E⋅ is the expected value, and var⋅ is the variance value.Although MIC and Pearson correlation coefficients shine in characterizing the correlation between variables, deficiencies are seen in determining whether the variables are linearly or nonlinearly correlated. Therefore, we introduce a nonlinear discriminant coefficient, which combines the strengths of MIC and Pearson’s correlation coefficient to determine whether a variable is nonlinear. Here is the calculation:(23)NLDV=1−Px,yα+1−αMICx,y,in which NLDV represents the value of nonlinear discrimination, which represents the degree of nonlinear correlation of process variables. NLDV∈0,1 can be inferred from (23). If NLDV=1 represents a complete nonlinear correlation between variables and NLDV=0 represents a completely linear relationship between variables, to illustrate other cases of the value of NLDV, a threshold of γ∈0.3,0.5 is set. If NLDV>γ means that the variables are nonlinearly correlated, this paper takes γ=0.4, α represents the weight coefficient, and α=1/2 in this paper [30].


### 3.2. Bayesian Inference Fusion Strategy

Since the process variables are divided into multiple subspaces for separate monitoring, how to integrate the monitoring results becomes a problem. Reference [22] proposed a decision logic to determine the process running state. To improve the robustness of the model, this study adopts a Bayesian inference fusion strategy to determine the final monitoring results of the process. Bayesian inference is based on a probabilistic approach, which is similar to that in recent studies [31, 32]. Jarque–Bera and the nonlinear discriminant method divided the process dataset into Gaussian linear subspace, Gaussian nonlinear subspace, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace. The corresponding monitoring statistics of each subspace are established. The failure probability of T2 in a Gaussian linear subspace is defined as follows:(24)PT2F|XPCA=PT2XPCA|FPT2FPT2XPCA,PT2XPCA=PT2XPCA|NPT2N+PT2XPCA|FPT2F,in which N and F represent normal operating conditions and abnormal operating conditions, respectively. Given a confidence level α, PT2F=α, and PT2N=1−α. The calculation process of PT2XPCA|N and PT2XPCA|F is as follows:(25)PT2XPCA|N=exp−TXPCA2Tlim2,PT2XPCA|F=exp−Tlim2TXPCA2,in which Tlim2 is T2 control limit.Similarly, definition of the failure probability of the remaining three subspaces shares similarities with the calculation method of the Gaussian linear subspace given above. Combination of the statistical magnitude ofT2 and SPE of Gaussian linear subspace and Gaussian nonlinear subspace and that of non-Gaussian linear subspace and non-Gaussian nonlinear subspace is as follows:(26)BICT2=PT22F|XPCA+PT22F|XKPCA+PI22F|XICA+PI22F|XKICAPT2F|XPCA+PT2F|XKPCA+PI2F|XICA+PI2F|XKICA,BICSPE=PSPE2F|XPCA+PSPE2F|XKPCA+PSPE2F|XICA+PSPEF|XKICAPSPEF|XPCA+PSPEF|XKPCA+PSPEF|XICA+PSPEF|XKICA.Following weighted statistic formula is established according to the above formula to facilitate the monitoring:(27)P2=PT22F|XPCA+PT22F|XKPCA+PI22F|XICA+PI22F|XKICA+PSPE2F|XPCA+PSPE2F|XKPCA+PSPE2F|XICA+PSPE2F|XKICA,P=PT2F|XPCA+PT2F|XKPCA+PI2F|XICA+PI2F|XKICA+PSPEF|XPCA+PSPEF|XKPCA+PSPEF|XICA+PSPEF|XKICA,BIC=P2P.If the calculation result of the above formula exceeds the confidence levelα, it indicates possible failures of the industrial process; otherwise, the industrial process is in a normal operating state.


## 3.1. Nonlinear Discriminant Method

It is of great significance to divide the linearly correlated variables and the nonlinearly correlated variables into two subblocks for separate monitoring since the complex industrial process variables may have both linear and nonlinear correlations. This section proposes a nonlinear discriminant method for this problem, which is described at length as follows.Mutual information (MI), a nonlinear evaluation method based on communication entropy theory, quantitatively describes the nonlinear correlation between two random variables [29, 30]. If there is a strong correlation between these two variables, their MI values will become relatively large. On the contrary, if the two variables are approximately independent, the MI value will become very small. The proposal of mutual information provides great convenience for characterizing the relationship between two variables [30], but the computational complexity of the joint probability in mutual information is relatively large. Therefore, this study selects the maximum mutual information coefficient (MIC) with low computational complexity to indicate the degree of correlation between variables.Although the maximum mutual information coefficient has a strong ability to represent linear and nonlinear correlations, it cannot clearly indicate whether the variables are linear or nonlinear. The Pearson correlation coefficient can just make up for the lack of MIC. It is obtained by dividing the covariance of the two variables by the product of the standard deviation. The Pearson correlation coefficient value is located in [−1, 1]: a value equal to 1 indicates that the two variables are perfectly linearly correlated, a value equal to −1 indicates that the two variables are perfectly linearly negatively correlated, and a value equal to 0 indicates that there is no linear relationship between the two variables. The calculation process of the MIC and Pearson correlation coefficient is as follows:(22)Ix,y=∫px,ylog2px,ypxpydxdy,MICx,y=maxa×b<BIx,ylog2mina,b,Px,y=covx,yvarxvary=Exy−ExEyEx2−E2xEy2−E2y,in which MICx,y represents the maximum mutual information coefficient of the two variables, and Px,y represents the Pearson correlation coefficient between the two variables. E⋅ is the expected value, and var⋅ is the variance value.Although MIC and Pearson correlation coefficients shine in characterizing the correlation between variables, deficiencies are seen in determining whether the variables are linearly or nonlinearly correlated. Therefore, we introduce a nonlinear discriminant coefficient, which combines the strengths of MIC and Pearson’s correlation coefficient to determine whether a variable is nonlinear. Here is the calculation:(23)NLDV=1−Px,yα+1−αMICx,y,in which NLDV represents the value of nonlinear discrimination, which represents the degree of nonlinear correlation of process variables. NLDV∈0,1 can be inferred from (23). If NLDV=1 represents a complete nonlinear correlation between variables and NLDV=0 represents a completely linear relationship between variables, to illustrate other cases of the value of NLDV, a threshold of γ∈0.3,0.5 is set. If NLDV>γ means that the variables are nonlinearly correlated, this paper takes γ=0.4, α represents the weight coefficient, and α=1/2 in this paper [30].


## 3.2. Bayesian Inference Fusion Strategy

Since the process variables are divided into multiple subspaces for separate monitoring, how to integrate the monitoring results becomes a problem. Reference [22] proposed a decision logic to determine the process running state. To improve the robustness of the model, this study adopts a Bayesian inference fusion strategy to determine the final monitoring results of the process. Bayesian inference is based on a probabilistic approach, which is similar to that in recent studies [31, 32]. Jarque–Bera and the nonlinear discriminant method divided the process dataset into Gaussian linear subspace, Gaussian nonlinear subspace, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace. The corresponding monitoring statistics of each subspace are established. The failure probability of T2 in a Gaussian linear subspace is defined as follows:(24)PT2F|XPCA=PT2XPCA|FPT2FPT2XPCA,PT2XPCA=PT2XPCA|NPT2N+PT2XPCA|FPT2F,in which N and F represent normal operating conditions and abnormal operating conditions, respectively. Given a confidence level α, PT2F=α, and PT2N=1−α. The calculation process of PT2XPCA|N and PT2XPCA|F is as follows:(25)PT2XPCA|N=exp−TXPCA2Tlim2,PT2XPCA|F=exp−Tlim2TXPCA2,in which Tlim2 is T2 control limit.Similarly, definition of the failure probability of the remaining three subspaces shares similarities with the calculation method of the Gaussian linear subspace given above. Combination of the statistical magnitude ofT2 and SPE of Gaussian linear subspace and Gaussian nonlinear subspace and that of non-Gaussian linear subspace and non-Gaussian nonlinear subspace is as follows:(26)BICT2=PT22F|XPCA+PT22F|XKPCA+PI22F|XICA+PI22F|XKICAPT2F|XPCA+PT2F|XKPCA+PI2F|XICA+PI2F|XKICA,BICSPE=PSPE2F|XPCA+PSPE2F|XKPCA+PSPE2F|XICA+PSPEF|XKICAPSPEF|XPCA+PSPEF|XKPCA+PSPEF|XICA+PSPEF|XKICA.Following weighted statistic formula is established according to the above formula to facilitate the monitoring:(27)P2=PT22F|XPCA+PT22F|XKPCA+PI22F|XICA+PI22F|XKICA+PSPE2F|XPCA+PSPE2F|XKPCA+PSPE2F|XICA+PSPE2F|XKICA,P=PT2F|XPCA+PT2F|XKPCA+PI2F|XICA+PI2F|XKICA+PSPEF|XPCA+PSPEF|XKPCA+PSPEF|XICA+PSPEF|XKICA,BIC=P2P.If the calculation result of the above formula exceeds the confidence levelα, it indicates possible failures of the industrial process; otherwise, the industrial process is in a normal operating state.


## 4. Process Monitoring Program

The flowchart of PCA-KPCA-ICA-KICA-BI is shown in Figure1, and the specific description is as follows.Figure 1
Flowchart for the PCA-KPCA-ICA-KICA-BI monitoring method.


### 4.1. Spatial Decomposition

Step 1 normalized the historical datasetX under normal operating conditions.Step 2 calculated the value of each column of the datasetX.Step 3 divided the datasetX into Gaussian subspace and non-Gaussian subspace by JB value and JBα.Step 4 divided the Gaussian subspace and the non-Gaussian subspace into Gaussian linear subspace, Gaussian nonlinear subspace, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace through the nonlinear discrimination method proposed in Section3.1.


### 4.2. Fault Detection

#### 4.2.1. Offline Model

Step 1 adopted the historical datasetX under normal operating conditions for training; Step 2 performed space decomposition; Step 3 detected the Gaussian linear subspace, Gaussian nonlinear subspace, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace making use of PCA, KPCA, ICA, and KICA models; Step 4 calculated the confidence limits for each subspace.


#### 4.2.2. Online Monitoring

Step 1 collected the currently monitored industrial process system datasetXNew and normalized it.Step 2 dividedXNew into Gaussian linear subspaces, Gaussian nonlinear subspaces, non-Gaussian linear subspaces, and non-Gaussian nonlinear subspaces using spatial decomposition.Step 3 calculated the monitoring statistics of each word space.Step 4 learned the statistics in Step 3 and the confidence limits in Step 4 of the offline model. The final statistic BIC is calculated using the Bayesian inference fusion strategy of (27) in Section 3.2, where BIC>α indicates a fault, and the opposite indicates that no fault has occurred.


## 4.1. Spatial Decomposition

Step 1 normalized the historical datasetX under normal operating conditions.Step 2 calculated the value of each column of the datasetX.Step 3 divided the datasetX into Gaussian subspace and non-Gaussian subspace by JB value and JBα.Step 4 divided the Gaussian subspace and the non-Gaussian subspace into Gaussian linear subspace, Gaussian nonlinear subspace, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace through the nonlinear discrimination method proposed in Section3.1.


## 4.2. Fault Detection

### 4.2.1. Offline Model

Step 1 adopted the historical datasetX under normal operating conditions for training; Step 2 performed space decomposition; Step 3 detected the Gaussian linear subspace, Gaussian nonlinear subspace, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace making use of PCA, KPCA, ICA, and KICA models; Step 4 calculated the confidence limits for each subspace.


### 4.2.2. Online Monitoring

Step 1 collected the currently monitored industrial process system datasetXNew and normalized it.Step 2 dividedXNew into Gaussian linear subspaces, Gaussian nonlinear subspaces, non-Gaussian linear subspaces, and non-Gaussian nonlinear subspaces using spatial decomposition.Step 3 calculated the monitoring statistics of each word space.Step 4 learned the statistics in Step 3 and the confidence limits in Step 4 of the offline model. The final statistic BIC is calculated using the Bayesian inference fusion strategy of (27) in Section 3.2, where BIC>α indicates a fault, and the opposite indicates that no fault has occurred.


## 4.2.1. Offline Model

Step 1 adopted the historical datasetX under normal operating conditions for training; Step 2 performed space decomposition; Step 3 detected the Gaussian linear subspace, Gaussian nonlinear subspace, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace making use of PCA, KPCA, ICA, and KICA models; Step 4 calculated the confidence limits for each subspace.


## 4.2.2. Online Monitoring

Step 1 collected the currently monitored industrial process system datasetXNew and normalized it.Step 2 dividedXNew into Gaussian linear subspaces, Gaussian nonlinear subspaces, non-Gaussian linear subspaces, and non-Gaussian nonlinear subspaces using spatial decomposition.Step 3 calculated the monitoring statistics of each word space.Step 4 learned the statistics in Step 3 and the confidence limits in Step 4 of the offline model. The final statistic BIC is calculated using the Bayesian inference fusion strategy of (27) in Section 3.2, where BIC>α indicates a fault, and the opposite indicates that no fault has occurred.


## 5. Case Study

To verify the effectiveness of the PCA-KPCA-ICA-KICA-BI hybrid model proposed in this study, two cases were adopted to verify the performance of the model.


### 5.1. Tennessee Eastman (TE) Chemical Process

The Tennessee Eastman (TE) chemical process described by Downs and Vogel consists of five main units: reactor, product condenser, vapor-liquid separator, recycle compressor, and product stripper. The reactor includes four reactive substances (A, C, D, and E) and an inert substance (B). After the reactants are fed into the reactor, products G and H and by-product F are generated through a series of chemical reactions. The product stream is condensed in a condenser and then separated by a vapor-liquid separator. The uncondensed product is sent back to the reactor by the centrifugal compressor for re-reaction, and the condensed product is sent to the stripper for stripping [33]. The flowchart of the TE chemical industry is shown in Figure 2. The TE process contains 41 measured variables (22 continuous process variables, 19 component measured variables) and 12 manipulated variables. The dataset adopted for this study was downloaded from https://web.mit.edu/braatzgroup/links.html. Table 1 presents all the variables included in the TE process, and the dataset contains 52 variables and 960 samples. For the convenience of subsequent experimental analysis, Table 2 lists 21 different faults (faults are introduced after the 161st sample). In the following datasets of faults 11, 16, and 19, the PCA-KPCA-ICA-KICA-BI hybrid model proposed in this study is compared with the traditional single model to verify the effectiveness of the hybrid model.Figure 2
Control system of the TE process.Table 1
Process variables in the TE process.
No.Process measurements1A feed2D feed3E feed4Total feed5Recycle flow6Reactor feed rate7Reactor pressure8Reactor level9Reactor temperature10Purge rate11Product separator temperature12Product separator level13Product separator pressure14Product separator underflow15Stripper level16Stripper pressure17Stripper underflow18Stripper temperature19Stripper steam flow20Compressor work21Reactor cooling water outlet temperature22Separator cooling water outlet temperature23CompositionA24CompositionB25CompositionC26CompositionD27CompositionE28CompositionF29CompositionA30CompositionB31CompositionC32CompositionD33CompositionE34CompositionF35CompositionG36CompositionH37CompositionD38CompositionE39CompositionF40CompositionG41CompositionH42D feed flow43C feed flow44A feed flow45Total feed flow valve46Compressor recycle valve47Purge valve48Separator pot liquid flow valve49Stripper liquid product flow valve50Stripper steam valve51Reactor cooling water flow52Condenser cooling water flowTable 2
Faults of the TE process.
No.Process variableType1A/C feed ratio, B composition constantStep2B composition, A/C ratio constantStep3D feed temperatureStep4Reactor cooling water inlet temperatureStep5Condenser cooling water inlet temperatureStep6A feed lossStep7C header pressure loss-reduced availabilityStep8A, B, C feed compositionRandom variation9D feed temperatureRandom variation10C feed temperatureRandom variation11Reactor cooling water inlet temperatureRandom variation12Condenser cooling water inlet temperatureRandom variation13Reaction kineticsSlow drift14Reactor cooling water valveSticking15Condenser cooling water valveSticking16UnknownUnknown17UnknownUnknown18UnknownUnknown19UnknownUnknown20UnknownUnknown21Valve position constantConstant position


#### 5.1.1. Subspace Division

Based on the offline modeling in Section4.2, the Jarque–Bera test divided the variable space into Gaussian subspaces and non-Gaussian subspaces, and the nonlinear discriminant method classified the variable space into Gaussian linear subspaces, Gaussian nonlinear subspaces, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace. The results are shown in Table 3.Table 3
Variable division in each subspace.
SubspaceVariable no.Gaussian linear subspace1, 9, 11, 12, 15, 17, 20, 22, 29, 44, 48, 49, 51, 52Gaussian nonlinear subspace4, 5, 6, 8, 14, 21, 23, 26, 27, 24, 25, 28, 30, 35, 37, 38, 41, 42, 43, 45Non-Gaussian linear subspace7, 13, 16, 18, 19, 50Non-Gaussian nonlinear subspace2, 3, 10, 31, 32, 33, 34, 36, 39, 40, 46, 47


#### 5.1.2. TE Process Fault Detection

Fault 11 is caused by random changes to reactor cooling water inlet temperature. PCA, KPCA, ICA, and KICA models are monitored in four subspaces. Figure3 shows the fault detection results of fault 11 by traditional single PCA, KPCA, and ICA models and the hybrid model proposed in this study. As shown in Figures 3(a)–3(c), traditional single PCA, KPCA, and ICA models can detect fault 11; the SPE and T2 fault detection accuracy of the PCA model are 0.755 and 0.529, respectively; the SPE and T2 fault detection accuracy of the KPCA model are 0.205 and 0.327, respectively; and the SPE andI2 fault detection accuracy of the ICA model are 0.229 and 0.372, respectively. As shown in Figure 3(d), the hybrid model proposed in this study can detect more fault samples than the single models, and the fault detection accuracy is 0.924, far exceeding the number of failure detection samples for a traditional single model.Figure 3
Monitoring results of fault 11 in the TE process: (a) PCA; (b) KPCA; (c) ICA; (d) PCA-KPCA-ICA-KICA-BI.
(a)(b)(c)(d)To further verify the efficient performance of the PCA-KPCA-ICA-KICA hybrid model, the unknown fault 16 was selected for testing. The fault detection results shown in Figure4(d) revealed that the fault detection accuracy of the hybrid model reached a very ideal result, 0.928. Furthermore, tests on a single model were carried out to compare and illustrate the superiority of this hybrid model. Figures 4(a)–4(c) show that the SPE and T2 fault detection accuracy of the PCA model were 0.273 and 0.592, respectively; the SPE and T2 fault detection accuracy of the KPCA model were 0.535 and 0.275, respectively; and the SPE and I2 fault detection accuracy of the ICA model were 0.348 and 0.802, respectively.Figure 4
Monitoring results of fault 16 in the TE process: (a) PCA; (b) KPCA; (c) ICA; (d) PCA-KPCA-ICA-KICA-BI.
(a)(b)(c)(d)The detection results of fault 11 and 16 in Figures3 and 4 show that TE process variables have linear, nonlinear, Gaussian, and non-Gaussian distribution characteristics. The traditional single model only considers one distribution case and can only detect TE process faults caused by linearly distributed process variables, nonlinearly distributed process variables, or non-Gaussian distributed process variables, so the fault detection accuracy is relatively low. The hybrid model proposed in this study fully considers the multi-characteristic distribution of complex industrial process variables and divides the process variables into multiple subspaces using the JB test and the nonlinear evaluation method proposed in this study, and then a model with targeted distribution is used for each subspace for fault detection. Finally, the fault detection results are given using a Bayesian inference fusion strategy, and the results show that the proposed hybrid PCA-KPCA-ICA-KICA-BI model has higher fault detection accuracy for multifeatured complex industrial process fault detection.The method proposed in this study considers the advantages of a single model and gives full play to the fault detection advantages of the single model using the JB test and nonlinear evaluation methods. Table4 gives the TE process 21 faults for the traditional PCA model, KPCA model, ICA model, PCA-ICA hybrid model, and PCA-KPCA-ICA-KICA-BI hybrid model proposed in this study. The proposed hybrid model and the existing PCA-ICA hybrid model have higher fault detection accuracy than the existing single PCA, KPCA, and ICA models, but the PCA-ICA hybrid model does not consider the distribution characteristics of the nonlinear process variables, and the decision logic of the hybrid model is insufficient, resulting in lower fault detection. The accuracy of the existing hybrid model is lower than that of the hybrid model proposed in this study. The existing single fault detection model, the PCA-ICA hybrid model, and the hybrid model proposed in this study have a fault detection accuracy close to 1 for faults 1, 2, 6, 7, 12, 14, 17, and 18 of the TE process; however, the fault detection accuracy of faults 3, 9, and 15 is not very satisfactory. This confirms that the magnitude and obviousness of faults can lead to a relatively large difference in fault detection accuracy. Therefore, to address the problem of small fault magnitude not being easily detected, due to the ability of exponentially weighted moving average (EWMA) control scheme to detect small changes, a strategy that combines the exponentially weighted sliding average control scheme with the PCA model is proposed to detect small faults in industrial processes [34]; how to detect initial faults or faults with a low signal-to-noise ratio will be a very interesting problem. Figures 3 and 4 show that a single model is inferior to the hybrid model in fault detection performance despite its ability to detect certain fault samples, indicating its deficiency, that is, only considering one distribution of variables. Simply put, the hybrid model shines in dealing with the increasingly complex industrial processes.Table 4
Detection rates of 21 faults.
Fault no.PCAKPCAICAPCA-ICAProposedT2SPET2SPEI2SPETI2csSPEcsBIC10.9920.9910.9970.9930.9970.9970.9940.9920.99520.9850.9790.9850.9850.9860.9860.9860.9780.98830.0090.0070.0270.0700.1620.1760.170.2120.33640.4930.9410.9750.9770.7750.8730.4010.991150.2750.2940.3780.496110.3520.4250.72460.99210.9981110.9920.9981711111111180.9760.9760.9830.9840.9690.9740.9770.9740.97890.1370.1820.1870.2250.1120.1320.1750.20.282100.3340.3410.5010.6970.7490.7620.5940.7020.895110.5920.7960.3270.2050.3720.2290.4030.7720.924120.9750.9820.9830.9820.9970.9970.9920.9650.995130.9440.9520.9590.9590.9540.9550.9530.950.9621410.9880.9990.99610.9990.9730.9971150.1710.2470.2280.2890.0410.1520.1740.2240.302160.5920.2730.2750.5350.8020.3480.6820.7340.928170.7520.8930.9370.9420.8620.8940.7850.9520.979180.8960.9140.9130.9170.8930.8960.910.9180.930190.1750.2930.1790.3940.5980.3740.3970.6540.984200.3350.5980.5470.6350.7580.7330.7360.7420.872210.4820.5990.4890.5620.6950.6470.6820.7040.825TE process failure simulation experiments demonstrate the considerable advantages of the hybrid model proposed in this study. To further validate the performance of the model, the continuous stirred reactor process is used in Section5.2.


### 5.2. CSTR Case

Figure5 shows the simple flowchart of the CSTR process consisting of 9 variables. Here is the vector form of the dataset:(28)X=x1x2x3x4x5x6x7x8x9T=TCT0CAACASFSFCCATFAT,where TC denotes the cooling water temperature, T0 denotes the inlet temperature, CAA and CAS denote the inlet concentrations, FS denotes the solvent flow, FC denotes the cooling water flow, CA denotes the outlet concentration, T denotes the temperature, and FA denotes the reactant flow [22, 33].Figure 5
Simplified schematic of the CSTR [22, 33].In this study, using the same simulation conditions as Yoon and Macgregor [33], 960 normal operating CSTR data samples were collected through the established CSTR process simulation platform as the training set, and another 160 normal samples and 800 fault samples were collected as the test set. The test set was used to verify the fault detection performance of the hybrid model proposed in this study.


#### 5.2.1. Subspace Division

In view of the offline modeling in Section4.2, first, Jarque–Bera test divided the CSTR process variable space into Gaussian subspaces and non-Gaussian subspaces, and the nonlinear discriminant method classified the variable space into Gaussian linear subspaces, Gaussian nonlinear subspaces, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace, as shown in Table 5.Table 5
CSTR process variable division in each subspace.
SubspaceVariable no.Gaussian linear subspace1, 2, 8Gaussian nonlinear subspace3, 4, 5, 6, 7Non-Gaussian linear subspace9Non-Gaussian nonlinear subspace


#### 5.2.2. CSTR Process Fault Detection

Figure6 shows the fault detection results of the CSTR process. Figures 6(a)–6(d) show that the hybrid model proposed in this paper integrates the distribution characteristics of the process variables and uses the BI method to make a comprehensive decision on the fault detection results with significant advantages, where the fault detection rate of the PCA-KPCA-ICA-KICA-BI hybrid model is 1 and is lower than the FPR of the single model. Since the CSTR process is a typical nonlinear process [22, 33], the PCA model and the ICA model cannot handle the nonlinear distributed characteristic variables well, resulting in lower fault detection rates, with 0.83 for the PCA model and 0.08 for the ICA model. KPCA can map the nonlinear data to a high-dimensional space to become linear, so the KPCA model has good applicability in nonlinear process fault detection, which is verified in Figure 6(b). Comparison of Figures 6(b) and 6(d) shows that the fault detection rate of the KPCA model is comparable to that of the hybrid model proposed in this paper, which indicates that full consideration of the distribution of process variables and the use of a targeted fault detection model is helpful in improving the fault detection accuracy. The fault detection results of the CSTR process show that the hybrid PCA-KPCA-ICA-KICA-BI model can handle complex multi-characteristic process variables well and improve the fault detection accuracy of complex industrial processes.Figure 6
Monitoring results of faults in the CSTR process: (a) PCA; (b) KPCA; (c) ICA; (d) PCA-KPCA-ICA-KICA-BI.
(a)(b)(c)(d)


## 5.1. Tennessee Eastman (TE) Chemical Process

The Tennessee Eastman (TE) chemical process described by Downs and Vogel consists of five main units: reactor, product condenser, vapor-liquid separator, recycle compressor, and product stripper. The reactor includes four reactive substances (A, C, D, and E) and an inert substance (B). After the reactants are fed into the reactor, products G and H and by-product F are generated through a series of chemical reactions. The product stream is condensed in a condenser and then separated by a vapor-liquid separator. The uncondensed product is sent back to the reactor by the centrifugal compressor for re-reaction, and the condensed product is sent to the stripper for stripping [33]. The flowchart of the TE chemical industry is shown in Figure 2. The TE process contains 41 measured variables (22 continuous process variables, 19 component measured variables) and 12 manipulated variables. The dataset adopted for this study was downloaded from https://web.mit.edu/braatzgroup/links.html. Table 1 presents all the variables included in the TE process, and the dataset contains 52 variables and 960 samples. For the convenience of subsequent experimental analysis, Table 2 lists 21 different faults (faults are introduced after the 161st sample). In the following datasets of faults 11, 16, and 19, the PCA-KPCA-ICA-KICA-BI hybrid model proposed in this study is compared with the traditional single model to verify the effectiveness of the hybrid model.Figure 2
Control system of the TE process.Table 1
Process variables in the TE process.
No.Process measurements1A feed2D feed3E feed4Total feed5Recycle flow6Reactor feed rate7Reactor pressure8Reactor level9Reactor temperature10Purge rate11Product separator temperature12Product separator level13Product separator pressure14Product separator underflow15Stripper level16Stripper pressure17Stripper underflow18Stripper temperature19Stripper steam flow20Compressor work21Reactor cooling water outlet temperature22Separator cooling water outlet temperature23CompositionA24CompositionB25CompositionC26CompositionD27CompositionE28CompositionF29CompositionA30CompositionB31CompositionC32CompositionD33CompositionE34CompositionF35CompositionG36CompositionH37CompositionD38CompositionE39CompositionF40CompositionG41CompositionH42D feed flow43C feed flow44A feed flow45Total feed flow valve46Compressor recycle valve47Purge valve48Separator pot liquid flow valve49Stripper liquid product flow valve50Stripper steam valve51Reactor cooling water flow52Condenser cooling water flowTable 2
Faults of the TE process.
No.Process variableType1A/C feed ratio, B composition constantStep2B composition, A/C ratio constantStep3D feed temperatureStep4Reactor cooling water inlet temperatureStep5Condenser cooling water inlet temperatureStep6A feed lossStep7C header pressure loss-reduced availabilityStep8A, B, C feed compositionRandom variation9D feed temperatureRandom variation10C feed temperatureRandom variation11Reactor cooling water inlet temperatureRandom variation12Condenser cooling water inlet temperatureRandom variation13Reaction kineticsSlow drift14Reactor cooling water valveSticking15Condenser cooling water valveSticking16UnknownUnknown17UnknownUnknown18UnknownUnknown19UnknownUnknown20UnknownUnknown21Valve position constantConstant position


### 5.1.1. Subspace Division

Based on the offline modeling in Section4.2, the Jarque–Bera test divided the variable space into Gaussian subspaces and non-Gaussian subspaces, and the nonlinear discriminant method classified the variable space into Gaussian linear subspaces, Gaussian nonlinear subspaces, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace. The results are shown in Table 3.Table 3
Variable division in each subspace.
SubspaceVariable no.Gaussian linear subspace1, 9, 11, 12, 15, 17, 20, 22, 29, 44, 48, 49, 51, 52Gaussian nonlinear subspace4, 5, 6, 8, 14, 21, 23, 26, 27, 24, 25, 28, 30, 35, 37, 38, 41, 42, 43, 45Non-Gaussian linear subspace7, 13, 16, 18, 19, 50Non-Gaussian nonlinear subspace2, 3, 10, 31, 32, 33, 34, 36, 39, 40, 46, 47


### 5.1.2. TE Process Fault Detection

Fault 11 is caused by random changes to reactor cooling water inlet temperature. PCA, KPCA, ICA, and KICA models are monitored in four subspaces. Figure3 shows the fault detection results of fault 11 by traditional single PCA, KPCA, and ICA models and the hybrid model proposed in this study. As shown in Figures 3(a)–3(c), traditional single PCA, KPCA, and ICA models can detect fault 11; the SPE and T2 fault detection accuracy of the PCA model are 0.755 and 0.529, respectively; the SPE and T2 fault detection accuracy of the KPCA model are 0.205 and 0.327, respectively; and the SPE andI2 fault detection accuracy of the ICA model are 0.229 and 0.372, respectively. As shown in Figure 3(d), the hybrid model proposed in this study can detect more fault samples than the single models, and the fault detection accuracy is 0.924, far exceeding the number of failure detection samples for a traditional single model.Figure 3
Monitoring results of fault 11 in the TE process: (a) PCA; (b) KPCA; (c) ICA; (d) PCA-KPCA-ICA-KICA-BI.
(a)(b)(c)(d)To further verify the efficient performance of the PCA-KPCA-ICA-KICA hybrid model, the unknown fault 16 was selected for testing. The fault detection results shown in Figure4(d) revealed that the fault detection accuracy of the hybrid model reached a very ideal result, 0.928. Furthermore, tests on a single model were carried out to compare and illustrate the superiority of this hybrid model. Figures 4(a)–4(c) show that the SPE and T2 fault detection accuracy of the PCA model were 0.273 and 0.592, respectively; the SPE and T2 fault detection accuracy of the KPCA model were 0.535 and 0.275, respectively; and the SPE and I2 fault detection accuracy of the ICA model were 0.348 and 0.802, respectively.Figure 4
Monitoring results of fault 16 in the TE process: (a) PCA; (b) KPCA; (c) ICA; (d) PCA-KPCA-ICA-KICA-BI.
(a)(b)(c)(d)The detection results of fault 11 and 16 in Figures3 and 4 show that TE process variables have linear, nonlinear, Gaussian, and non-Gaussian distribution characteristics. The traditional single model only considers one distribution case and can only detect TE process faults caused by linearly distributed process variables, nonlinearly distributed process variables, or non-Gaussian distributed process variables, so the fault detection accuracy is relatively low. The hybrid model proposed in this study fully considers the multi-characteristic distribution of complex industrial process variables and divides the process variables into multiple subspaces using the JB test and the nonlinear evaluation method proposed in this study, and then a model with targeted distribution is used for each subspace for fault detection. Finally, the fault detection results are given using a Bayesian inference fusion strategy, and the results show that the proposed hybrid PCA-KPCA-ICA-KICA-BI model has higher fault detection accuracy for multifeatured complex industrial process fault detection.The method proposed in this study considers the advantages of a single model and gives full play to the fault detection advantages of the single model using the JB test and nonlinear evaluation methods. Table4 gives the TE process 21 faults for the traditional PCA model, KPCA model, ICA model, PCA-ICA hybrid model, and PCA-KPCA-ICA-KICA-BI hybrid model proposed in this study. The proposed hybrid model and the existing PCA-ICA hybrid model have higher fault detection accuracy than the existing single PCA, KPCA, and ICA models, but the PCA-ICA hybrid model does not consider the distribution characteristics of the nonlinear process variables, and the decision logic of the hybrid model is insufficient, resulting in lower fault detection. The accuracy of the existing hybrid model is lower than that of the hybrid model proposed in this study. The existing single fault detection model, the PCA-ICA hybrid model, and the hybrid model proposed in this study have a fault detection accuracy close to 1 for faults 1, 2, 6, 7, 12, 14, 17, and 18 of the TE process; however, the fault detection accuracy of faults 3, 9, and 15 is not very satisfactory. This confirms that the magnitude and obviousness of faults can lead to a relatively large difference in fault detection accuracy. Therefore, to address the problem of small fault magnitude not being easily detected, due to the ability of exponentially weighted moving average (EWMA) control scheme to detect small changes, a strategy that combines the exponentially weighted sliding average control scheme with the PCA model is proposed to detect small faults in industrial processes [34]; how to detect initial faults or faults with a low signal-to-noise ratio will be a very interesting problem. Figures 3 and 4 show that a single model is inferior to the hybrid model in fault detection performance despite its ability to detect certain fault samples, indicating its deficiency, that is, only considering one distribution of variables. Simply put, the hybrid model shines in dealing with the increasingly complex industrial processes.Table 4
Detection rates of 21 faults.
Fault no.PCAKPCAICAPCA-ICAProposedT2SPET2SPEI2SPETI2csSPEcsBIC10.9920.9910.9970.9930.9970.9970.9940.9920.99520.9850.9790.9850.9850.9860.9860.9860.9780.98830.0090.0070.0270.0700.1620.1760.170.2120.33640.4930.9410.9750.9770.7750.8730.4010.991150.2750.2940.3780.496110.3520.4250.72460.99210.9981110.9920.9981711111111180.9760.9760.9830.9840.9690.9740.9770.9740.97890.1370.1820.1870.2250.1120.1320.1750.20.282100.3340.3410.5010.6970.7490.7620.5940.7020.895110.5920.7960.3270.2050.3720.2290.4030.7720.924120.9750.9820.9830.9820.9970.9970.9920.9650.995130.9440.9520.9590.9590.9540.9550.9530.950.9621410.9880.9990.99610.9990.9730.9971150.1710.2470.2280.2890.0410.1520.1740.2240.302160.5920.2730.2750.5350.8020.3480.6820.7340.928170.7520.8930.9370.9420.8620.8940.7850.9520.979180.8960.9140.9130.9170.8930.8960.910.9180.930190.1750.2930.1790.3940.5980.3740.3970.6540.984200.3350.5980.5470.6350.7580.7330.7360.7420.872210.4820.5990.4890.5620.6950.6470.6820.7040.825TE process failure simulation experiments demonstrate the considerable advantages of the hybrid model proposed in this study. To further validate the performance of the model, the continuous stirred reactor process is used in Section5.2.


## 5.1.1. Subspace Division

Based on the offline modeling in Section4.2, the Jarque–Bera test divided the variable space into Gaussian subspaces and non-Gaussian subspaces, and the nonlinear discriminant method classified the variable space into Gaussian linear subspaces, Gaussian nonlinear subspaces, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace. The results are shown in Table 3.Table 3
Variable division in each subspace.
SubspaceVariable no.Gaussian linear subspace1, 9, 11, 12, 15, 17, 20, 22, 29, 44, 48, 49, 51, 52Gaussian nonlinear subspace4, 5, 6, 8, 14, 21, 23, 26, 27, 24, 25, 28, 30, 35, 37, 38, 41, 42, 43, 45Non-Gaussian linear subspace7, 13, 16, 18, 19, 50Non-Gaussian nonlinear subspace2, 3, 10, 31, 32, 33, 34, 36, 39, 40, 46, 47


## 5.1.2. TE Process Fault Detection

Fault 11 is caused by random changes to reactor cooling water inlet temperature. PCA, KPCA, ICA, and KICA models are monitored in four subspaces. Figure3 shows the fault detection results of fault 11 by traditional single PCA, KPCA, and ICA models and the hybrid model proposed in this study. As shown in Figures 3(a)–3(c), traditional single PCA, KPCA, and ICA models can detect fault 11; the SPE and T2 fault detection accuracy of the PCA model are 0.755 and 0.529, respectively; the SPE and T2 fault detection accuracy of the KPCA model are 0.205 and 0.327, respectively; and the SPE andI2 fault detection accuracy of the ICA model are 0.229 and 0.372, respectively. As shown in Figure 3(d), the hybrid model proposed in this study can detect more fault samples than the single models, and the fault detection accuracy is 0.924, far exceeding the number of failure detection samples for a traditional single model.Figure 3
Monitoring results of fault 11 in the TE process: (a) PCA; (b) KPCA; (c) ICA; (d) PCA-KPCA-ICA-KICA-BI.
(a)(b)(c)(d)To further verify the efficient performance of the PCA-KPCA-ICA-KICA hybrid model, the unknown fault 16 was selected for testing. The fault detection results shown in Figure4(d) revealed that the fault detection accuracy of the hybrid model reached a very ideal result, 0.928. Furthermore, tests on a single model were carried out to compare and illustrate the superiority of this hybrid model. Figures 4(a)–4(c) show that the SPE and T2 fault detection accuracy of the PCA model were 0.273 and 0.592, respectively; the SPE and T2 fault detection accuracy of the KPCA model were 0.535 and 0.275, respectively; and the SPE and I2 fault detection accuracy of the ICA model were 0.348 and 0.802, respectively.Figure 4
Monitoring results of fault 16 in the TE process: (a) PCA; (b) KPCA; (c) ICA; (d) PCA-KPCA-ICA-KICA-BI.
(a)(b)(c)(d)The detection results of fault 11 and 16 in Figures3 and 4 show that TE process variables have linear, nonlinear, Gaussian, and non-Gaussian distribution characteristics. The traditional single model only considers one distribution case and can only detect TE process faults caused by linearly distributed process variables, nonlinearly distributed process variables, or non-Gaussian distributed process variables, so the fault detection accuracy is relatively low. The hybrid model proposed in this study fully considers the multi-characteristic distribution of complex industrial process variables and divides the process variables into multiple subspaces using the JB test and the nonlinear evaluation method proposed in this study, and then a model with targeted distribution is used for each subspace for fault detection. Finally, the fault detection results are given using a Bayesian inference fusion strategy, and the results show that the proposed hybrid PCA-KPCA-ICA-KICA-BI model has higher fault detection accuracy for multifeatured complex industrial process fault detection.The method proposed in this study considers the advantages of a single model and gives full play to the fault detection advantages of the single model using the JB test and nonlinear evaluation methods. Table4 gives the TE process 21 faults for the traditional PCA model, KPCA model, ICA model, PCA-ICA hybrid model, and PCA-KPCA-ICA-KICA-BI hybrid model proposed in this study. The proposed hybrid model and the existing PCA-ICA hybrid model have higher fault detection accuracy than the existing single PCA, KPCA, and ICA models, but the PCA-ICA hybrid model does not consider the distribution characteristics of the nonlinear process variables, and the decision logic of the hybrid model is insufficient, resulting in lower fault detection. The accuracy of the existing hybrid model is lower than that of the hybrid model proposed in this study. The existing single fault detection model, the PCA-ICA hybrid model, and the hybrid model proposed in this study have a fault detection accuracy close to 1 for faults 1, 2, 6, 7, 12, 14, 17, and 18 of the TE process; however, the fault detection accuracy of faults 3, 9, and 15 is not very satisfactory. This confirms that the magnitude and obviousness of faults can lead to a relatively large difference in fault detection accuracy. Therefore, to address the problem of small fault magnitude not being easily detected, due to the ability of exponentially weighted moving average (EWMA) control scheme to detect small changes, a strategy that combines the exponentially weighted sliding average control scheme with the PCA model is proposed to detect small faults in industrial processes [34]; how to detect initial faults or faults with a low signal-to-noise ratio will be a very interesting problem. Figures 3 and 4 show that a single model is inferior to the hybrid model in fault detection performance despite its ability to detect certain fault samples, indicating its deficiency, that is, only considering one distribution of variables. Simply put, the hybrid model shines in dealing with the increasingly complex industrial processes.Table 4
Detection rates of 21 faults.
Fault no.PCAKPCAICAPCA-ICAProposedT2SPET2SPEI2SPETI2csSPEcsBIC10.9920.9910.9970.9930.9970.9970.9940.9920.99520.9850.9790.9850.9850.9860.9860.9860.9780.98830.0090.0070.0270.0700.1620.1760.170.2120.33640.4930.9410.9750.9770.7750.8730.4010.991150.2750.2940.3780.496110.3520.4250.72460.99210.9981110.9920.9981711111111180.9760.9760.9830.9840.9690.9740.9770.9740.97890.1370.1820.1870.2250.1120.1320.1750.20.282100.3340.3410.5010.6970.7490.7620.5940.7020.895110.5920.7960.3270.2050.3720.2290.4030.7720.924120.9750.9820.9830.9820.9970.9970.9920.9650.995130.9440.9520.9590.9590.9540.9550.9530.950.9621410.9880.9990.99610.9990.9730.9971150.1710.2470.2280.2890.0410.1520.1740.2240.302160.5920.2730.2750.5350.8020.3480.6820.7340.928170.7520.8930.9370.9420.8620.8940.7850.9520.979180.8960.9140.9130.9170.8930.8960.910.9180.930190.1750.2930.1790.3940.5980.3740.3970.6540.984200.3350.5980.5470.6350.7580.7330.7360.7420.872210.4820.5990.4890.5620.6950.6470.6820.7040.825TE process failure simulation experiments demonstrate the considerable advantages of the hybrid model proposed in this study. To further validate the performance of the model, the continuous stirred reactor process is used in Section5.2.


## 5.2. CSTR Case

Figure5 shows the simple flowchart of the CSTR process consisting of 9 variables. Here is the vector form of the dataset:(28)X=x1x2x3x4x5x6x7x8x9T=TCT0CAACASFSFCCATFAT,where TC denotes the cooling water temperature, T0 denotes the inlet temperature, CAA and CAS denote the inlet concentrations, FS denotes the solvent flow, FC denotes the cooling water flow, CA denotes the outlet concentration, T denotes the temperature, and FA denotes the reactant flow [22, 33].Figure 5
Simplified schematic of the CSTR [22, 33].In this study, using the same simulation conditions as Yoon and Macgregor [33], 960 normal operating CSTR data samples were collected through the established CSTR process simulation platform as the training set, and another 160 normal samples and 800 fault samples were collected as the test set. The test set was used to verify the fault detection performance of the hybrid model proposed in this study.


### 5.2.1. Subspace Division

In view of the offline modeling in Section4.2, first, Jarque–Bera test divided the CSTR process variable space into Gaussian subspaces and non-Gaussian subspaces, and the nonlinear discriminant method classified the variable space into Gaussian linear subspaces, Gaussian nonlinear subspaces, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace, as shown in Table 5.Table 5
CSTR process variable division in each subspace.
SubspaceVariable no.Gaussian linear subspace1, 2, 8Gaussian nonlinear subspace3, 4, 5, 6, 7Non-Gaussian linear subspace9Non-Gaussian nonlinear subspace


### 5.2.2. CSTR Process Fault Detection

Figure6 shows the fault detection results of the CSTR process. Figures 6(a)–6(d) show that the hybrid model proposed in this paper integrates the distribution characteristics of the process variables and uses the BI method to make a comprehensive decision on the fault detection results with significant advantages, where the fault detection rate of the PCA-KPCA-ICA-KICA-BI hybrid model is 1 and is lower than the FPR of the single model. Since the CSTR process is a typical nonlinear process [22, 33], the PCA model and the ICA model cannot handle the nonlinear distributed characteristic variables well, resulting in lower fault detection rates, with 0.83 for the PCA model and 0.08 for the ICA model. KPCA can map the nonlinear data to a high-dimensional space to become linear, so the KPCA model has good applicability in nonlinear process fault detection, which is verified in Figure 6(b). Comparison of Figures 6(b) and 6(d) shows that the fault detection rate of the KPCA model is comparable to that of the hybrid model proposed in this paper, which indicates that full consideration of the distribution of process variables and the use of a targeted fault detection model is helpful in improving the fault detection accuracy. The fault detection results of the CSTR process show that the hybrid PCA-KPCA-ICA-KICA-BI model can handle complex multi-characteristic process variables well and improve the fault detection accuracy of complex industrial processes.Figure 6
Monitoring results of faults in the CSTR process: (a) PCA; (b) KPCA; (c) ICA; (d) PCA-KPCA-ICA-KICA-BI.
(a)(b)(c)(d)


## 5.2.1. Subspace Division

In view of the offline modeling in Section4.2, first, Jarque–Bera test divided the CSTR process variable space into Gaussian subspaces and non-Gaussian subspaces, and the nonlinear discriminant method classified the variable space into Gaussian linear subspaces, Gaussian nonlinear subspaces, non-Gaussian linear subspace, and non-Gaussian nonlinear subspace, as shown in Table 5.Table 5
CSTR process variable division in each subspace.
SubspaceVariable no.Gaussian linear subspace1, 2, 8Gaussian nonlinear subspace3, 4, 5, 6, 7Non-Gaussian linear subspace9Non-Gaussian nonlinear subspace


## 5.2.2. CSTR Process Fault Detection

Figure6 shows the fault detection results of the CSTR process. Figures 6(a)–6(d) show that the hybrid model proposed in this paper integrates the distribution characteristics of the process variables and uses the BI method to make a comprehensive decision on the fault detection results with significant advantages, where the fault detection rate of the PCA-KPCA-ICA-KICA-BI hybrid model is 1 and is lower than the FPR of the single model. Since the CSTR process is a typical nonlinear process [22, 33], the PCA model and the ICA model cannot handle the nonlinear distributed characteristic variables well, resulting in lower fault detection rates, with 0.83 for the PCA model and 0.08 for the ICA model. KPCA can map the nonlinear data to a high-dimensional space to become linear, so the KPCA model has good applicability in nonlinear process fault detection, which is verified in Figure 6(b). Comparison of Figures 6(b) and 6(d) shows that the fault detection rate of the KPCA model is comparable to that of the hybrid model proposed in this paper, which indicates that full consideration of the distribution of process variables and the use of a targeted fault detection model is helpful in improving the fault detection accuracy. The fault detection results of the CSTR process show that the hybrid PCA-KPCA-ICA-KICA-BI model can handle complex multi-characteristic process variables well and improve the fault detection accuracy of complex industrial processes.Figure 6
Monitoring results of faults in the CSTR process: (a) PCA; (b) KPCA; (c) ICA; (d) PCA-KPCA-ICA-KICA-BI.
(a)(b)(c)(d)


## 6. Conclusion

A hybrid fault detection method based on principal component analysis, kernel principal component analysis, independent component analysis, kernel independent component analysis, and Bayesian inference (PCA-KPCA-ICA-KICA-BI) is proposed for complex industrial processes. First, the Jarque–Bera test and the nonlinear discriminant method divided the feature variables into Gaussian linear blocks, Gaussian nonlinear blocks, non-Gaussian linear blocks, and non-Gaussian nonlinear blocks. Each division was monitored by the PCA-KPCA-ICA-KICA-BI model, and the corresponding results were given comprehensively by the Bayesian inference. Finally, the efficient performance is verified by the Tennessee Eastman (TE) and Continuous Stirred Tank Reactor (CSTR) processes. The traditional single model has a fault detection accuracy of 0.68, and the hybrid PCA-ICA model has a detection accuracy of 0.72, which means that more faults are considered normal, while the hybrid model proposed in this paper has a fault detection accuracy of 0.86, which is a significant improvement.Complex industrial processes are characterized by strong coupling of variables, and any small fault can cause huge safety hazards, which makes it difficult to collect data on actual industrial process faults. Therefore, this study used TE and CSTR processes with complex characteristics, such as high-dimensional, linear, nonlinear, Gaussian, and non-Gaussian features, to verify the fault detection performance of the hybrid model proposed in this paper. The analysis of the fault detection results shows that the PCA-KPCA-ICA-KICA-BI hybrid model can fully consider the variable distribution characteristics of TE and CSTR processes and has a higher fault detection accuracy than the traditional single models and the PCA-ICA hybrid model. The application of the hybrid model proposed in this study to the TE process and the CSTR process shows that the hybrid model has good prospects for securing complex industrial processes. To further optimize and improve the PCA-KPCA-ICA-KICA-BI hybrid model and to enhance the applicability of the model, further research directions could be suggested in the following areas:(1)
Initial failures with insignificant changes in complex industrial process data could be detected.(2)
Compound failures could be detected.(3)
Most current fault detection models require a large number of data samples. However, in the pursuit of a safe and sound industrial process, intriguing is how to realize fault detection based on a small number of samples.



---

*Source: 3054860-2022-08-18.xml*