# Fault Diagnosis of Complex Industrial Process Using KICA and Sparse SVM

Authors: Jie Xu; Jin Zhao; Baoping Ma; Shousong Hu
Journal: Mathematical Problems in Engineering
 (2013)

---

## Abstract

New approaches are proposed for complex industrial process monitoring and fault diagnosis based on kernel independent component analysis (KICA) and sparse support vector machine (SVM). The KICA method is a two-phase algorithm: whitened kernel principal component analysis (KPCA). The data are firstly mapped into high-dimensional feature subspace. Then, the ICA algorithm seeks the projection directions in the KPCA whitened space. Performance monitoring is implemented through constructing the statistical index and control limit in the feature space. If the statistical indexes exceed the predefined control limit, a fault may have occurred. Then, the nonlinear score vectors are calculated and fed into the sparse SVM to identify the faults. The proposed method is applied to the simulation of Tennessee Eastman (TE) chemical process. The simulation results show that the proposed method can identify various types of faults accurately and rapidly.


---

## Body

## 1. Introduction

The demand for safe operation in the industry has propelled research into fault detection and diagnosis methods. Based on principal component analysis (PCA), independent component analysis (ICA), partial least squares (PLSs), and canonical correlation analysis (CVA), the statistical method can represent high-dimensional process data in a reduced dimension, which contains most of the variance of the original data. These methods have been used in various applications [1–4].ICA is a statistical approach that has the potential ability for blind source separation (BSS) without the prior information about the mixtures under the source signals that are statistically independent [5]. However, ICA fails to separate the nonlinearly mixed source due to its intrinsic linearity. One new nonlinear ICA technique for tackling the nonlinear problem has been in development, which is called KICA [6]. KICA can efficiently compute ICs in high-dimensional feature spaces using the kernel matrix K. However, KICA algorithm is difficult to diagnose faults since the mapping function is unknown.Sparse SVM is a learning strategy based on the Newton method and conjugate gradient method. It is proposed in this paper to diagnose faults, which is able to accelerate not only the model selection process but also the training speed.This paper is organized as follows. In Section2, the kernel ICA algorithm is introduced. In Section 3, sparse SVM is proposed. The experimental results of this methods proposed in this paper are also presented in Section 4. Finally, our work of this paper is summarized in the last section.


## 2. KICA Algorithm

Letx1,x2,…xm∈Rm be the n training samples for kernel PCA learning. By the nonlinear mapping Φ:x∈Rm→z∈Rn, the measure inputs are extended into the hyperdimensional feature space. The mapping of  xi is simply noted as Φ(xi). The sample covariance in the feature space is given by
(1)SF=1N∑i=1NΦ(xi)Φ(xi)T.LetQ=[Φ(x1),…,Φ(xn)], then SF can be expressed by SF=(1/N)QQT. Let us form the Gram matrix K=QQT. The matrix is an N×N matrix, and its elements are as follows:
(2)[K]ij=Kij=〈Φ(xi)·Φ(xj)〉=K(xi,xj).We can center the Gram matrixK by
(3)K~=K-ENK-KEN+ENKEN,EN=1N[1⋯1⋮⋮1⋯1]∈RN×N.NormalizedK~ using the following formula:
(4)K^=K~trace(K~)/N.Calculate the eigenvectorsα1≥α2≥⋯≥αd of K corresponding to d largest positive eigenvalues λ1≥λ2≥⋯≥λd. Then, the d largest positive eigenvalues of SF are λ1/N,λ2/N,…,λd/N, and the associated orthonormal eigenvectors v1,v2,…,vd can be expressed by
(5)vi=1λiQαi(i=1,2,…,d).
Denote H=[α1,α2,…,αd], Λ=diag(λ1,λ2,…,λd) and its matrix form is V=QHΛ-1/2.The mapped data in feature space can be whitened by the following transformation:(6)y=GTΦ(x).
Specifically
(7)y=GTΦ(x)=NΛ-1HTQTΦ(x)=NΛ-1HTQT[Φ(x1),…,Φ(xN)]TΦ(x)=NΛ-1HT[K~(x1,x),…,K~(xN,x)]T=NΛ-1HTK~,
where  H=[α1,α2,…,αd], Λ=[λ1,λ2,…,λd].Centering and whitening are carried out in the KPCA feature space. We designate the data after preprocessing asXnormal, W, as well as Snormal are obtained from the fast ICA algorithm in the feature space. Data dimension can be reduced by selecting a few rows of W based on the assumption that the rows with the largest sun of squares coefficient have the largest effect on the variation of S. The detailed algorithm for fast ICA can be easily found in the literatures [7]. The selected a row of W constitutes a matrix Wd (dominant part of W), and the remaining rows of W constitute a matrix We.For the new sampling dataXnew(k), independent vector that corresponds to the principal component space and residual space is S^newd(k)=WdXnew(k), S^newe(k)=WeXnew(k), respectively [8].We defined three process monitoring statistics [8]:
(8)U2(k)=S^newd(k)TS^newd(k),SPE(k)=e(k)Te(k)=(x(k)-x^(k))T(x(k)-x^(k)),Ue2(k)=S^newe(k)TS^newe(k).
And their confidence limits can be obtained by kernel density estimation. For more details about kernel density estimation, it can be found in the literature [9].Specific implementation steps are shown in Figure1.Figure 1
Fault detection algorithm flowchart.


## 3. Sparse SVM

Consider the second-order norm soft margin constraints; thus obtain the following optimization problem:(9)minw,ξ〈w,w〉+C∑i=1nξi2s.t.=yi(〈w,Φ(xi)〉+b)≥1-ξi,ξi≥0.Consider (9) of the dual problem:
(10)maxα,α*-∑i,j=1nyiyjαiαj(k(xi,xj)+δi,jC)+2∑i=1nαis.t.∑i=1nyiαi=0,αi≥0,i=1,2,…,n.The optimization problem can be written as an unconstrained one [10, 11]:
(11)minwwTw+C∑i=1LL(yi,wTΦ(xi)+b).Let us now consider nonlinear SVMs with a kernel functionk and an associated the reproducing kernel Hilbert space H. The optimization problem (11) becomes
(12)minf∈Hλ∥f∥H2+∑i=1nL(yi,f(xi)).We have made a change of variable by introducing the regularization parameterλ=1/C. We have also dropped the offset b for the sake of simplicity.Suppose now that the loss functionL is differentiable with respect to its second argument. Using the reproducing property, (12) becomes
(13)minβλβTKβ+∑i=1nL(yi,KiTβ).For a given value of the vectorβ, we say that a point xi is a support vector if yif(xi)<1 and if the loss on this point is nonzero. Let us reorder the training points such that the first  nsv points are support vectors. Finally, let Isv be the n×n diagonal matrix with the first nsv entries being 1 and the others 0. The gradient of (13) with respect to β is
(14)∇=2λKβ+2∑i=1nsvKiyi(yiKiTβ-1)=2(λβ+KIsv(Kβ-y)).
And the Hessian is H=2(λK+KIsvK).Each Newton step consists of the following update:β←β-γH-1∇, where the step size is found by one-dimensional search method.Combining (14) and H as ∇=Hβ-2KIsvy, we find that after the update:
(15)β=(λK+KIsvK)-1·KIsvy=(λIn+IsvK)-1Isvy.Using the fact that the lower left blockλIn+IsvK is 0, the update (15) turns out to be βsv=(λInsv+Ksv)-1ysv.Therefore, the algorithm of SVM based on the modified Newton optimization is performed as shown in Algorithm1.Algorithm 1:SVM based on the modified Newton optimization.
Function:β=PrimalSVMENM(K,y,λ)
sv
←
{
1
,
…
,
n
}; β_old=0
Repeat
β
=
0;
β
sv
=
(
λ
I
n
sv
+
K
sv
,
sv
)
-
1
y
sv;
step
=
β
-
β
_
old;
γ
=
1;
WhileΩ(β+γ·step)>Ω(β)+μγ·(∇T·step)
γ←ρ·γ;
End
β←β+γ·step;
β_old←β;
sv←i,i satisfy yi(KiTβ)<1;
Untilsv has not changed.
Whereμ∈(0,0.5), ρ∈(0,1).We now consider the optimization parameter is [b,βT]. The augmented Hessian is
(16)H=S[1TIsv11TKIsvKIsv1λK+KIsvK],
where 1 should be understood as a vector of all 1.We still use the former method of choice to choose the right “basis functions” and realize the purpose of building sparse SVM. Assume that the choice of “basis functions” collection isJ; then the selection of the next “basis function” j is equivalent to ensuring that βJ is constant and optimizing the following questions:
(17)minλ[βJTβJ][KJJKJjKjJKjj][βJβj]+∑i∈sv(1-yi(KiJβJ+Kijβj))2.Through the introduction of the Cholesky decomposition and the corresponding rank 1 update strategy, based on unconstrained optimization SpSVM with the following steps to achieve the following:(1)
update matrixλInsv+Ksv using the Cholesky decomposition;(2)
optimizeβJ using Newton’s method (Algorithm 1). If sv changes, update matrix λInsv+Ksv using rank 1 update strategy;(3)
if the “basis function” collectionJ is equal to the size of preset value, end of the program, otherwise, turn to step 4;(4)
select the next “basis function”j:J←J+j, transferred to step 1.


## 4. Experimental Results

The proposed method is applied to the simulation of the Tennessee Eastman chemical process to achieve fault detection and fault diagnosis. SVM training involved in the QP and the QCQP problem can be by optimizing the package MOSEK method based on the interior point, and the SVM training data are preprocessing.The process has 12 manipulated variables, 22 continuous process measurements, and 19 composition measurements sampled less frequently. The process contains the Gaussian noise. TE process includes 20 predefined failure modes, representing a step, random changes, slow drift, and other fault type, its process flow diagram and detailed process, see the text [12]. We use the control strategy in paper [13] to conduct closed-loop control. Simulation data for a 960×52 matrix are obtained as follows: 52 represent the number of process variables, and 960 represent the sampling points of each variable. The simulation is running 48 hours. The failure was in the introduction at the first 8 hours.In order to verify the effectiveness of the KICA monitoring algorithms, KPCA and KICA were applied to detect fault 5. The monitoring charts of KPCA for fault 5 are shown in Figure2. The figures indicate that T2 and SPE statistics can also detect the happening of faults, but there will be false positives and false negatives in some sampling points. The monitoring charts of KICA for fault 5 are shown in Figure 3. In contrast to KPCA, it can be seen that the U2, Ue2, and SPE statistics all respond strongly to the faults.KPCA monitoring for fault 5.
(a)
(b)KICA monitoring for fault 5.
(a)
(b)
(c)If the statistics exceed the control limits, it indicates a failure occurrence. We select 52 variables for the condition attributes and select 15 groups for the decision attribute, respectively, on behalf of 15 different types of failures. The training set is composed of 600 data, 40 samples data of each category. Test set is composed of 1200 data, 80 samples data of each category. We use 1-a.-r-mix strategy to achieve the multiclass classification SVM. The factorC is then set to 1000. The size of the alternative set κ in SpSVM is 20. Specific diagnostic results are shown in Table 1.Table 1
Results of fault identification.
Diagnostic precision
n
sv
v
SV
M
1vr
MK
L
DT
SpSV
M
1vr
SV
M
1vr
MK
L
DT
SpSV
M
1vr
0
100%
100%
100%
97
40
19
0.02
99.2%
98.1%
98.9%
130
48
29
0.05
98.8%
96.2%
98.2%
165
61
46


## 5. Conclusions

In this paper, KICA and sparse SVM are proposed for the complex industrial process monitoring. KICA can efficiently compute ICs in high-dimensional feature spaces by means of integral operations. The method using sparse SVM can reduce the number of support vectors without reducing the classification accuracy. The application results show the effectiveness of proposed method.



---

*Source: 987345-2013-04-11.xml*