# Nonlinear Fault Separation for Redundancy Process Variables Based on FNN in MKFDA Subspace

Authors: Ying-ying Su; Shan Liang; Jing-zhe Li; Xiao-gang Deng; Tai-fu Li; Cheng Zeng
Journal: Journal of Applied Mathematics
 (2014)

---

## Abstract

Nonlinear faults are difficultly separated for amounts of redundancy process variables in process industry. This paper introduces an improved kernel fisher distinguish analysis method (KFDA). All the original process variables with faults are firstly optimally classified in multi-KFDA (MKFDA) subspace to obtain fisher criterion values. Multikernel is used to consider different distributions for variables. Then each variable is eliminated once from original sets, and new projection is computed with the same MKFDA direction. From this, differences between new Fisher criterion values and the original ones are tested. If it changed obviously, the effect of eliminated variable should be much important on faults called false nearest neighbors (FNN). The same test is applied to the remaining variables in turn. Two nonlinear faults crossed in Tennessee Eastman process are separated with lower observation variables for further study. Results show that the method in the paper can eliminate redundant and irrelevant nonlinear process variables as well as enhancing the accuracy of classification.


---

## Body

## 1. Introduction

With developments of modern process industry, multivariate monitor from sensors has showed their multicollinearity, nonlinear correlative coupling, time delay, and redundancy. It makes complexity increasing with exponent to fault separation and diagnosis, called “Curse of Dimension” [1, 2]. On the other hand, right ratio of fault classification decreases with multivariate and redundancy process variables. Therefore, many attentions have been paid on two points of view that are variable selection and dimension reduction [3, 4].Among the study of variable selection, the existed methods can be broadly classified into three categories: random search techniques, measure-based method, and intelligent computation. In random search, each process variable is directly deleted or involved in the classification model one time in turn to search the most suitable input sets under a certain criterion, such as forward selection, backward selection, and stepwise that are simple and easily realized methods [5]. While it was studied by Masion and Gunst [6] that these methods would result in mistaken results, variable set appears multicollinearity. Measure-based method appears to select variable with computing relevancy among all variables, as well as that between variables and labels. The variables with highest similar characteristic will be gathered in one kind. According to different definition, K-L information measure, minimum description length, and mutual information are used [7–9]. Intelligent computation deepens to solve nonlinear variable selection problem, such as neurnal network that is once used to nonlinear model, while its selection criterion is uncertain [10].Dimension reduction is different from variable selection, which mainly depends on transformation and information extraction of original variable matrix. It projects original variables with a certain mapping to a new subspace and extracts information in lower dimension, such as principal component analysis (PCA) [11] and partial least squares (PLS) [12]. Original variables with linear-relative process variables are linearly projected according to the maximum direction of covariance matrix. Meanwhile, the maximum original information can be kept as most as possible. Contribution chart method is the way to calculate contribution of each variable to certain fault with T2 statics and SPE [13, 14] for PCA. The above linear methods have been extended to nonlinear ones after kernel method presented [15–20], such as kernel principal component analysis (KPCA), kernel partial least squares (KPLS), and kernel fisher discriminant analysis (KFDA). Kernel method converts a linear classification learning algorithm into nonlinear one, by mapping the original observations into a higher-dimensional space. So that linear classifier in the new space equals to a nonlinear classifier in the original space.However, nonlinear information projected to the new feature space has higher dimension, and data matrix has lost their original physical meaning in original sample space. If we separated nonlinear faults crossed together in original space, the dimension of classifier with kernel method would become huge, while right ratio would decrease with redundancy and multicollinearity variables.The objective of this paper is to deepen dimension reduction method for the above problems with measure method in variable selection called MKFDA-FNN. Nonlinear process variables are projected in higher-dimension space with MKFDA. Discriminant vector and its corresponding feature vector with maximum separation are computed to cluster original variables with highest similarity. With embed-dimension increasing, false nearest neighbors (FNN) with high similarity are able to be removed in turn. Thus, nonlinear redundancy and multicollinearity process variables can be removed from input sets to nonlinear classifier. Finally, we give an actual fault separation problem in classical chemical process Tennessee Eastman (TE) to further study.


## 2. Problem Description

In fault separation problem presented above, it equals to screen original process variables related to certain faults as most as possible. Multivariate data matrix considered initially with normal and fault information is described in Figure1, where X1,X2,…,Xn are process variables with n-dimension, Z11-1,Z12-1,…,Z1f-1 present time-delay variables of X1 at different sample time, Z21-1,Z22-1,…,Z2g-1 present time-delay variables of X2, and Zn1-1,Zn2-1,…,Znh-1 present time-delay variables of Xn at different sample-time. In this way, original data matrix is composed of n-dimension process/control variables and their delay variables in
(1)X(f+g+h)×l(k)=[x1(k-1)x1(k-2)⋯x1(k-l)x1(k-2)x1(k-3)⋯x1(k-l-1)⋮⋮⋮⋮x1(k-f)x1(k-f-1)⋯x1(k-f-l+1)x2(k-1)x2(k-2)⋯x2(k-l)x2(k-2)x2(k-3)⋯x2(k-l-1)⋮⋮⋮⋮x2(k-g)x2(k-g-1)⋯x2(k-g-l+1)⋮⋮⋮⋮xn(k-1)xn(k-2)⋯xn(k-l)xn(k-2)xn(k-3)⋯xn(k-l-1)⋮⋮⋮⋮xn(k-h)xn(k-h-1)⋯xn(k-h-l+1)],
where f, g, and h present maximum delay order of process/control variables x1, x2, and xn, k presents current sample time, and l is sample length.Figure 1
The fault diagnosis with multivariate.


## 3. Multivariate Fault Separation Based on MKFDA-FNN

To fault separation problem with nonlinear redundancy process/control variables, an approach is proposed in Figure2. Correlated nonlinear variables are firstly projected to a higher-dimension MKFDA subspace. Furthermore, in order to find fairly useful variables, the importance of each input is measured in subspace with distance measure inspired by FNN. Accordingly, redundant variables are recognized. It makes separation of faults crossed together easily.Figure 2
Nonlinear fault diagnosis with redundancy process variables based on FNN in MKFDA subspace.


### 3.1. False Nearest Neighbors

FNN is the feature selection method on the basis of phase space reconstruction (PSR) in high-dimension data space [21]. With embed-dimension increasing, movement locus becomes open, and false nearest neighbors with high similarity are able to be removed in turn. It restores the locus of chaos. Its algorithm is as follows.Ind-dimension phase space including original variables and their time delay, each phase vector x(i)={x(i),x(i+τ),…,x(i+(d-1)τ)} has one nearest neighbors xNN(i). Their 2-norm distance is
(2)Rd+12(i)=Rd2(i)+∥x(i+τd)-xNN(i+τd)∥.Whend-dimension is increased to d+1, the above phase vector is changed as new one, noted as Rd+1(i) in
(3)Rd(i)=∥x(i)-xNN(i)∥.IfRd+1(i) was much bigger than Rd(i), it means the projection of two nonneighbor phase vector from higher dimension to lower one. So the two neighbors are the false nearest neighbors.Note that(4)a1(i,d)=∥x(i+τd)-xNN(i+τd)∥Rd(i).Ifa1(i,d) is larger than Rτ, xNN(i) should be fault nearest neighbor of x(i). Threshold Rτ is determined between interval (10, 50). Once there appeared noise in process data, the following judge criterion should be involved. If Rd+1(i)/RA≥2, xNN(i) should be nearest fault neighbor of x(i), where RA is
(5)RA=1N∑i=1N[x(i)-x-],x-=1N∑i=1Nx(i).The distance measure between vectors can explain the similarity of false nearest neighbors factually in (6). Assume that there was a data space Q with d-dimension variable, and one sample vector is A=(q1,q2,…,qi,…,qd). We set variable qi as zero, standing for vector A without variable qi that is noted as B=(q1,q2,…,0,…,qd) in Figure 3.Figure 3
Data spaceQ with d-dimension variable.The similarity betweenA and B is
(6)δ=∥A-B∥2.If distance measure is small, it shows that vectorsA and B have highly similarity. That is, the removed variable qi makes little impact on nonlinear pattern, and process variable qi has low interpreting ability. Otherwise, if it was much bigger, it reveals that B much differs from A. Process variable qi is important to interpreting of nonlinear pattern. B is false nearest neighbors of A.


### 3.2. Kernel Fisher Discriminant Analysis

KFDA is most useful to nonlinear classification problems [22]. Nonlinear discriminant vector in original space is extracted to linear optimal discriminate vector in high-dimension feature space H with conventional fisher discriminant analysis (FDA). Since dimension of H is much higher, it is hard to directly confirm nonlinear mapping function from original space to the feature space. Reproducing kernel-based method widely developed in machine learning (ML) can achieve this goal. Nonlinear mapping is indirectly found according to k(x,y)=Φ(x)TΦ(y) in Gram-space [23], where Φ:Rd→H.Conventional kernel function can be selected as follows [6].(i)
Polynomial kernel functionK(x,x′)=(x·x′+c)d, d=1,2,…,N, c is constant.(ii)
Gaussian kernel functionK(x,x′)=exp⁡(-(∥x-x′∥/2σ2)), δ is the parameter of breadth.(iii)
Sigmoid kernel functionK(x,x′)=tanh(f〈x,x′〉+θ).Assume that original sample set wasX={x1,x2,…,xN} with d-dimension and N-samples, where Xi is the sample of ith type, Ni=|Xi|, and i=1,2. There exists nonlinear mapping function Φ:Rd→H. It transforms nonlinear original sample space Rd to linear classification in high-dimension data space H; that is, Φ(x)∈H, x∈Rd. In space H, distance scatter of intraclass and classes with training data is Sω and Sb in (7) and (9), respectively,
(7)Sω=S1+S2,(8)Si=∑x∈Xi(Φ(x)-mi)(Φ(x)-mi)T,i=1,2,(9)Sb=(m1-m2)(m1-m2)T,
where mi is the mean of ith type in feature space. KFDA is to find a projection direction w, which meets the following two properties: (1) data that has similar characteristic should be gathered together as most as possible; (2) the ones with different characteristic should be gathered as far as possible. So a key is to search projection direction w* and its corresponding discriminant function g(x)=(w*)Tx-y0. Similarly with linear FDA, the optimal projection direction w* is to search vector w, which maximizes fisher criterion function (10), where w* is optimal projection direction:
(10)JH(w)=wTSbwwTSωw.Since dimension of feature spaceH is usually high and Φ is indirect mapping function, discriminant vector is hard to compute directly. Thus, each solution w is expressed as linear combination of samples in (11), according to kernel-based method,
(11)w=∑i=1NαiΦ(xi)=ψα,
where α=(α1,α2,…,αN)T.Moreover, nonlinear transformation functionΦ(x1),Φ(x2),…,Φ(xN) of samples can be projected to feature space H with direction w in
(12)wTΦ(xi)=αTψTΦ(xi)=αT(Φ(x1),Φ(x2),…,Φ(xN))TΦ(xi),=αT(k(x1,xi),k(x2,xi),…,k(xN,xi)),=αTkxi,From (11), for all x∈Rd, assume that kx=(k(x1,x),k(x2,x),…,k(x2,x))T and projection of mean vector mi with direction w* in feature space H is
(13)w*mi=αTψT(1Ni∑j=1NiΦ(xji))=αTui,
where ui=((1/Ni)∑j=1NiΦ(x1)Φ(xji), (1/Ni)∑j=1NiΦ(x2)Φ(xji),…,)T.From (12) and (13), we have
(14)wTSbw=αTkbα,wTSωw=αTkωα,
where kb=(u1-u2)(u1-u2)T, kw=kw1+kw2, kwi=∑j=1Ni(kxji-ui)(kxji-ui)T.Since fisher criterion function is optimal solution of (15), vector w can be resolved as α in the following fisher criterion (16) [24]:
(15)JH(w)=wTSbwwTSbw,(16)JH(α)=αTkbααTkwα.Furthermore, the solution of optimal vectorα* and y0 can be solved [25] with
(17)α*=kw-1(u1-u2),y0=(w*)Tm1+(w*)Tm22=(α*)T(u1+u2)2.Thus, the corresponding function of kernel fisher discriminant function is obtained as(18)g(x)=(a*)Tkx-y0.


### 3.3. Multikernel Fisher Discriminant Analysis

From Section3.2, the solution of maximizing (15) equals to the solution of maximizing (16). Assume that α*=kw-1(u1-u2) is optimal solution to classification effect, whereas α* is both determined by kernel scatter matrix kw and difference of kernel mean vector (u1-u2). In the condition of independent and identically distributed, kernel mean of samples is independent with number of samples. It indicates difference of kernel mean vector (u1-u2) doing nothing with the unbalance of samples. So α* is only determined by kernel scatter matrix kw for intraclass. If distribution of different variables differed, it should result in the contributions kw1,kw2,…,kw not in the similar interval. Besides that the solution of α* is not the optimal one. Hence, in order to avoid the influence of different distribution for samples, we presented multikernel fisher discriminant analysis method. It advances the kernel criterion function kw=(kw1+kw2) into
(19)kw=λkw1+(1-λ)kw2,
where λ(λ∈[0,1]) is the adjustable MATLAB parameter and kw1and kw2 are the kernel matrix computed with each suitable kernel function from Section 3.2 (i)/(ii)/(iii).In this way, the influence with different sample distributions is considered with the suitable kernel function.The above algorithm in this paper can be chiefly described in Table1. In this way, the contribution of each original process variable qi to the certain fault is measured.Table 1
Steps designed in this paper.
Inputs
T
=
{
(
x
1
,
y
1
)
,
(
x
2
,
y
2
)
,
…
,
(
x
N
,
y
N
)
}, xi∈Rd, yi∈{+1,-1}, i=1,2,…,N
Step  1
Initiateλ∈[0,1] and compute ui, i=1,2
Step  2
Select suitable multikernel function
Step  3
Compute the kernel mean vector between two kinds withkwi=∑j=1Ni(Kxji-ui)(Kxji-ui)T
Step  4
Compute the kernel scatter matrix of intraclasskw=λkw1+(1-λ)kw2
Step  5
Computeα*=kw-1(u1-u2), y0=αT(u1-u2)
Step  6
Get the optimal solution of (16)
Step  7
Place the inspected process variable as zero in original samples
Step  8
Project the new samples into the feature space
Step  9
Compute the contribution of one variable at one time with FNN in MKFDA
Step  10
Repeat the above course for the remaining variables
Outputs
The distance measureδ=[δ1,δ2,…,δn] of each original variable is obtained


## 3.1. False Nearest Neighbors

FNN is the feature selection method on the basis of phase space reconstruction (PSR) in high-dimension data space [21]. With embed-dimension increasing, movement locus becomes open, and false nearest neighbors with high similarity are able to be removed in turn. It restores the locus of chaos. Its algorithm is as follows.Ind-dimension phase space including original variables and their time delay, each phase vector x(i)={x(i),x(i+τ),…,x(i+(d-1)τ)} has one nearest neighbors xNN(i). Their 2-norm distance is
(2)Rd+12(i)=Rd2(i)+∥x(i+τd)-xNN(i+τd)∥.Whend-dimension is increased to d+1, the above phase vector is changed as new one, noted as Rd+1(i) in
(3)Rd(i)=∥x(i)-xNN(i)∥.IfRd+1(i) was much bigger than Rd(i), it means the projection of two nonneighbor phase vector from higher dimension to lower one. So the two neighbors are the false nearest neighbors.Note that(4)a1(i,d)=∥x(i+τd)-xNN(i+τd)∥Rd(i).Ifa1(i,d) is larger than Rτ, xNN(i) should be fault nearest neighbor of x(i). Threshold Rτ is determined between interval (10, 50). Once there appeared noise in process data, the following judge criterion should be involved. If Rd+1(i)/RA≥2, xNN(i) should be nearest fault neighbor of x(i), where RA is
(5)RA=1N∑i=1N[x(i)-x-],x-=1N∑i=1Nx(i).The distance measure between vectors can explain the similarity of false nearest neighbors factually in (6). Assume that there was a data space Q with d-dimension variable, and one sample vector is A=(q1,q2,…,qi,…,qd). We set variable qi as zero, standing for vector A without variable qi that is noted as B=(q1,q2,…,0,…,qd) in Figure 3.Figure 3
Data spaceQ with d-dimension variable.The similarity betweenA and B is
(6)δ=∥A-B∥2.If distance measure is small, it shows that vectorsA and B have highly similarity. That is, the removed variable qi makes little impact on nonlinear pattern, and process variable qi has low interpreting ability. Otherwise, if it was much bigger, it reveals that B much differs from A. Process variable qi is important to interpreting of nonlinear pattern. B is false nearest neighbors of A.


## 3.2. Kernel Fisher Discriminant Analysis

KFDA is most useful to nonlinear classification problems [22]. Nonlinear discriminant vector in original space is extracted to linear optimal discriminate vector in high-dimension feature space H with conventional fisher discriminant analysis (FDA). Since dimension of H is much higher, it is hard to directly confirm nonlinear mapping function from original space to the feature space. Reproducing kernel-based method widely developed in machine learning (ML) can achieve this goal. Nonlinear mapping is indirectly found according to k(x,y)=Φ(x)TΦ(y) in Gram-space [23], where Φ:Rd→H.Conventional kernel function can be selected as follows [6].(i)
Polynomial kernel functionK(x,x′)=(x·x′+c)d, d=1,2,…,N, c is constant.(ii)
Gaussian kernel functionK(x,x′)=exp⁡(-(∥x-x′∥/2σ2)), δ is the parameter of breadth.(iii)
Sigmoid kernel functionK(x,x′)=tanh(f〈x,x′〉+θ).Assume that original sample set wasX={x1,x2,…,xN} with d-dimension and N-samples, where Xi is the sample of ith type, Ni=|Xi|, and i=1,2. There exists nonlinear mapping function Φ:Rd→H. It transforms nonlinear original sample space Rd to linear classification in high-dimension data space H; that is, Φ(x)∈H, x∈Rd. In space H, distance scatter of intraclass and classes with training data is Sω and Sb in (7) and (9), respectively,
(7)Sω=S1+S2,(8)Si=∑x∈Xi(Φ(x)-mi)(Φ(x)-mi)T,i=1,2,(9)Sb=(m1-m2)(m1-m2)T,
where mi is the mean of ith type in feature space. KFDA is to find a projection direction w, which meets the following two properties: (1) data that has similar characteristic should be gathered together as most as possible; (2) the ones with different characteristic should be gathered as far as possible. So a key is to search projection direction w* and its corresponding discriminant function g(x)=(w*)Tx-y0. Similarly with linear FDA, the optimal projection direction w* is to search vector w, which maximizes fisher criterion function (10), where w* is optimal projection direction:
(10)JH(w)=wTSbwwTSωw.Since dimension of feature spaceH is usually high and Φ is indirect mapping function, discriminant vector is hard to compute directly. Thus, each solution w is expressed as linear combination of samples in (11), according to kernel-based method,
(11)w=∑i=1NαiΦ(xi)=ψα,
where α=(α1,α2,…,αN)T.Moreover, nonlinear transformation functionΦ(x1),Φ(x2),…,Φ(xN) of samples can be projected to feature space H with direction w in
(12)wTΦ(xi)=αTψTΦ(xi)=αT(Φ(x1),Φ(x2),…,Φ(xN))TΦ(xi),=αT(k(x1,xi),k(x2,xi),…,k(xN,xi)),=αTkxi,From (11), for all x∈Rd, assume that kx=(k(x1,x),k(x2,x),…,k(x2,x))T and projection of mean vector mi with direction w* in feature space H is
(13)w*mi=αTψT(1Ni∑j=1NiΦ(xji))=αTui,
where ui=((1/Ni)∑j=1NiΦ(x1)Φ(xji), (1/Ni)∑j=1NiΦ(x2)Φ(xji),…,)T.From (12) and (13), we have
(14)wTSbw=αTkbα,wTSωw=αTkωα,
where kb=(u1-u2)(u1-u2)T, kw=kw1+kw2, kwi=∑j=1Ni(kxji-ui)(kxji-ui)T.Since fisher criterion function is optimal solution of (15), vector w can be resolved as α in the following fisher criterion (16) [24]:
(15)JH(w)=wTSbwwTSbw,(16)JH(α)=αTkbααTkwα.Furthermore, the solution of optimal vectorα* and y0 can be solved [25] with
(17)α*=kw-1(u1-u2),y0=(w*)Tm1+(w*)Tm22=(α*)T(u1+u2)2.Thus, the corresponding function of kernel fisher discriminant function is obtained as(18)g(x)=(a*)Tkx-y0.


## 3.3. Multikernel Fisher Discriminant Analysis

From Section3.2, the solution of maximizing (15) equals to the solution of maximizing (16). Assume that α*=kw-1(u1-u2) is optimal solution to classification effect, whereas α* is both determined by kernel scatter matrix kw and difference of kernel mean vector (u1-u2). In the condition of independent and identically distributed, kernel mean of samples is independent with number of samples. It indicates difference of kernel mean vector (u1-u2) doing nothing with the unbalance of samples. So α* is only determined by kernel scatter matrix kw for intraclass. If distribution of different variables differed, it should result in the contributions kw1,kw2,…,kw not in the similar interval. Besides that the solution of α* is not the optimal one. Hence, in order to avoid the influence of different distribution for samples, we presented multikernel fisher discriminant analysis method. It advances the kernel criterion function kw=(kw1+kw2) into
(19)kw=λkw1+(1-λ)kw2,
where λ(λ∈[0,1]) is the adjustable MATLAB parameter and kw1and kw2 are the kernel matrix computed with each suitable kernel function from Section 3.2 (i)/(ii)/(iii).In this way, the influence with different sample distributions is considered with the suitable kernel function.The above algorithm in this paper can be chiefly described in Table1. In this way, the contribution of each original process variable qi to the certain fault is measured.Table 1
Steps designed in this paper.
Inputs
T
=
{
(
x
1
,
y
1
)
,
(
x
2
,
y
2
)
,
…
,
(
x
N
,
y
N
)
}, xi∈Rd, yi∈{+1,-1}, i=1,2,…,N
Step  1
Initiateλ∈[0,1] and compute ui, i=1,2
Step  2
Select suitable multikernel function
Step  3
Compute the kernel mean vector between two kinds withkwi=∑j=1Ni(Kxji-ui)(Kxji-ui)T
Step  4
Compute the kernel scatter matrix of intraclasskw=λkw1+(1-λ)kw2
Step  5
Computeα*=kw-1(u1-u2), y0=αT(u1-u2)
Step  6
Get the optimal solution of (16)
Step  7
Place the inspected process variable as zero in original samples
Step  8
Project the new samples into the feature space
Step  9
Compute the contribution of one variable at one time with FNN in MKFDA
Step  10
Repeat the above course for the remaining variables
Outputs
The distance measureδ=[δ1,δ2,…,δn] of each original variable is obtained


## 4. Fault Separation of Tennessee Eastman with Redundancy Variables

### 4.1. Tennessee Eastman Chemical Process

Tennessee Eastman (TE) is a classical chemical process created by Eastman Chemical Company in 1993 [26]. Its technological process is shown in Figure 4. There are four reactants (A, C, D, and E) and two products (G, H). Besides that, there is one inert material B and byproduct F.Figure 4
The technological process of Tennessee Eastman.In TE process, the dynamic TE model is composed of five major units: a reactor, a separator, a stripper, a condenser, and a compressor. Each unit can be expressed with some equations, in all of 148 algebraic equations and 30 differential equations. So it becomes one of the most complex models and is widely used to test study algorithm with control, system monitor, fault diagnosis, and so forth. Here, we take Tennessee Eastman as the study object to measure its fault separation ability with our method.


### 4.2. Nonlinear Fault Separation of Redundancy Variables

In TE process, there are 41 observed variables and 12 manipulated variables from controller, some of which are nonlinear redundancy variables. Moreover, there are 20 types of classical fault in TE process shown in Table2. Since Fault9 and Fault11 are nonlinear overlapped together shown in Figure 5, we take their fault separation as the study goal, meanwhile, 53 process variables must be screened for their multicollinearity and nonlinear redundancy. Process data of TE is simulated at one-minute sampling time in MATLAB software from Downs [27]. All the measurements have Gaussian noise. A total of 1000 samples are collected for training, where 800 data are collected for Fault9 and 200 for Fault11. In addition, 835 samples are applied to test separation validity with 644 for Fault9 and 171 for Fault11.Table 2
State distribution in TE process.
Fault
Disturbance
Type
1
A/C feed ratio, B composition constant
Step
2
B composition, A/C ratio constant
Step
3
D feed temperature
Step
4
Reactor cooling water inlet temperature
Step
5
Condenser cooling water inlet temperature
Step
6
A feed loss
Step
7
C header pressure loss-reduced availability
Step
8
A, B, C feed composition
Random
9
D feed temperature
Random
10
C feed temperature
Random
11
Reactor cooling water inlet temperature
Random
12
Condenser cooling water inlet temperature
Random
13
Reaction kinetics
Slow drift
14
Reactor cooling water valve
Sticking
15
Condenser cooling water valve
Sticking
16–20
Unknown
UnknownFigure 5
The distribution of Fault9 and Fault11, described in 2-dimension diagram withx-axis of process variableVab.13 and y-axis of process variable Vab.21.


### 4.3. Results and Discussion

If we distinguished Fault9 and Fault11, there are 53 variables to be considered in all. Therefore, we compute the contribution of 53 variables with mentioned method to see the importance of each process variables on faults. Multikernel function is selected as Gaussian kernel and polynomial kernel, each comprised of 50%. The contributions of each variable to the faults are computed with steps in Section3.3 that is shown in Figure 6 and Table 3. From large to small, the proper importance of all the 53 process variables is reordered as {Vab.21, Vab.13, Vab.9, Vab.16, Vab.7, Vab.20, Vab.11, Vab.2, Vab.12, Vab.8, Vab.19, Vab.5, Vab.22, Vab.6, Vab.3, Vab.18, Vab.14, Vab.15, Vab.17, Vab.10, Vab.41, Vab.40, Vab.27, Vab.23, Vab.29, Vab.31, Vab.26, Vab.33, Vab.25, Vab.32, Vab.4, Vab.24, Vab.30, Vab.35, Vab.34, Vab.37, Vab.36, Vab.28, Vab.39, Vab.38, Vab.1, Vab.53, Vab.52, Vab.51, Vab.50, Vab.49, Vab.48, Vab.47, Vab.46, Vab.45, Vab.44, Vab.43, Vab.42}.Table 3
The contributions of 53 process variables to fault separation.
Process variable
Contribution
Process variable
Contribution
Vab.21
2.8273
Vab.17
0.0000
Vab.13
2.1145
Vab.10
0.0000
Vab.9
1.2318
Vab.41
0.0000
Vab.16
1.1313
Vab.40
0.0000
Vab.7
0.2687
Vab.27
0.0000
Vab.20
0.1319
Vab.23
0.0000
Vab.11
0.0522
Vab.29
0.0000
Vab.2
0.0355
Vab.31
0.0000
Vab.12
0.0259
Vab.26
0.0000
Vab.8
0.0191
Vab.33
0.0000
Vab.19
0.0092
Vab.25
0.0000
Vab.5
0.0012
Vab.32
0.0000
Vab.22
0.0006
Vab.4
0.0000
Vab.6
0.0004
Vab.24
0.0000
Vab.3
0.0002
Vab.30
0.0000
Vab.18
0.0000
Vab.35
0.0000
Vab.14
0.0000
⋮
⋮
Vab.15
0.0000
Vab.42
0.0000Figure 6
Contribution of all the 53 process variables to distinguish Fault9 and Fault11.In the Following, the curves of the first two importantVab.21 and Vab.13 in TE process are given in Figures 7(a) and 7(b) and Figures 8(a) and 8(b), respectively. It expresses the strong variation of process variables Vab.21 and Vab.13, actually.The changing of processVab.21 in actual TE.
(a)
ProcessVab.21 (reactor coolant temperature) in Fault9
(b)
ProcessVab.21 (reactor coolant temperature) in Fault11The changing of processVab.13 in actual TE.
(a)
ProcessVab.13 (product separation pressure) in Fault9
(b)
ProcessVab.13 (product separation pressure) in Fault11According to the sequence of each process variable, the different feature sets are constructed as {Vab.21}, {Vab.21, Vab.13}, {Vab.21, Vab.13, Vab.9}, and so on. Nonlinear pattern classification of Fault9 and Fault11 is tested with support vector machine (SVM), which is widely used in pattern recognition. The parameters of SVM are optimized with cross-validation c=2035 and g=1024. With the above variable sets, the accuracy of fault separation between Fault9 and Fault11 is successively tested. The results are shown in Figure 9 and Table 4. It reveals that the separation accuracy becomes lower when the considered variables increase.Table 4
The accuracy with different feature sets with testing data.
Feature set
Combination of variables
Accuracy
Set1
Vab.21
83.521%
Set2
Vab.21, Vab.13
85.731%
Set3
Vab.21, Vab.13, Vab.9
89.652%
Set4
Vab.21, Vab.13, Vab.9, Vab.16
92.123%
Set5
Vab.21, Vab.13, Vab.9, Vab.16, Vab.7
94.547%
Set6
Vab.21, Vab.13, Vab.9, Vab.16, Vab.7, Vab.20
92.532%
⋮
⋮
⋮
Set53
Vab.21, Vab.13, Vab.9, Vab.16, Vab.7, Vab.20, … Vab.42
72.119%Figure 9
The accuracy with different feature sets to indentify Fault9 and Fault11 with testing data.From the above results, we conclude that(1) if all the 53 process variables were used to separate Fault9 and Fault 11, right ratio is merely 72.12%. It indicates that not all of the variables are directly related to certain fault. Some redundancy or irrelevant variables may decrease the classification accuracy and must be eliminated. (2) If the feature were selected as the first five process variables {Vab.21, Vab.13, Vab.9, Vab.16, Vab.7}, the accuracy increases to the highest as 94.55%. It means that the above five process variables are key to the fault separation. (3) If the model should be simplified at most, the process variable {Vab.21} is the best feature variable. We can recognize Fault9 and Fault11 according to the process changing of Vab.21.On the other hand, Fault9 stands for the random disturbance toD feed temperature. Fault11 is random disturbance to reactor cooling water inlet temperature. While {Vab.21, Vab.13, Vab.9, Vab.16, Vab.7} are the reactor coolant temperature, product separation pressure, reactor temperature, stripper pressure, reactor pressure, respectively, it is easy to see that the five selected variables are fairly relative to Fault9 and Fault11. The simulation results keep pace with the reality.


## 4.1. Tennessee Eastman Chemical Process

Tennessee Eastman (TE) is a classical chemical process created by Eastman Chemical Company in 1993 [26]. Its technological process is shown in Figure 4. There are four reactants (A, C, D, and E) and two products (G, H). Besides that, there is one inert material B and byproduct F.Figure 4
The technological process of Tennessee Eastman.In TE process, the dynamic TE model is composed of five major units: a reactor, a separator, a stripper, a condenser, and a compressor. Each unit can be expressed with some equations, in all of 148 algebraic equations and 30 differential equations. So it becomes one of the most complex models and is widely used to test study algorithm with control, system monitor, fault diagnosis, and so forth. Here, we take Tennessee Eastman as the study object to measure its fault separation ability with our method.


## 4.2. Nonlinear Fault Separation of Redundancy Variables

In TE process, there are 41 observed variables and 12 manipulated variables from controller, some of which are nonlinear redundancy variables. Moreover, there are 20 types of classical fault in TE process shown in Table2. Since Fault9 and Fault11 are nonlinear overlapped together shown in Figure 5, we take their fault separation as the study goal, meanwhile, 53 process variables must be screened for their multicollinearity and nonlinear redundancy. Process data of TE is simulated at one-minute sampling time in MATLAB software from Downs [27]. All the measurements have Gaussian noise. A total of 1000 samples are collected for training, where 800 data are collected for Fault9 and 200 for Fault11. In addition, 835 samples are applied to test separation validity with 644 for Fault9 and 171 for Fault11.Table 2
State distribution in TE process.
Fault
Disturbance
Type
1
A/C feed ratio, B composition constant
Step
2
B composition, A/C ratio constant
Step
3
D feed temperature
Step
4
Reactor cooling water inlet temperature
Step
5
Condenser cooling water inlet temperature
Step
6
A feed loss
Step
7
C header pressure loss-reduced availability
Step
8
A, B, C feed composition
Random
9
D feed temperature
Random
10
C feed temperature
Random
11
Reactor cooling water inlet temperature
Random
12
Condenser cooling water inlet temperature
Random
13
Reaction kinetics
Slow drift
14
Reactor cooling water valve
Sticking
15
Condenser cooling water valve
Sticking
16–20
Unknown
UnknownFigure 5
The distribution of Fault9 and Fault11, described in 2-dimension diagram withx-axis of process variableVab.13 and y-axis of process variable Vab.21.


## 4.3. Results and Discussion

If we distinguished Fault9 and Fault11, there are 53 variables to be considered in all. Therefore, we compute the contribution of 53 variables with mentioned method to see the importance of each process variables on faults. Multikernel function is selected as Gaussian kernel and polynomial kernel, each comprised of 50%. The contributions of each variable to the faults are computed with steps in Section3.3 that is shown in Figure 6 and Table 3. From large to small, the proper importance of all the 53 process variables is reordered as {Vab.21, Vab.13, Vab.9, Vab.16, Vab.7, Vab.20, Vab.11, Vab.2, Vab.12, Vab.8, Vab.19, Vab.5, Vab.22, Vab.6, Vab.3, Vab.18, Vab.14, Vab.15, Vab.17, Vab.10, Vab.41, Vab.40, Vab.27, Vab.23, Vab.29, Vab.31, Vab.26, Vab.33, Vab.25, Vab.32, Vab.4, Vab.24, Vab.30, Vab.35, Vab.34, Vab.37, Vab.36, Vab.28, Vab.39, Vab.38, Vab.1, Vab.53, Vab.52, Vab.51, Vab.50, Vab.49, Vab.48, Vab.47, Vab.46, Vab.45, Vab.44, Vab.43, Vab.42}.Table 3
The contributions of 53 process variables to fault separation.
Process variable
Contribution
Process variable
Contribution
Vab.21
2.8273
Vab.17
0.0000
Vab.13
2.1145
Vab.10
0.0000
Vab.9
1.2318
Vab.41
0.0000
Vab.16
1.1313
Vab.40
0.0000
Vab.7
0.2687
Vab.27
0.0000
Vab.20
0.1319
Vab.23
0.0000
Vab.11
0.0522
Vab.29
0.0000
Vab.2
0.0355
Vab.31
0.0000
Vab.12
0.0259
Vab.26
0.0000
Vab.8
0.0191
Vab.33
0.0000
Vab.19
0.0092
Vab.25
0.0000
Vab.5
0.0012
Vab.32
0.0000
Vab.22
0.0006
Vab.4
0.0000
Vab.6
0.0004
Vab.24
0.0000
Vab.3
0.0002
Vab.30
0.0000
Vab.18
0.0000
Vab.35
0.0000
Vab.14
0.0000
⋮
⋮
Vab.15
0.0000
Vab.42
0.0000Figure 6
Contribution of all the 53 process variables to distinguish Fault9 and Fault11.In the Following, the curves of the first two importantVab.21 and Vab.13 in TE process are given in Figures 7(a) and 7(b) and Figures 8(a) and 8(b), respectively. It expresses the strong variation of process variables Vab.21 and Vab.13, actually.The changing of processVab.21 in actual TE.
(a)
ProcessVab.21 (reactor coolant temperature) in Fault9
(b)
ProcessVab.21 (reactor coolant temperature) in Fault11The changing of processVab.13 in actual TE.
(a)
ProcessVab.13 (product separation pressure) in Fault9
(b)
ProcessVab.13 (product separation pressure) in Fault11According to the sequence of each process variable, the different feature sets are constructed as {Vab.21}, {Vab.21, Vab.13}, {Vab.21, Vab.13, Vab.9}, and so on. Nonlinear pattern classification of Fault9 and Fault11 is tested with support vector machine (SVM), which is widely used in pattern recognition. The parameters of SVM are optimized with cross-validation c=2035 and g=1024. With the above variable sets, the accuracy of fault separation between Fault9 and Fault11 is successively tested. The results are shown in Figure 9 and Table 4. It reveals that the separation accuracy becomes lower when the considered variables increase.Table 4
The accuracy with different feature sets with testing data.
Feature set
Combination of variables
Accuracy
Set1
Vab.21
83.521%
Set2
Vab.21, Vab.13
85.731%
Set3
Vab.21, Vab.13, Vab.9
89.652%
Set4
Vab.21, Vab.13, Vab.9, Vab.16
92.123%
Set5
Vab.21, Vab.13, Vab.9, Vab.16, Vab.7
94.547%
Set6
Vab.21, Vab.13, Vab.9, Vab.16, Vab.7, Vab.20
92.532%
⋮
⋮
⋮
Set53
Vab.21, Vab.13, Vab.9, Vab.16, Vab.7, Vab.20, … Vab.42
72.119%Figure 9
The accuracy with different feature sets to indentify Fault9 and Fault11 with testing data.From the above results, we conclude that(1) if all the 53 process variables were used to separate Fault9 and Fault 11, right ratio is merely 72.12%. It indicates that not all of the variables are directly related to certain fault. Some redundancy or irrelevant variables may decrease the classification accuracy and must be eliminated. (2) If the feature were selected as the first five process variables {Vab.21, Vab.13, Vab.9, Vab.16, Vab.7}, the accuracy increases to the highest as 94.55%. It means that the above five process variables are key to the fault separation. (3) If the model should be simplified at most, the process variable {Vab.21} is the best feature variable. We can recognize Fault9 and Fault11 according to the process changing of Vab.21.On the other hand, Fault9 stands for the random disturbance toD feed temperature. Fault11 is random disturbance to reactor cooling water inlet temperature. While {Vab.21, Vab.13, Vab.9, Vab.16, Vab.7} are the reactor coolant temperature, product separation pressure, reactor temperature, stripper pressure, reactor pressure, respectively, it is easy to see that the five selected variables are fairly relative to Fault9 and Fault11. The simulation results keep pace with the reality.


## 5. Conclusions

Nonlinear redundancy and multicollinearity variables can decrease the accuracy in classifier that must be eliminated. For the problem, FNN in MKFDA subspace is studied in the paper. Nonlinear variables are projected to a new linear higher dimension subspace with single-kernel fisher discriment analysis to get optimal classification with the intra-class nearest and inter-class farthest as most as possible. Furthermore, conventional single-kernel KFDA is expanded to multikernel method to solve the influence of each process variable with different distribution function. In order to reduce the higher dimension emerging in multi-KFDA subspace, FNN is composed to recognize the importance of each process variables on faults. According to simulation results in TE process, original variables are reduced to 5 in this paper, and the accuracy of tested right ratio reaches to 94.55% compared with tested right ratio 72.12% in the classifier between Fault9 and Fault11.



---

*Source: 729763-2014-02-04.xml*