# Fault Isolation for Nonlinear Systems Using Flexible Support Vector Regression

Authors: Yufang Liu; Bin Jiang; Hui Yi; Cuimei Bo
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

While support vector regression is widely used as both a function approximating tool and a residual generator for nonlinear system fault isolation, a drawback for this method is the freedom in selecting model parameters. Moreover, for samples with discordant distributing complexities, the selection of reasonable parameters is even impossible. To alleviate this problem we introduce the method of flexible support vector regression (F-SVR), which is especially suited for modelling complicated sample distributions, as it is free from parameters selection. Reasonable parameters for F-SVR are automatically generated given a sample distribution. Lastly, we apply this method in the analysis of the fault isolation of high frequency power supplies, where satisfactory results have been obtained.


---

## Body

## 1. Introduction

With the increasing use of complex systems, there has been great interest in the development of techniques to fault isolations. Generally, the major approaches for fault isolation can be divided into two categories, namely, model-based and data-driven techniques. The fundamental aspect of a model-based fault isolation is a process model that runs parallel to the process [1]. With traditional methods like observers, approximating the function between state vectors and input/output vectors is successful due to precise mathematical modelling by the use of filters. While these methods have successfully modelled linear systems, when applied to nonlinear systems like chemical processing, precise devices, and aerodynamic systems, they often fail to construct a sufficient model because their mechanism models are hard to be formed. Model-based approaches have advantages in terms of on-board implementation considerations, but their reliability may decrease as the nonlinear system complexities increase [2].Therefore, data-driven techniques have been introduced to more accurately construct process models as these methods are free from the requirement to analytically derive equations for a given system, shown in Figure1. One feasible method is to use the artificial neural network (ANN). ANN utilizes experience risk minimization (ERM) principle to construct the process model, where the target function is numerically approximated by minimizing residuals between function estimates and outputs of the process data. Applications of ANN-based fault isolation have been widely addressed in the literature. For example, Sadough Vanini et al. [2] used the dynamic neural networks to isolate the fault of a dual spool gas turbine engine. Filippetti et al. [3] applied fuzzy-NN to the fault isolation of induction motor drives. However, the learning ability of ANN is dependent on the number of training samples. It requires massive samples to ensure the modelling performance. But in most practical applications, not many of fault samples can be acquired.Figure 1
Framework of data-driven fault isolation.More recently, the principle of structure risk minimization [4] has been introduced in fault isolation through the utilization of support vector regression (SVR) [5, 6] as it can provide more accurate results than using neural networks in condition of smaller training samples. It was constructed on the basis of statistics learning theory that provides the theoretical proofs of learning from finite samples. Much has been addressed in the literature where SVR shows superiorities to ANN in process modeling [7].However, the performance of SVR-based modelling is greatly affected by its parameters. Although SVR has been well studied and many remarkable achievements have been obtained, the theoretical estimation of regression parameter remains unsolved in the last decade. There is no general consensus on the selection of proper parameters, but only some practical recommendations on this issue. This greatly increases the difficulty for common operators to master the SVR-based approach. Moreover, in some complicated cases, there are even no reasonable parameter settings that could be found. A rigorous selection of regression parameters can lead to the overlearning of training samples, while slack selections can lead to underlearning. There exist no parameters that yield good trade-off between overlearning and underlearning.In this paper we introduce a flexible SVR (F-SVR) approach [8] to more accurately implement models that construct different residual generators for fault isolation. By automatically dividing training samples into several regions, this method is not only free from parameter selection, but also able to learn well and to generalize well for complicated cases.


## 2. Problem Statement

Support vector regression (SVR) is a process modeling tool that approximates the function between inputs and outputs:(1)
y
=
f
(
x
)
=
w
x
+
b
.Herex and y represent the input and output vectors, respectively, f is the modeled function, w is its weight vector, and b is the bias decided by the vector w.The SVR-based modelling can be viewed as a process of finding the optimal weight vectorw
0 with a proper parameter vector α
0 for a given data set {
(
x
1
,
y
1
)
,
…
,
(
x
n
,
y
n
)
}:
(2)
(
α
0
,
w
0
)
=
arg
min:
α
,
w
R
S
R
M
(
α
,
w
)
=
arg
min:
α
,
w
Residual
+
ϕ
(
w
)
=
arg
min:
α
,
w
∫
T
o
T
k
‍
L
(
y
,
f
(
x
,
α
,
w
)
)
p
(
x
,
y
)
d
x
d
y
,
where L
(
y
i
,
f
(
x
i
,
w
,
α
)
)
=
C
i
·
|
y
i
-
(
∑
i
=
1
l
β
i
K
(
x
,
x
i
)
+
b
)
|
ε is the loss function, p
(
x
,
y
) is the unknown joint distribution of x and y, C
i is the regularized parameter, ε is the insensitive parameter, K
(
·
) is the kernel function, ϕ
(
w
) denotes the generalization ability for the regression, and α
=
{
C
,
ε
,
K
(
·
)
}. The optimal weight vector w
0 could be obtained by Lagrangian approaches. Thus the core problem for the SVR modeling is the selection of parameters.The framework for SVR-based fault isolation is shown in Figure2. Different operating models are constructed by SVRs with given parameters. The residuals between estimated outputs and real outputs are generated for fault isolation.Figure 2
Scheme of SVR-based fault isolation.However, the selection of SVR parameters is not easy. With the fixed regression parameterα, the drawback of SVR is the hardness of the trade-off between overlearning and underlearning. Moreover, in some complicated cases, even no reasonable parameters could be found. As shown in Figure 2, due to the different complexities of sample distributions, the requirements of parameters are discordant. If a rigorous parameter α
1 is selected, the regression is overlearning in region A. In contrast, if a slack parameter α
2 is selected, the regression fails to learn in region B (underlearning). No parameter that can adequately fit all of the cases exists. Consequently, we advocate using the F-SVR approach in order to overcome this drawback [8], as the regression parameter is automatically generated and is variable instead of fixed (Figure 3).Figure 3
The drawback for SVR modeling.


## 3. Fault Isolation Using Flexible Support Vector Regression (F-SVR)

### 3.1. The Principle of Flexible Support Vector Regression

We proposed a parameter-free algorithm for process modeling, namely, flexible support vector regression. The F-SVR attempts to divide the training samples intok regions according to the distribution complexity, and for the ith region, parameter α
i is generated. By minimization (3), the function between x and y is approximated:
(3)
R
(
w
,
α
)
=
∬
T
1
L
(
y
,
f
(
x
,
w
,
α
1
)
)
p
(
x
,
y
)
d
x
d
y
+
∬
T
2
L
(
y
,
f
(
x
,
w
,
α
2
)
)
p
(
x
,
y
)
d
x
d
y
+
⋯
+
∬
T
k
L
(
y
,
f
(
x
,
w
,
α
k
)
)
p
(
x
,
y
)
d
x
d
y
.With given parameters, (3) can be minimized by solving a q
u
a
d
r
a
t
i
c
p
r
o
g
r
a
m
m
i
n
g  (QP) problem. Supposing the training samples were divided into two areas, (
x
i
,
y
i
)
i
=
1
q and (
x
i
,
y
i
)
i
=
q
+
1
l, and α
=
{
C
,
σ
,
ε
} are parameters for the two areas, the minimization of (3) could be termed as
(4)
min:
1
2
(
w
,
w
)
+
C
1
(
∑
i
=
1
q
‍
ξ
i
*
+
∑
i
=
1
q
‍
ξ
i
)
+
C
2
(
∑
i
=
q
+
1
l
‍
ξ
i
*
2
+
∑
i
=
q
+
1
l
‍
ξ
i
2
)
S.t.
y
i
-
(
w
,
x
i
)
-
b
≤
ε
+
ξ
i
*
(
w
,
x
i
)
+
b
-
y
i
≤
ε
+
ξ
i
ξ
i
*
≥
0
;
ξ
i
≥
0
.And the Lagrangian for (4) is
(5)
L
(
w
,
ξ
*
,
ξ
,
α
*
,
α
,
γ
*
,
γ
)
=
1
2
(
w
,
w
)
+
C
1
(
∑
i
=
1
q
‍
ξ
i
*
+
∑
i
=
1
q
‍
ξ
i
)
+
C
2
(
∑
i
=
q
+
1
l
‍
ξ
i
*
2
+
∑
i
=
q
+
1
l
‍
ξ
i
2
)
-
∑
i
=
1
l
‍
α
i
[
y
i
-
w
·
x
i
-
b
+
ε
+
ξ
i
]
-
∑
i
=
1
l
‍
α
i
*
[
w
·
x
i
+
b
-
y
i
+
ε
+
ξ
i
*
]
-
∑
i
=
1
l
‍
(
γ
i
*
ξ
i
*
+
γ
i
ξ
i
)
.Taking the partial derivative for (5), we get
(6)
∂
L
∂
w
=
w
+
∑
i
=
1
l
‍
α
i
x
i
-
∑
i
=
1
l
‍
α
i
*
x
i
=
0
,
∂
L
∂
b
=
∑
i
=
1
l
‍
(
α
i
-
α
i
*
)
=
0
,
∂
L
∂
ξ
i
=
{
C
1
-
α
i
-
γ
i
,
1
≤
i
≤
q
=
0
,
2
C
2
ξ
i
-
α
i
-
γ
i
,
q
<
i
≤
l
∂
L
∂
ξ
i
*
=
{
C
1
-
α
i
*
-
γ
i
*
,
1
≤
i
≤
q
=
0
.
2
C
2
ξ
i
*
-
α
i
*
-
γ
i
*
,
q
<
i
≤
lMake the dual problem for (4):
(7)
max:
-
ε
∑
i
=
1
l
‍
(
α
i
*
+
α
i
)
+
∑
i
=
1
l
‍
y
i
(
α
i
*
-
α
i
)
-
1
2
∑
i
,
j
=
1
l
‍
(
α
i
*
-
α
i
)
(
α
j
*
-
α
j
)
(
x
i
·
x
j
)
-
∑
i
=
q
+
1
l
‍
1
4
C
2
(
α
i
2
+
α
i
*
2
)
S.t.
∑
i
=
1
l
‍
(
α
i
*
-
α
i
)
=
0
;
0
≤
α
i
*
≤
C
i
;
0
≤
α
i
≤
C
i
,
where
(8)
C
i
=
{
C
1
,
1
≤
i
≤
q
C
2
,
q
+
1
≤
i
≤
l
.
As α
·
α
*
≡
0, (7) could be written with the following form:
(9)
min:
1
2
(
α
-
α
*
)
T
Q
(
α
-
α
*
)
+
y
T
(
α
-
α
*
)
+
ε
e
T
(
α
+
α
*
)
-
(
α
-
α
*
)
T
v
(
α
-
α
*
)
,
where Q
=
K
(
x
i
,
x
j
), v
=
[
0
,
…
,
(
1
/
4
)
C
q
+
1
-
1
,
…
,
(
1
/
4
)
C
l
-
1
], and e is the unit vector. Further, the regression could be written as the QP problem
(10)
min:
1
2
[
α
T
,
(
α
*
)
T
]
[
D
-
D
-
D
D
]
[
α
α
*
]
+
c
T
[
α
α
*
]
S.t.
z
T
[
α
α
*
]
=
0
;
0
≤
α
i
≤
C
i
;
0
≤
α
i
*
≤
C
i
,
where D
=
Q
-
v, c
=
[
ε
e
+
y
,
ε
e
-
y
], and
(11)
z
i
=
{
1
,
i
=
1,2
…
q
-
1
,
i
=
q
+
1
,
…
,
l
.
This form of QP problem can be solved by an active set method [9]. The feasibility for complicated cases that more regions are divided can similarly be proved.


### 3.2. Detailed Process of F-SVR Modeling

F-SVR modeling contains three major steps, shown in Figure4.Figure 4
The flowchart for F-SVR modeling.Step 1 (sample divisions).
This section shows howT
i in (3) is determined. Given training samples (
x
i
,
y
i
)
i
=
1
n, formula (12) is utilized to estimate the distribution complexity
(12)
C
P
=
∑
i
=
1
n
-
1
‍
|
A
i
|
/
(
n
-
1
)
∑
i
=
1
n
-
2
‍
cos
⁡
θ
i
/
(
n
-
2
)
=
n
-
2
n
-
1
·
∑
i
=
1
n
-
1
‍
|
A
i
|
∑
i
=
1
n
-
2
‍
cos
⁡
θ
i
,
where A
i
=
(
x
i
+
1
,
y
i
+
1
)
-
(
x
i
,
y
i
) and cos
⁡
θ
i
=
|
A
i
·
A
i
+
1
|
/
(
|
A
i
|
·
|
A
i
+
1
|
). Supposing the training samples have been divided into k areas, formula (13) is implemented to evaluate the performance of division
(13)
S
=
∑
i
=
1
k
‍
-
C
i
∑
j
=
1
k
‍
C
j
log
⁡
C
i
∑
j
=
1
k
‍
C
j
.
The samples are divided randomly for several times, and the division with the smallest S value is treated as the best division:
(14)
j
*
=
arg
min
⁡
j
S
j
,
j
=
1,2
,
…
m
,
where m is number of times that random division is made and n is the number of training samples. And in this paper, we set m
=
(
k
*
n
)
/
10. The ith region denotes x
∈
T
i.Step 2 (setting parameters for each region).
Once the best divisionj
* is obtained, the C
P values for all areas C
P
j
*
=
{
C
P
j
*
1
,
C
P
j
*
2
,
…
,
C
P
j
*
k
} can also be obtained. In flexible support vector regression approach, the following empirical formulas are given to set the hyperparameter α
i
=
{
ε
i
,
C
i
,
σ
i
}:
(15)
α
i
:
{
ε
i
=
0.5
*
st
d
2
(
X
i
)
C
P
j
*
i
C
i
=
1000
C
P
j
*
i
∑
i
=
1
k
‍
C
P
j
*
i
σ
i
=
5
C
P
j
*
i
,
where X
i
=
{
x
∖
x
∈
the
i
th
region
}.Remark 1.
The empirical setting of parameters for each region is referred to in Cherkassky’s work [10] in 2004.Step 3 (function approximation using selected support vectors).
We use the conventional SVR with parametersα
i to extract informative samples for the ith region. As shown in Figure 6(a), the red samples are selected as support vectors (SVs). If m samples are selected as SVs for the training set (
x
i
,
y
i
)
i
=
1
n, the regression problem f
(
x
)
=
∑
i
=
1
n
β
i
K
(
x
,
x
i
)
+
b
0 can be approximated by the regression problem of the SVs [8]:
(16)
f
(
x
)
=
∑
i
=
1
m
‍
γ
i
K
(
x
,
S
V
i
)
+
b
0
′
.
Thus, the minimization of (3) can be simplified as
(17)
w
*
=
arg
min:
w
∫
T
o
T
k
‍
L
(
y
,
f
(
S
V
,
α
,
w
)
)
p
(
S
V
,
y
)
d
S
V
·
d
y
.
In (17), rigorous parameters α
=
{
0,1000,0.01
} are set to ensure the learning ability of regression. This problem can be solved by using the Lagrangian method in the same way that is used in conventional SVR (Figure 5).
As is shown in Figure6(b), F-SVR (unlike conventional SVR) successfully approximates the distribution function without overlearning or underlearning. As we mentioned earlier, F-SVR is free from the issues that can arise in the manual selection of parameters as all of the parameters are generated automatically.Figure 5
Setting different parameters for different regions.F-SVR modeling for complicated distributions.
(a)
The selection of support vectors
(b)
Regression with selected SVs


### 3.3. The Basic Scheme of F-SVR-Based Fault Isolation

Fault isolation algorithms attempt to reveal which fault is occurring in the operating system. In our method, we determine which model of the system is most likely to be accurate by initially constructing the modes for all faulty statuses and then calculating the deviations between the real outputs and all of the model outputs. As is shown in Figure7, once the best model has been located, the fault type can then be isolated.Figure 7
Scheme of F-SVR-based fault isolation.Compared with conventional SVR-based fault isolation, the most significant contribution of our work is that we have alleviated the problem of parameter setting. What is required for F-SVR-based fault isolation are only process samples for different operating cases.The detailed process of F-SVR-based fault isolation is as follows. Given(
x
i
,
y
i
)
i
=
1
k
,
x
i
,
y
i
∈
R
n as the training samples from k different operating statuses concluding the normal status and all faulty statuses, where (
x
i
,
y
i
) represents the training samples for the ith status, the function G
i
(
·
) between input vector x
i and output vector y
i is initially approximated by the F-SVR method. After the training samples (
x
,
y
) construct the input/output models for all of the statuses, they are sent to these models to generate the residuals between the real outputs and the model outputs, thereby forming the residual vector:
(18)
Residual
i
=
y
-
G
i
(
x
)
,
i
=
1
,
…
,
k
,
r
=
[
Residual
_
1
⋮
Residual
_
k
]
k
×
n
.We define a functionR
(
i
) to measure the deviation between the real output and the model output of the ith status
(19)
R
(
i
)
=
∑
‍
|
y
-
G
i
(
x
)
|
.Faults can be isolated by analysis of the residual vector. In this paper, we simply regard the testing samples belonging to thei
*th status:
(20)
i
*
=
arg
min
⁡
i
R
(
i
)
,
i
=
1
,
…
,
k
.


## 3.1. The Principle of Flexible Support Vector Regression

We proposed a parameter-free algorithm for process modeling, namely, flexible support vector regression. The F-SVR attempts to divide the training samples intok regions according to the distribution complexity, and for the ith region, parameter α
i is generated. By minimization (3), the function between x and y is approximated:
(3)
R
(
w
,
α
)
=
∬
T
1
L
(
y
,
f
(
x
,
w
,
α
1
)
)
p
(
x
,
y
)
d
x
d
y
+
∬
T
2
L
(
y
,
f
(
x
,
w
,
α
2
)
)
p
(
x
,
y
)
d
x
d
y
+
⋯
+
∬
T
k
L
(
y
,
f
(
x
,
w
,
α
k
)
)
p
(
x
,
y
)
d
x
d
y
.With given parameters, (3) can be minimized by solving a q
u
a
d
r
a
t
i
c
p
r
o
g
r
a
m
m
i
n
g  (QP) problem. Supposing the training samples were divided into two areas, (
x
i
,
y
i
)
i
=
1
q and (
x
i
,
y
i
)
i
=
q
+
1
l, and α
=
{
C
,
σ
,
ε
} are parameters for the two areas, the minimization of (3) could be termed as
(4)
min:
1
2
(
w
,
w
)
+
C
1
(
∑
i
=
1
q
‍
ξ
i
*
+
∑
i
=
1
q
‍
ξ
i
)
+
C
2
(
∑
i
=
q
+
1
l
‍
ξ
i
*
2
+
∑
i
=
q
+
1
l
‍
ξ
i
2
)
S.t.
y
i
-
(
w
,
x
i
)
-
b
≤
ε
+
ξ
i
*
(
w
,
x
i
)
+
b
-
y
i
≤
ε
+
ξ
i
ξ
i
*
≥
0
;
ξ
i
≥
0
.And the Lagrangian for (4) is
(5)
L
(
w
,
ξ
*
,
ξ
,
α
*
,
α
,
γ
*
,
γ
)
=
1
2
(
w
,
w
)
+
C
1
(
∑
i
=
1
q
‍
ξ
i
*
+
∑
i
=
1
q
‍
ξ
i
)
+
C
2
(
∑
i
=
q
+
1
l
‍
ξ
i
*
2
+
∑
i
=
q
+
1
l
‍
ξ
i
2
)
-
∑
i
=
1
l
‍
α
i
[
y
i
-
w
·
x
i
-
b
+
ε
+
ξ
i
]
-
∑
i
=
1
l
‍
α
i
*
[
w
·
x
i
+
b
-
y
i
+
ε
+
ξ
i
*
]
-
∑
i
=
1
l
‍
(
γ
i
*
ξ
i
*
+
γ
i
ξ
i
)
.Taking the partial derivative for (5), we get
(6)
∂
L
∂
w
=
w
+
∑
i
=
1
l
‍
α
i
x
i
-
∑
i
=
1
l
‍
α
i
*
x
i
=
0
,
∂
L
∂
b
=
∑
i
=
1
l
‍
(
α
i
-
α
i
*
)
=
0
,
∂
L
∂
ξ
i
=
{
C
1
-
α
i
-
γ
i
,
1
≤
i
≤
q
=
0
,
2
C
2
ξ
i
-
α
i
-
γ
i
,
q
<
i
≤
l
∂
L
∂
ξ
i
*
=
{
C
1
-
α
i
*
-
γ
i
*
,
1
≤
i
≤
q
=
0
.
2
C
2
ξ
i
*
-
α
i
*
-
γ
i
*
,
q
<
i
≤
lMake the dual problem for (4):
(7)
max:
-
ε
∑
i
=
1
l
‍
(
α
i
*
+
α
i
)
+
∑
i
=
1
l
‍
y
i
(
α
i
*
-
α
i
)
-
1
2
∑
i
,
j
=
1
l
‍
(
α
i
*
-
α
i
)
(
α
j
*
-
α
j
)
(
x
i
·
x
j
)
-
∑
i
=
q
+
1
l
‍
1
4
C
2
(
α
i
2
+
α
i
*
2
)
S.t.
∑
i
=
1
l
‍
(
α
i
*
-
α
i
)
=
0
;
0
≤
α
i
*
≤
C
i
;
0
≤
α
i
≤
C
i
,
where
(8)
C
i
=
{
C
1
,
1
≤
i
≤
q
C
2
,
q
+
1
≤
i
≤
l
.
As α
·
α
*
≡
0, (7) could be written with the following form:
(9)
min:
1
2
(
α
-
α
*
)
T
Q
(
α
-
α
*
)
+
y
T
(
α
-
α
*
)
+
ε
e
T
(
α
+
α
*
)
-
(
α
-
α
*
)
T
v
(
α
-
α
*
)
,
where Q
=
K
(
x
i
,
x
j
), v
=
[
0
,
…
,
(
1
/
4
)
C
q
+
1
-
1
,
…
,
(
1
/
4
)
C
l
-
1
], and e is the unit vector. Further, the regression could be written as the QP problem
(10)
min:
1
2
[
α
T
,
(
α
*
)
T
]
[
D
-
D
-
D
D
]
[
α
α
*
]
+
c
T
[
α
α
*
]
S.t.
z
T
[
α
α
*
]
=
0
;
0
≤
α
i
≤
C
i
;
0
≤
α
i
*
≤
C
i
,
where D
=
Q
-
v, c
=
[
ε
e
+
y
,
ε
e
-
y
], and
(11)
z
i
=
{
1
,
i
=
1,2
…
q
-
1
,
i
=
q
+
1
,
…
,
l
.
This form of QP problem can be solved by an active set method [9]. The feasibility for complicated cases that more regions are divided can similarly be proved.


## 3.2. Detailed Process of F-SVR Modeling

F-SVR modeling contains three major steps, shown in Figure4.Figure 4
The flowchart for F-SVR modeling.Step 1 (sample divisions).
This section shows howT
i in (3) is determined. Given training samples (
x
i
,
y
i
)
i
=
1
n, formula (12) is utilized to estimate the distribution complexity
(12)
C
P
=
∑
i
=
1
n
-
1
‍
|
A
i
|
/
(
n
-
1
)
∑
i
=
1
n
-
2
‍
cos
⁡
θ
i
/
(
n
-
2
)
=
n
-
2
n
-
1
·
∑
i
=
1
n
-
1
‍
|
A
i
|
∑
i
=
1
n
-
2
‍
cos
⁡
θ
i
,
where A
i
=
(
x
i
+
1
,
y
i
+
1
)
-
(
x
i
,
y
i
) and cos
⁡
θ
i
=
|
A
i
·
A
i
+
1
|
/
(
|
A
i
|
·
|
A
i
+
1
|
). Supposing the training samples have been divided into k areas, formula (13) is implemented to evaluate the performance of division
(13)
S
=
∑
i
=
1
k
‍
-
C
i
∑
j
=
1
k
‍
C
j
log
⁡
C
i
∑
j
=
1
k
‍
C
j
.
The samples are divided randomly for several times, and the division with the smallest S value is treated as the best division:
(14)
j
*
=
arg
min
⁡
j
S
j
,
j
=
1,2
,
…
m
,
where m is number of times that random division is made and n is the number of training samples. And in this paper, we set m
=
(
k
*
n
)
/
10. The ith region denotes x
∈
T
i.Step 2 (setting parameters for each region).
Once the best divisionj
* is obtained, the C
P values for all areas C
P
j
*
=
{
C
P
j
*
1
,
C
P
j
*
2
,
…
,
C
P
j
*
k
} can also be obtained. In flexible support vector regression approach, the following empirical formulas are given to set the hyperparameter α
i
=
{
ε
i
,
C
i
,
σ
i
}:
(15)
α
i
:
{
ε
i
=
0.5
*
st
d
2
(
X
i
)
C
P
j
*
i
C
i
=
1000
C
P
j
*
i
∑
i
=
1
k
‍
C
P
j
*
i
σ
i
=
5
C
P
j
*
i
,
where X
i
=
{
x
∖
x
∈
the
i
th
region
}.Remark 1.
The empirical setting of parameters for each region is referred to in Cherkassky’s work [10] in 2004.Step 3 (function approximation using selected support vectors).
We use the conventional SVR with parametersα
i to extract informative samples for the ith region. As shown in Figure 6(a), the red samples are selected as support vectors (SVs). If m samples are selected as SVs for the training set (
x
i
,
y
i
)
i
=
1
n, the regression problem f
(
x
)
=
∑
i
=
1
n
β
i
K
(
x
,
x
i
)
+
b
0 can be approximated by the regression problem of the SVs [8]:
(16)
f
(
x
)
=
∑
i
=
1
m
‍
γ
i
K
(
x
,
S
V
i
)
+
b
0
′
.
Thus, the minimization of (3) can be simplified as
(17)
w
*
=
arg
min:
w
∫
T
o
T
k
‍
L
(
y
,
f
(
S
V
,
α
,
w
)
)
p
(
S
V
,
y
)
d
S
V
·
d
y
.
In (17), rigorous parameters α
=
{
0,1000,0.01
} are set to ensure the learning ability of regression. This problem can be solved by using the Lagrangian method in the same way that is used in conventional SVR (Figure 5).
As is shown in Figure6(b), F-SVR (unlike conventional SVR) successfully approximates the distribution function without overlearning or underlearning. As we mentioned earlier, F-SVR is free from the issues that can arise in the manual selection of parameters as all of the parameters are generated automatically.Figure 5
Setting different parameters for different regions.F-SVR modeling for complicated distributions.
(a)
The selection of support vectors
(b)
Regression with selected SVs


## 3.3. The Basic Scheme of F-SVR-Based Fault Isolation

Fault isolation algorithms attempt to reveal which fault is occurring in the operating system. In our method, we determine which model of the system is most likely to be accurate by initially constructing the modes for all faulty statuses and then calculating the deviations between the real outputs and all of the model outputs. As is shown in Figure7, once the best model has been located, the fault type can then be isolated.Figure 7
Scheme of F-SVR-based fault isolation.Compared with conventional SVR-based fault isolation, the most significant contribution of our work is that we have alleviated the problem of parameter setting. What is required for F-SVR-based fault isolation are only process samples for different operating cases.The detailed process of F-SVR-based fault isolation is as follows. Given(
x
i
,
y
i
)
i
=
1
k
,
x
i
,
y
i
∈
R
n as the training samples from k different operating statuses concluding the normal status and all faulty statuses, where (
x
i
,
y
i
) represents the training samples for the ith status, the function G
i
(
·
) between input vector x
i and output vector y
i is initially approximated by the F-SVR method. After the training samples (
x
,
y
) construct the input/output models for all of the statuses, they are sent to these models to generate the residuals between the real outputs and the model outputs, thereby forming the residual vector:
(18)
Residual
i
=
y
-
G
i
(
x
)
,
i
=
1
,
…
,
k
,
r
=
[
Residual
_
1
⋮
Residual
_
k
]
k
×
n
.We define a functionR
(
i
) to measure the deviation between the real output and the model output of the ith status
(19)
R
(
i
)
=
∑
‍
|
y
-
G
i
(
x
)
|
.Faults can be isolated by analysis of the residual vector. In this paper, we simply regard the testing samples belonging to thei
*th status:
(20)
i
*
=
arg
min
⁡
i
R
(
i
)
,
i
=
1
,
…
,
k
.


## 4. Experiments and Real Applications

### 4.1. Numerical Experiments

In our first attempt to validate our method by a numerical experiment, we used a data set with complicated distributions. White noise (SNR = 30 db) is added to the analytical equation shown in (21), where the training set consists of 600 samples extracted from x
∈
(
-
1,1
]. In order to approximate the distribution, both F-SVR and least-square SVR (LS-SVR) [11] are implemented:
(21)
y
1
=
(
x
-
0.5
)
2
+
4
sin
(
3
x
2
)
+
x
,
y
2
=
4
x
(
1
-
x
)
(
2
sin
(
30
x
+
24
)
+
3
)
,
t
=
(
1
-
x
)
2
,
y
=
y
1
(
1
-
t
)
+
y
2
*
t
;
x
∈
(
-
1,1
]
.To provide a fair comparison, the parameters for LS-SVR were optimized first using a grid search strategy. The evaluating index for LS-SVR is(22)
F
=
d
N
·
∑
i
=
1
N
‍
(
y
^
i
-
y
i
)
2
N
-
1
,
where d is the number of SVs and y
^ is the function output of LS-SVR. A small F value implies that the regression produces generalization and accurate learning. As is shown in Figure 8, the best parameter set for LS-SVR is {
γ
=
10, σ
2
=
0.2
}.Figure 8
Parameter settings for LS-SVR.The performance of the two methods is shown in Figure9. It is clear in this case that F-SVR is a more accurate method for modeling data compared to the well-known LS-SVR.Figure 9
Comparison of learning ability.


### 4.2. Fault Isolation for High Frequency Power Supply

The high frequency power supply (HFPS) is a nonlinear device that has been widely used in power plants for dedusting purpose. As its structure is highly complicated, its precise model is usually hard to be constructed using classical analytical approaches [12–18]. Thus, data-driven approaches are utilized for process modeling and fault isolation. However, when applied to different power plants, the HFPS yields very different input/output functions due to the change of its loads and working environments. This means that there is no general process model for HFPS in all conditions. For each HFPS that has been installed, a particular process model should be constructed. Therefore, for conventional SVR-based fault isolation, experienced operators are required to select the modeling parameters at site.In this section, F-SVR was applied in order to isolate the fault of the high frequency power supply. Three operating cases were investigated: normal status, overcurrent fault, and learning excitation fault. Data for the3 operating cases of HFPS was prepared in Figure 10 and an overcurrent fault sample was used as the input data for testing in Figure 11.Process data for different operating cases of HFPS.
(a)
HFPS
(b)
Process data for normal status
(c)
Process data for overcurrent fault
(d)
Process data for learning excitation faultFigure 11
The testing samples.The basic scheme for HFPS fault isolation is designed in Figure12. Firstly, process data for each operating case is acquired; then, F-SVR is implemented to approximate the unknown function between input (time) and output (the first primary current) for each operating case. As the models for all operating cases have been established, the residual vectors can be generated and then by finding the model with the smallest residual the fault can be isolated.Figure 12
Fault isolation scheme for HFPS.In this experiment, F-SVR is implemented to approximate the functions between the input and the output, namely,G
1
(
x
), G
2
(
x
), and G
3
(
x
). As shown in Figure 13, the functions we obtained using F-SVR accurately describe the relationship between the input and the output of the unknown functions for the different operating cases without setting parameters.Function approximation using F-SVR and LS-SVR.
(a)
Function approximation for normal status:G
1
(
x
)
(b)
Function approximation for overcurrent fault:G
2
(
x
)
(c)
Function approximation for leaning excitation fault:G
3
(
x
)SinceG
1
(
x
), G
2
(
x
), and G
3
(
x
) were already obtained by the F-SVR method, the residuals could be generated using (12). The corresponding residuals are recorded in Table 1. According to Table 1, the testing sample belongs to the overcurrent fault (G
2
(
x
) yields the smallest residual). Based on prior knowledge of the testing sample, the diagnostic result is consistent and shows the feasibility of our method.Table 1
Diagnostic performance of F-SVR and LS-SVR.
Operating status
Residuals (R
i)
Number of SVs (d)
Parameters setting
F-SVR
LS-SVR
F-SVR
LS-SVR
F-SVR
LS-SVR
Normal (G
1)
15.912
16.231
26
23
Auto
Manual
Overcurrent (G
2)
11.296
14.131
38
28
Auto
Manual
Learning excitation (G
3)
17.104
17.932
23
20
Auto
ManualThe LS-SVR method [11, 19] is also implemented to give a comparison. As is shown in Table 1, the LS-SVR method also makes a correct diagnosis and it has a better ability to generalize than our method (the number of SVs is smaller). However, our method yields a smaller value of residuals. This implies that our method produces a better modeling accuracy. Most importantly, all parameters are required to be selected manually in LS-SVR but are selected automatically in F-SVR.


## 4.1. Numerical Experiments

In our first attempt to validate our method by a numerical experiment, we used a data set with complicated distributions. White noise (SNR = 30 db) is added to the analytical equation shown in (21), where the training set consists of 600 samples extracted from x
∈
(
-
1,1
]. In order to approximate the distribution, both F-SVR and least-square SVR (LS-SVR) [11] are implemented:
(21)
y
1
=
(
x
-
0.5
)
2
+
4
sin
(
3
x
2
)
+
x
,
y
2
=
4
x
(
1
-
x
)
(
2
sin
(
30
x
+
24
)
+
3
)
,
t
=
(
1
-
x
)
2
,
y
=
y
1
(
1
-
t
)
+
y
2
*
t
;
x
∈
(
-
1,1
]
.To provide a fair comparison, the parameters for LS-SVR were optimized first using a grid search strategy. The evaluating index for LS-SVR is(22)
F
=
d
N
·
∑
i
=
1
N
‍
(
y
^
i
-
y
i
)
2
N
-
1
,
where d is the number of SVs and y
^ is the function output of LS-SVR. A small F value implies that the regression produces generalization and accurate learning. As is shown in Figure 8, the best parameter set for LS-SVR is {
γ
=
10, σ
2
=
0.2
}.Figure 8
Parameter settings for LS-SVR.The performance of the two methods is shown in Figure9. It is clear in this case that F-SVR is a more accurate method for modeling data compared to the well-known LS-SVR.Figure 9
Comparison of learning ability.


## 4.2. Fault Isolation for High Frequency Power Supply

The high frequency power supply (HFPS) is a nonlinear device that has been widely used in power plants for dedusting purpose. As its structure is highly complicated, its precise model is usually hard to be constructed using classical analytical approaches [12–18]. Thus, data-driven approaches are utilized for process modeling and fault isolation. However, when applied to different power plants, the HFPS yields very different input/output functions due to the change of its loads and working environments. This means that there is no general process model for HFPS in all conditions. For each HFPS that has been installed, a particular process model should be constructed. Therefore, for conventional SVR-based fault isolation, experienced operators are required to select the modeling parameters at site.In this section, F-SVR was applied in order to isolate the fault of the high frequency power supply. Three operating cases were investigated: normal status, overcurrent fault, and learning excitation fault. Data for the3 operating cases of HFPS was prepared in Figure 10 and an overcurrent fault sample was used as the input data for testing in Figure 11.Process data for different operating cases of HFPS.
(a)
HFPS
(b)
Process data for normal status
(c)
Process data for overcurrent fault
(d)
Process data for learning excitation faultFigure 11
The testing samples.The basic scheme for HFPS fault isolation is designed in Figure12. Firstly, process data for each operating case is acquired; then, F-SVR is implemented to approximate the unknown function between input (time) and output (the first primary current) for each operating case. As the models for all operating cases have been established, the residual vectors can be generated and then by finding the model with the smallest residual the fault can be isolated.Figure 12
Fault isolation scheme for HFPS.In this experiment, F-SVR is implemented to approximate the functions between the input and the output, namely,G
1
(
x
), G
2
(
x
), and G
3
(
x
). As shown in Figure 13, the functions we obtained using F-SVR accurately describe the relationship between the input and the output of the unknown functions for the different operating cases without setting parameters.Function approximation using F-SVR and LS-SVR.
(a)
Function approximation for normal status:G
1
(
x
)
(b)
Function approximation for overcurrent fault:G
2
(
x
)
(c)
Function approximation for leaning excitation fault:G
3
(
x
)SinceG
1
(
x
), G
2
(
x
), and G
3
(
x
) were already obtained by the F-SVR method, the residuals could be generated using (12). The corresponding residuals are recorded in Table 1. According to Table 1, the testing sample belongs to the overcurrent fault (G
2
(
x
) yields the smallest residual). Based on prior knowledge of the testing sample, the diagnostic result is consistent and shows the feasibility of our method.Table 1
Diagnostic performance of F-SVR and LS-SVR.
Operating status
Residuals (R
i)
Number of SVs (d)
Parameters setting
F-SVR
LS-SVR
F-SVR
LS-SVR
F-SVR
LS-SVR
Normal (G
1)
15.912
16.231
26
23
Auto
Manual
Overcurrent (G
2)
11.296
14.131
38
28
Auto
Manual
Learning excitation (G
3)
17.104
17.932
23
20
Auto
ManualThe LS-SVR method [11, 19] is also implemented to give a comparison. As is shown in Table 1, the LS-SVR method also makes a correct diagnosis and it has a better ability to generalize than our method (the number of SVs is smaller). However, our method yields a smaller value of residuals. This implies that our method produces a better modeling accuracy. Most importantly, all parameters are required to be selected manually in LS-SVR but are selected automatically in F-SVR.


## 5. Conclusions

SVR is one of the most efficient tools for fault diagnosis because it is able to accurately model a function between the input and the output using process data. However, even though the SVR approach has been utilized for over a decade, there is still no consensus within the community on how to adequately select regression parameters. Given that the F-SVR method offers an automatic selection for regression parameters, we chose it to implement the fault isolation for nonlinear systems. We demonstrated on both a numerical experiment and the fault isolation for HFPS that F-SVR is especially suited for cases that yield complicated sample distributions. This is because this method generates reasonable parameters for each region by dividing the training samples into different regions according to the sample distribution complexity. Based on this work, we hope that F-SVR will become more widely recognized as a preferred fault isolation for nonlinear systems.



---

*Source: 713018-2014-04-24.xml*