# Optimal Control for a Linear System Subject to a General ARIMA Disturbance

Authors: Hongyan Xie; Fangyi He
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

A novel run-to-run (R2R) control algorithm based on Kalman filter approach is proposed to deal with a linear system with
a general ARIMA(p,d,q) process, in the presence of measurement error and adjustment error together with a random initial bias.
We mathematically prove its optimality. The performance of the newly proposed controller and the proportional-integral (PI) controller is evaluated and compared under multiple scenarios through Monte Carlo simulations. Almost all the results reflect the new controller’s superiority.


---

## Body

## 1. Introduction and Motivation

Optimizing a sequence of actions to obtain some future goal is the general topic of control theory (see [1]). Many optimal control methods have been widely used in actuarial science (e.g., [2, 3]), production management (e.g., [4–6]), and quality management (e.g., [7–9]). Specifically, disturbance rejection is one of the major concerns in quality control problems, such as machine setup adjustment problems and R2R process control problems in semiconductor manufacturing (see [10, 11]). Grubbs [12] originally studies the machine setup adjustment problem and develops an optimal adjustment rule for a linear system with normal disturbances, which is referred to by Trietsch [13] as the “harmonic rule.” Vander Wiel et al. [14] give an optimal control algorithm for a linear system with ARMA(1,1) disturbances. For a linear system with IMA(1,1) disturbances, Ingolfsson and Sachs [15] and Box et al. [16] introduce the exponential weighted moving average (EWMA) control algorithm and prove its optimality; He et al. [17] recently develop an optimal control algorithm, the ARMA controller, for a linear system with a general ARMA(p
,
q) disturbance.In this paper, we will extend the results in [17] and develop an optimal control algorithm for a linear system with a general ARIMA(p
,
d
,
q) disturbance, where d
=
1,2
,
…. The ARIMA disturbance has been widely used to describe process dynamics (see [18–21]). Since ARIMA(p
,
d
,
q) processes can be used to model a large class of nonstationary disturbances, many machine setup problems and R2R control problems can be solved under this new framework. Similar to [17], the newly proposed controller will consider a more realistic case such that both measurement error and adjustment error exist, and the initial bias of the process is a random variable. We present the problem as follows.Supposing a process to be controlledy
t can be expressed as
(1)
y
t
=
g
t
+
n
t
+
e
t
,
where g
t is the state of y
t at time t; n
t is an ARIMA(p
,
d
,
q) process satisfying
(2)
ϕ
B
1
-
B
d
n
t
=
θ
B
z
t
,
t
=
0
,
±
1
,
±
2
,
…
,
where ϕ
(
z
)
=
1
-
ϕ
1
z
-
ϕ
2
z
2
-
⋯
-
ϕ
p
z
p, θ
(
z
)
=
1
+
θ
1
z
+
θ
2
z
2
+
⋯
+
θ
q
z
q, and z
t is a white noise process with mean 0 and variance σ
z
2, that is, z
t
~
WN
(
0
,
σ
z
2
); e
t is the measurement error for y
t and e
t
~
WN
(
0
,
σ
e
2
). B is the backward shift operator defined by
(3)
B
j
n
t
=
n
t
-
j
,
j
=
0,1
,
2
,
…
.
Neither g
t nor n
t can be measured or observed directly. At time t
≥
1, suppose we need to make an adjustment of magnitude Δ
g
t to g
t to bring the process output to target τ in the next run. That is,
(4)
g
t
+
1
=
g
t
+
Δ
g
t
+
w
t
,
where w
t is the adjustment error and w
t
~
WN
(
0
,
σ
w
2
). In practice, the process adjustment Δ
g
t is assumed to be done by adjusting one controllable factor, u
t, via the following model:
(5)
Δ
g
t
=
β
Δ
u
t
,
β
≠
0
,
where β is called the process gain and Δ
u
t
=
u
t
+
1
-
u
t. Assume E
e
t
w
t
=
E
w
t
z
t
=
E
z
t
e
t
=
0, t
=
0
,
±
1
,
±
2
,
…. The initial value g
0 is assumed to be a random variable with E
g
0
z
t
=
E
g
0
e
t
=
E
g
0
w
t
=
0, t
=
0
,
±
1
,
±
2
,
…, and E
(
g
0
)
=
g
^
0
,
Var
(
g
0
)
=
σ
g
2. Without loss of generality, the target τ is assumed to be 0 in the rest of this paper. For a positive time index T, we hope to determine the optimal Δ
u
t, t
=
0,1
,
…
,
T
-
1 that satisfy
(6)
min
⁡
Δ
u
t
,
t
=
0,1
,
…
,
T
-
1
E
0
∑
t
=
0
T
-
1
‍
y
t
+
1
2
,
where E
t
(
·
) is the expectation operator conditional on all the information until time t. Problem (6) is a finite-horizon problem. When T
→
∞, we get the infinite-horizon problem that searches the optimal Δ
u
t
,
t
=
0,1
,
… satisfying
(7)
min
⁡
Δ
u
t
,
t
=
0,1
,
…
E
0
∑
t
=
0
∞
‍
y
t
+
1
2
.
The purpose of this paper is to find the solutions to both problems (6) and (7).The rest of this paper is organized as follows. In Section2 we derive the state-space representation of the system and get the recursive estimation formulae of the system states using Kalman filter. In Section 3, we develop a control algorithm for the system and prove its optimality without normal distribution assumptions. We further give the implementation steps of the controller in practice. Simulation studies are done in Section 4 under multiple scenarios. Section 5 gives an illustrative example for the application of the control algorithm. Concluding remarks are included in Section 6.


## 2. The State-Space Representation

Letm
=
max
⁡
(
p
,
q
) and ψ
1
,
…
,
ψ
m be the coefficients of z
,
z
2
,
…
,
z
m in the power series expansion of θ
(
z
)
/
ϕ
(
z
), |
z
|
≤
1. Brockwell and Davis [22] gave a state-space representation for a general ARIMA(p
,
d
,
q) process. By extending their results we can get the following Theorem 1 easily.Theorem 1.
The linear system (1)–(5) has a state-space representation as follows:(8)
y
t
=
1
G
′
-
1
d
+
1
d
d
-
1
d
d
d
-
1
-
1
d
-
1
d
d
-
2
⋯
d
R
t
+
z
t
+
e
t
,
t
=
1,2
,
…
(9)
R
t
+
1
=
1
0
0
0
F
0
0
A
G
′
B
R
t
+
1
0
0
β
Δ
u
t
+
w
t
H
z
t
A
z
t
,
t
=
1,2
,
…
,where A and B are the d
×
1 and d
×
d matrices defined by A
=
B
=
1 if d
=
1 and
(10)
A
=
0
0
⋯
0
1
′
,
B
=
0
1
0
⋯
0
0
0
1
⋯
0
⋮
⋮
⋮
⋱
⋮
0
0
0
⋯
1
-
1
d
+
1
d
d
-
1
d
d
d
-
1
-
1
d
-
1
d
d
-
2
⋯
d
if d
>
1; G and F are the m
×
1 and m
×
m matrices defined by G
=
1, F
=
ϕ
1 if m
=
1 and
(11)
G
=
1
0
0
⋯
0
′
,
F
=
0
1
0
⋯
0
0
0
1
⋯
0
⋮
⋮
⋮
⋱
⋮
0
0
0
⋯
1
ϕ
m
ϕ
m
-
1
ϕ
m
-
2
⋯
ϕ
1
if m
>
1; H
=
ψ
1
ψ
2
⋯
ψ
m
-
1
ψ
m
′, N
t
-
1
=
n
t
-
d
,
…
,
n
t
-
1
′, X
t
=
X
t
-
m
+
1
,
…
,
X
t
′, R
t
=
g
t
X
t
′
N
t
-
1
'
′.A proof of Theorem1 is presented in Appendix A. For simplicity, in the rest of this paper we denote
(12)
h
≡
1
G
′
-
1
d
+
1
d
d
-
1
d
d
d
-
1
-
1
d
-
1
d
d
-
2
⋯
d
′
,
L
≡
1
0
0
0
F
0
0
A
G
′
B
,
l
≡
1
0
0
,
m
=
0
H
σ
z
2
A
σ
z
2
,
M
=
1
0
0
0
diag
H
0
0
0
diag
A
,
Σ
≡
σ
w
2
0
0
0
1
m
1
m
′
σ
z
2
1
m
1
d
′
σ
z
2
0
1
d
1
m
′
σ
z
2
1
d
1
d
′
σ
z
2
,
where 1
k is defined to be a k
×
1 vector of ones.That the problems of optimal control and state estimation can be decoupled in certain cases is one of the most fundamental principles in feedback control theory. This is known as theseparation principle (see [23]). The Kalman filter produces the statistically optimal estimate of the state of system (8)-(9). Note the fact that the disturbances of (8) and (9) are correlated to each other, so we should use the Kalman filter formulae for correlated measurement and process noise. At any time t, let us define R
^
t
+
1
-
=
E
t
(
R
t
+
1
) and V
t
+
1
-
=
Va
r
t
(
R
t
+
1
) and assume R
^
1
-
=
[
g
^
0
0
0
]
′ and V
1
-
=
diag
(
σ
g
2
,
0
,
0
). Using the results on page 123 in Lewis’ book [24], we can directly get the following Lemma 2.Lemma 2.
For the systems (8)-(9), we have the following recursive formulae:
(13)
R
^
t
+
1
-
=
L
R
^
t
-
+
l
β
Δ
u
t
+
K
t
y
t
-
h
′
R
^
t
-
,
(14)
V
t
+
1
-
=
L
V
t
-
L
′
-
K
t
h
′
V
t
-
h
+
σ
z
2
+
σ
e
2
K
t
′
+
M
Σ
M
′
,
where K
t is called themodified Kalman Gain that
(15)
K
t
=
L
V
t
-
h
+
m
h
′
V
t
-
h
+
σ
z
2
+
σ
e
2
-
1
.


## 3. The Optimal Control Algorithm

In this section, we will derive an optimal control algorithm without the normal distribution assumptions onR
1, e
t
,
w
t, and z
t, t
=
0
,
±
1
,
…. This optimal control algorithm can be applied to both the finite-horizon and infinite-horizon problems.Theorem 3.
The optimal control algorithm that solves both(16)
min
⁡
Δ
u
t
,
t
=
0,1
,
…
,
T
-
1
E
0
∑
t
=
0
T
-
1
‍
y
t
+
1
2
,
min
⁡
Δ
u
t
,
t
=
0,1
,
…
E
0
∑
t
=
0
∞
‍
y
t
+
1
2
for the system (1)–(5) is
(17)
Δ
u
t
=
-
1
β
h
′
L
R
^
t
-
+
K
t
y
t
-
h
′
R
^
t
-
,
t
=
1,2
,
…
,
where R
^
t
- is updated according to
(18)
R
^
t
+
1
-
=
I
-
l
h
′
L
R
^
t
-
+
K
t
y
t
-
h
′
R
^
t
-
and (14), (15) are with initial values R
^
1
- and V
1
-.Proof.
For a given positive time indexT, let J
t
=
min
⁡
Δ
u
s
,
s
=
t
,
t
+
1
,
…
,
T
-
1
E
t
∑
s
=
t
T
-
1
‍
y
s
+
1
2 and J
T
≡
0. We can get the Bellman equation as
(19)
J
t
=
min
⁡
Δ
u
t
E
t
y
t
+
1
2
+
E
t
J
t
+
1
.
From (8), it is evident that E
t
y
t
+
1
2
=
E
t
R
t
+
1
′
h
h
′
R
t
+
1
+
σ
z
2
+
σ
e
2. Using the fact that
(20)
E
t
R
t
+
1
′
h
h
′
R
t
+
1
=
R
^
t
+
1
-
′
h
h
′
R
^
t
+
1
-
+
E
t
R
t
+
1
-
R
^
t
+
1
-
′
h
h
′
R
t
+
1
-
R
^
t
+
1
-
and the property of trace operator, we obtain that
(21)
E
t
y
t
+
1
2
=
R
^
t
+
1
-
′
h
h
′
R
^
t
+
1
-
+
trace
h
h
′
E
t
R
t
+
1
-
R
^
t
+
1
-
R
t
+
1
-
R
^
t
+
1
-
′
+
σ
z
2
+
σ
e
2
.
From (13) and the fact that h
′
l
≡
1, we have
(22)
E
t
y
t
+
1
2
=
L
R
^
t
-
+
l
β
Δ
u
t
+
K
t
y
t
-
h
′
R
^
t
-
′
h
h
′
·
L
R
^
t
-
+
l
β
Δ
u
t
+
K
t
y
t
-
h
′
R
^
t
-
+
trace
h
h
′
V
t
+
1
-
+
σ
z
2
+
σ
e
2
=
β
Δ
u
t
+
h
′
L
R
^
t
-
+
h
′
K
t
y
t
-
h
′
R
^
t
-
2
+
h
′
V
t
+
1
-
h
+
σ
z
2
+
σ
e
2
.
Let S
t
≡
h
′
V
t
+
1
-
h
+
σ
z
2
+
σ
e
2. It is evident that S
t is irrelated to Δ
u
t since V
t
+
1
- is updated based on (14) and (15). Now (19) changes to
(23)
J
t
=
min
⁡
Δ
u
t
β
Δ
u
t
+
h
′
L
R
^
t
-
+
h
′
K
t
y
t
-
h
′
R
^
t
-
2
+
E
t
J
t
+
1
+
S
t
.
When t
=
T
-
1, we get
(24)
J
T
-
1
=
min
⁡
Δ
u
T
-
1
β
Δ
u
T
-
1
+
h
′
L
R
^
T
-
1
-
+
h
′
K
T
-
1
y
T
-
1
-
h
′
R
^
T
-
1
-
2
+
S
T
-
1
.
Solving the first order condition, we get that
(25)
Δ
u
T
-
1
*
=
-
1
β
h
′
L
R
^
T
-
1
-
+
K
T
-
1
y
T
-
1
-
h
′
R
^
T
-
1
-
.
Putting (25) into (24), we get J
T
-
1
=
S
T
-
1. When t
=
T
-
2, we have
(26)
J
T
-
2
=
min
⁡
Δ
u
T
-
2
β
Δ
u
T
-
2
+
h
′
L
R
^
T
-
2
-
+
h
′
K
T
-
2
y
T
-
2
-
h
′
R
^
T
-
2
-
2
+
S
T
-
2
+
S
T
-
1
.
Solving the first order condition, we get
(27)
Δ
u
T
-
2
*
=
-
1
β
h
′
L
R
^
T
-
2
-
+
K
T
-
2
y
T
-
2
-
h
′
R
^
T
-
2
-
and J
T
-
2
=
S
T
-
2
+
S
T
-
1. Repeating the procedures above, we can prove that (17) is the optimal adjustment strategy at time t and get
(28)
J
0
=
min
⁡
Δ
u
t
,
t
=
0,1
,
…
,
T
-
1
E
0
∑
t
=
0
T
-
1
‍
y
t
+
1
2
=
∑
t
=
0
T
-
1
‍
S
t
=
h
′
∑
t
=
0
T
-
1
‍
V
t
+
1
-
h
+
T
σ
z
2
+
σ
e
2
.
At last, we can derive (18) by putting (17) into (13). Since (17) is irrelated to the given time index T, the control strategy also solves the infinite-horizon problem. Then we finish the proof.In practice, the steps to implement the proposed control algorithm are as follows.Step 0. Setup the algorithm’s parameters and initial values, such as β
^, g
^
0, ϕ
^
i
(
i
=
1,2
,
…
,
p
), θ
^
j
(
j
=
1,2
,
…
,
q
), R
^
1
-, V
1
-, based on experience or history data.Step 1. Compute K
t based on (15).Step 2. Collect the new observation y
t and compute the adjustment Δ
u
t based on (17).Step 3. Update R
^
t
+
1
- based on (18).Step 4. Update V
t
+
1
- based on (14); let t
=
t
+
1 and go back to Step 1.Note that Step 0 is an off-line procedure and Step 1 to Step 4 are on-line procedures. As the newly proposed control algorithm is specially designed for adjusting any ARIMA disturbances, we call it ARIMA controller in the rest of this paper.


## 4. Simulation Study

In this section, we will study the performance of the ARIMA controller under multiple scenarios through Monte Carlo simulations. For simplicity, we only focus on the system (1)–(5) with n
t being an ARIMA(1,1
,
1) disturbance, although the ARIMA controller can be applied to any general ARIMA(p
,
d
,
q) disturbance. Without loss of generality, we set the parameters in the ARIMA(1,1
,
1) disturbance as ϕ
1
=
0.7, θ
1
=
-
0.3, and σ
z
=
1. For comparison, a P
I controller’s performance is also evaluated. The P
I controller is widely used in feedback control, and it involves two separate constant parameters: the proportional and the integral values, denoted by k
P and k
I. The steps to implement a P
I controller can be found in [25].Tuning aP
I controller could be a challenging task. There are many ways of tuning, such as Ziegler-Nichols tuning, lambda tuning, robust loop shaping, optimization methods, and others (see [26]). Among them, optimization methods are powerful and direct ways. In order to do fair comparisons with the ARIMA controller, we need to choose k
P and k
I to make the P
I controller have the best performance for controlling the specified ARIMA(1,1
,
1) disturbance. To the best of our knowledge, there are no closed-form expressions for k
P and k
I to optimally control a ARIMA(1,1
,
1) process; then experimental tuning of the P
I parameters has to be performed. The procedure of optimally choosing k
P and k
I in this paper is described in Appendix B. For the ARIMA(1,1
,
1) disturbance with ϕ
1
=
0.7 and θ
1
=
-
0.3, we get k
P
=
1.223 and k
I
=
0.190 according to the optimization procedure.Four scenarios are examined in the following simulations. In Scenario 1, the effect of the measurement error and the adjustment error on the ARIMA controller and theP
I controller is investigated; in Scenario 2, the effect of the process initial bias g
0 on the two controllers is explored; in Scenario 3, we study how the estimate of the process gain β
^ affects the two controllers; in Scenario 4, we investigate both controllers’ performance when the disturbance parameters are not estimated accurately.We do 1000 replications for each case and run 100 steps fory
t in each replication. The first 100-run mean square error (MSE) of y
t is computed. The Average MSE (AMSE) of the 1000 replications is reported in Tables 1–4. Also the standard error in the AMSE (SEAMSE) is computed, which is
(29)
SEAMSE
=
SDMSE
Number
of
replicates
,
where SDMSE is the standard deviation of mean square errors, and the number of replicates here is 1000. AMSE measures the performance of the controllers, and SEAMSE reflects the variability of the AMSE. The smaller the AMSE is, the better performance the controller has.Table 1
AMSE when both types of error exist.n
t is the ARIMA(1, 1, 1) disturbance with ϕ
1
=
0.7 and θ
1
=
-
0.3.
ARIMA controllerϕ
^
1
=
0.7  θ
^
1
=
-
0.3
σ
e
σ
w
=
0
0.5
1.0
1.5
2.0
0
1.014 (.005)
1.298 (.006)
2.104 (.009)
3.392 (.015)
5.155 (.023)
0.5
1.657 (.008)
1.922 (.009)
2.695 (.012)
3.959 (.018)
5.709 (.026)
1.0
3.165 (.014)
3.431 (.016)
4.205 (.019)
5.471 (.025)
7.225 (.032)
1.5
5.286 (.024)
5.569 (.025)
6.381 (.029)
7.698 (.035)
9.503 (.043)
2.0
7.966 (.037)
8.269 (.038)
9.136 (.041)
10.530 (.047)
12.420 (.056)
P
I controller k
P
=
1.223  k
I
=
0.190
σ
e
σ
w
=
0
0.5
1.0
1.5
2.0
0
1.084 (.005)
1.379 (.006)
2.265 (.010)
3.742 (.017)
5.809 (.027)
0.5
1.872 (.009)
2.172 (.010)
3.063 (.015)
4.544 (.022)
6.616 (.032)
1.0
4.247 (.023)
4.552 (.025)
5.448 (.029)
6.934 (.036)
9.010 (.047)
1.5
8.211 (.048)
8.520 (.049)
9.420 (.053)
10.911 (.060)
12.992 (.071)
2.0
13.761 (.082)
14.075 (.083)
14.980 (.088)
16.475 (.095)
18.561 (.105)
The corresponding SEAMSE’s are enclosed in the parentheses.Table 2
AMSE when the initial bias’ mean and standard deviation vary.n
t is the ARIMA(1, 1, 1) disturbance with ϕ
1
=
0.7 and θ
1
=
-
0.3.
ARIMA controllerϕ
^
1
=
0.7  θ
^
1
=
-
0.3
g
^
0
σ
g
=
0
1
2
3
4
−4
1.176 (.006)
1.180 (.007)
1.203 (.008)
1.244 (.010)
1.304 (.013)
−3
1.106 (.006)
1.110 (.006)
1.134 (.007)
1.176 (.009)
1.237 (.011)
−2
1.055 (.005)
1.061 (.005)
1.085 (.006)
1.128 (.008)
1.190 (.010)
−1
1.025 (.005)
1.031 (.005)
1.057 (.005)
1.101 (.007)
1.163 (.009)
0
1.014 (.005)
1.021 (.005)
1.048 (.005)
1.093 (.006)
1.156 (.008)
1
1.024 (.005)
1.032 (.005)
1.059 (.005)
1.105 (.007)
1.169 (.009)
2
1.053 (.005)
1.062 (.005)
1.091 (.006)
1.137 (.008)
1.202 (.010)
3
1.103 (.006)
1.113 (.006)
1.142 (.007)
1.189 (.009)
1.255 (.011)
4
1.172 (.007)
1.183 (.007)
1.213 (.008)
1.261 (.011)
1.328 (.013)
P
I controller  k
P
=
1.223  k
I
=
0.190
g
^
0
σ
g
=
0
1
2
3
4
−4
1.279 (.007)
1.284 (.008)
1.311 (.010)
1.360 (.012)
1.432 (.016)
−3
1.194 (.006)
1.200 (.007)
1.228 (.008)
1.279 (.010)
1.351 (.013)
−2
1.133 (.006)
1.140 (.006)
1.170 (.007)
1.221 (.009)
1.294 (.011)
−1
1.096 (.005)
1.105 (.005)
1.135 (.006)
1.187 (.008)
1.261 (.010)
0
1.084 (.005)
1.093 (.005)
1.124 (.006)
1.177 (.007)
1.253 (.010)
1
1.095 (.005)
1.105 (.005)
1.137 (.006)
1.192 (.008)
1.268 (.010)
2
1.130 (.006)
1.141 (.006)
1.174 (.007)
1.230 (.009)
1.307 (.011)
3
1.189 (.007)
1.201 (.007)
1.235 (.008)
1.292 (.010)
1.370 (.013)
4
1.272 (.007)
1.285 (.008)
1.320 (.009)
1.378 (.012)
1.458 (.016)
The corresponding SEAMSE’s are enclosed in the parentheses. Neither measurement errors nor adjustment errors are considered in the simulations.Table 3
AMSE with different estimates of the process gain. The true process gainβ
=
2.5.
n
t is ARIMA(1, 1, 1) ϕ
=
0.7  θ
=
-
0.3
β
^
ARIMA controller
P
I controller
β
^
ARIMA controller
P
I controller
(ϕ
^
=
0.7  θ
^
=
-
0.3)
(k
P
=
1.223  k
I
=
0.190)
(ϕ
^
=
0.7  θ
^
=
-
0.3)
(k
P
=
1.223  k
I
=
0.190)
AMSE (SEAMSE)
AMSE (SEAMSE)
AMSE (SEAMSE)
AMSE (SEAMSE)
1.7
2.207 (.020)
5.113 (.084)
2.6
1.017 (.005)
1.088 (.005)
1.8
1.583 (.011)
2.213 (.021)
2.7
1.024 (.005)
1.099 (.005)
1.9
1.320 (.008)
1.597 (.011)
2.8
1.036 (.005)
1.115 (.006)
2.0
1.183 (.006)
1.343 (.008)
2.9
1.051 (.005)
1.135 (.006)
2.1
1.104 (.006)
1.215 (.006)
3.0
1.068 (.005)
1.159 (.006)
2.2
1.058 (.005)
1.145 (.006)
3.1
1.087 (.005)
1.186 (.006)
2.3
1.032 (.005)
1.107 (.005)
3.2
1.108 (.005)
1.215 (.007)
2.4
1.018 (.005)
1.089 (.005)
3.3
1.131 (.006)
1.246 (.007)
2.5
1.014 (.005)
1.084 (.005)
3.4
1.155 (.006)
1.279 (.007)Table 4
AMSE with uncertainties in the ARIMA parameters.
ARIMA controllerϕ
^
1
=
0.7  θ
^
1
=
-
0.3
ϕ
1
θ
1
=
-
0.5
−0.4
−0.3
−0.2
−0.1
0.5
1.177 (.006)
1.109 (.005)
1.061 (.005)
1.034 (.005)
1.027 (.005)
0.6
1.106 (.005)
1.056 (.005)
1.027 (.005)
1.018 (.005)
1.031 (.005)
0.7
1.052 (.005)
1.022 (.005)
1.014 (.005)
1.028 (.005)
1.064 (.005)
0.8
1.024 (.005)
1.021 (.005)
1.044 (.005)
1.091 (.006)
1.162 (.006)
0.9
1.073 (.006)
1.129 (.007)
1.218 (.008)
1.340 (.010)
1.495 (.012)
0.95
1.248 (.010)
1.398 (.013)
1.596 (.018)
1.843 (.023)
2.138 (.029)
P
I controller  k
P
=
1.223  k
I
=
0.190
ϕ
1
θ
1
=
-
0.5
−0.4
−0.3
−0.2
−0.1
0.5
1.199 (.006)
1.137 (.006)
1.097 (.005)
1.078 (.005)
1.081 (.005)
0.6
1.136 (.006)
1.095 (.005)
1.077 (.005)
1.081 (.005)
1.109 (.005)
0.7
1.092 (.005)
1.076 (.005)
1.084 (.005)
1.116 (.006)
1.172 (.006)
0.8
1.074 (.005)
1.087 (.005)
1.128 (.006)
1.197 (.007)
1.293 (.008)
0.9
1.094 (.005)
1.151 (.006)
1.242 (.007)
1.366 (.009)
1.523 (.011)
0.95
1.129 (.006)
1.220 (.007)
1.348 (.009)
1.515 (.011)
1.720 (.014)
The corresponding SEAMSE’s are included in the parentheses.


### 4.1. Effects of Measurement Error and Adjustment Error

In order to focus on the effects of the measurement errore
t and the adjustment error w
t on the controllers, we set the initial process bias g
0
=
0, that is, g
^
0
=
σ
g
=
0, and set the estimate of the process gain β
^ to the true value β. We also assume all the ARIMA(1,1
,
1) parameters can be accurately estimated, that is, ϕ
^
1
=
ϕ
1 and θ
^
1
=
θ
1.Table1 and Figure 1 show the performance of the two controllers affected by the measurement errors and adjustment errors. We set the values of σ
e and σ
w to be 0, 0.5, 1.0, 1.5 and 2.0, respectively, so there are 25 pairs of (σ
e
,
σ
w). For each pair of (σ
e
,
σ
w), we repeat the simulations. We can observe that for both controllers the AMSE of y
t increases when σ
e or σ
w increases. However, the AMSE of y
t using the ARIMA controller is always smaller than that using the P
I controller at a significant level for all the pairs of (
σ
e
,
σ
w
).Figure 1
The performance of the ARIMA controller and theP
I controller when both measurement error (σ
e) and adjustment error (σ
w) exist and vary.


### 4.2. Effects of Initial Bias

For varying sizes ofg
^
0 and σ
g in the prior distribution of the process initial bias, Table 2 and Figure 2 show the results of the AMSE of y
t using the ARIMA controller or the P
I controller for the ARIMA(1,1
,
1) disturbance. In order to focus on the effects of the initial bias, we set all the e
t and w
t to 0, β
^
=
β, ϕ
^
1
=
ϕ
1, and θ
^
1
=
θ
1.Figure 2
The performance of the ARIMA controller and theP
I controller when the process initial bias varies.It can be seen that for both controllers, the AMSE increases withσ
g when g
^
0 is fixed; and the AMSE increases with |
g
^
0
| when σ
g is fixed. For the same pair of (
g
^
0
,
σ
g
), the AMSE of y
t using the ARIMA controller is always smaller than that using the P
I controller at a significant level.


### 4.3. Effect of Estimation Uncertainties in the Process Gain

In order to focus on the effect of the estimate of the process gain brought to the controller’s performance, we assume both measurement error and adjustment error are absent, that is,e
t
=
w
t
=
0; the initial bias g
0 is 0, that is, g
^
0
=
σ
g
=
0; and all the ARIMA parameters are accurately estimated, that is, ϕ
^
1
=
ϕ
1 and θ
^
1
=
θ
1.We set the process gainβ to 2.5 and assume β
^
∈
[
1.7,3.4
]. Table 3 and Figure 3 present the performance of the two controllers when β
^ varies. It is shown that the ARIMA controller always outperforms the P
I controller for all the β
^s we examined at a significant level. Additionally, underestimated β (i.e., β
^
/
β
<
1) will hurt both controllers’ performance more than overestimated β (i.e., β
^
/
β
>
1). It is shown that if the process gain β is underestimated to some degree, both controllers would completely fail.Figure 3
The performance of the ARIMA controller and theP
I controller when the process gain is not accurately estimated.


### 4.4. Effect of Estimation Uncertainties in the Disturbance

Again, in order to focus on the effect of the estimation uncertainties in the disturbance, we assumee
t
=
w
t
=
g
^
0
=
σ
g
=
0 and β
^
=
β. Suppose the disturbance parameters are estimated as ϕ
^
1
=
0.7 and θ
^
1
=
-
0.3. However, the disturbance’s true parameters may not be equal to the estimates, that is, ϕ
1
≠
ϕ
^
1 and/or θ
1
≠
θ
^
1. We choose 5 values of ϕ
1 and 5 values of θ
1 as the true parameters for the ARIMA(1,1
,
1) disturbance, so there are a total of 25 disturbance ARIMA(1,1
,
1) models that are studied to investigate both controllers’ robustness.Table4 reports the results of the AMSE of y
t using the ARIMA controller and the P
I controller for different ARIMA(1,1
,
1) disturbances with varying ϕ
1 and θ
1. It is shown that the ARIMA controller’s performance is better than the P
I controller’s in most of the cases at a significant level. Only when ϕ
1
=
0.95 is the P
I controller’s performance better than the ARIMA controller’s. This is because a ARIMA(1,1
,
1) process will become a ARIMA(0,2
,
1) process when ϕ
1
=
1. As this ARIMA controller is purposely designed to control a ARIMA(1,1
,
1) disturbance, its performance will be contaminated when ϕ
1 is close to 1. The results also imply that if a ARIMA(0,2
,
1) disturbance is misidentified to be a ARIMA(1,1
,
1) disturbance, the ARIMA controller would have a bad performance.For illustration, Figure4 further presents the contour plots for the AMSE as a function of ϕ
1 and θ
1 by using the ARIMA controller and the P
I controller, respectively. It can be generally observed that both controllers that are designed based on underestimated disturbance parameters (i.e., ϕ
^
1
<
ϕ
1, θ
^
1
<
θ
1) will have worse performance than the ones designed based on overestimated disturbance parameters (i.e., ϕ
^
1
>
ϕ
1, θ
^
1
>
θ
1). Specially, as can be seen from Figure 4(a), when ϕ
1 is not greater than 0.9, the AMSE of y
t by using the ARIMA controller changes very slowly with varying ϕ
1 and θ
1. However, when ϕ
1 is greater than 0.9, the AMSE increases very quickly with ϕ
1 increasing. The parameter θ
1 has less effects than ϕ
1 on the performance of the ARIMA controller. In comparison with the ARIMA controller, the P
I controller’s performance is affected by both ϕ
1 and θ
1 in almost the same degree as shown in Figure 4(b). In all the change area of ϕ
1 and θ
1 except the area ϕ
1
>
0.9, the P
I controller’s performance is worse than the ARIMA controller’s. However, the maximum AMSE (the worst case) used by the P
I controller in the whole change area of ϕ
1 and θ
1 is smaller than that used by the ARIMA controller. In this sense, we can say that the P
I controller’s performance is more robust than the ARIMA controller’s when uncertainties exist in the ARIMA parameters.Contour plots of AMSE for the ARIMA controller and theP
I controller when the disturbance parameters (
ϕ
1
,
θ
1
) vary.
(a)
ARIMA controller
(b)
P
I controller


## 4.1. Effects of Measurement Error and Adjustment Error

In order to focus on the effects of the measurement errore
t and the adjustment error w
t on the controllers, we set the initial process bias g
0
=
0, that is, g
^
0
=
σ
g
=
0, and set the estimate of the process gain β
^ to the true value β. We also assume all the ARIMA(1,1
,
1) parameters can be accurately estimated, that is, ϕ
^
1
=
ϕ
1 and θ
^
1
=
θ
1.Table1 and Figure 1 show the performance of the two controllers affected by the measurement errors and adjustment errors. We set the values of σ
e and σ
w to be 0, 0.5, 1.0, 1.5 and 2.0, respectively, so there are 25 pairs of (σ
e
,
σ
w). For each pair of (σ
e
,
σ
w), we repeat the simulations. We can observe that for both controllers the AMSE of y
t increases when σ
e or σ
w increases. However, the AMSE of y
t using the ARIMA controller is always smaller than that using the P
I controller at a significant level for all the pairs of (
σ
e
,
σ
w
).Figure 1
The performance of the ARIMA controller and theP
I controller when both measurement error (σ
e) and adjustment error (σ
w) exist and vary.


## 4.2. Effects of Initial Bias

For varying sizes ofg
^
0 and σ
g in the prior distribution of the process initial bias, Table 2 and Figure 2 show the results of the AMSE of y
t using the ARIMA controller or the P
I controller for the ARIMA(1,1
,
1) disturbance. In order to focus on the effects of the initial bias, we set all the e
t and w
t to 0, β
^
=
β, ϕ
^
1
=
ϕ
1, and θ
^
1
=
θ
1.Figure 2
The performance of the ARIMA controller and theP
I controller when the process initial bias varies.It can be seen that for both controllers, the AMSE increases withσ
g when g
^
0 is fixed; and the AMSE increases with |
g
^
0
| when σ
g is fixed. For the same pair of (
g
^
0
,
σ
g
), the AMSE of y
t using the ARIMA controller is always smaller than that using the P
I controller at a significant level.


## 4.3. Effect of Estimation Uncertainties in the Process Gain

In order to focus on the effect of the estimate of the process gain brought to the controller’s performance, we assume both measurement error and adjustment error are absent, that is,e
t
=
w
t
=
0; the initial bias g
0 is 0, that is, g
^
0
=
σ
g
=
0; and all the ARIMA parameters are accurately estimated, that is, ϕ
^
1
=
ϕ
1 and θ
^
1
=
θ
1.We set the process gainβ to 2.5 and assume β
^
∈
[
1.7,3.4
]. Table 3 and Figure 3 present the performance of the two controllers when β
^ varies. It is shown that the ARIMA controller always outperforms the P
I controller for all the β
^s we examined at a significant level. Additionally, underestimated β (i.e., β
^
/
β
<
1) will hurt both controllers’ performance more than overestimated β (i.e., β
^
/
β
>
1). It is shown that if the process gain β is underestimated to some degree, both controllers would completely fail.Figure 3
The performance of the ARIMA controller and theP
I controller when the process gain is not accurately estimated.


## 4.4. Effect of Estimation Uncertainties in the Disturbance

Again, in order to focus on the effect of the estimation uncertainties in the disturbance, we assumee
t
=
w
t
=
g
^
0
=
σ
g
=
0 and β
^
=
β. Suppose the disturbance parameters are estimated as ϕ
^
1
=
0.7 and θ
^
1
=
-
0.3. However, the disturbance’s true parameters may not be equal to the estimates, that is, ϕ
1
≠
ϕ
^
1 and/or θ
1
≠
θ
^
1. We choose 5 values of ϕ
1 and 5 values of θ
1 as the true parameters for the ARIMA(1,1
,
1) disturbance, so there are a total of 25 disturbance ARIMA(1,1
,
1) models that are studied to investigate both controllers’ robustness.Table4 reports the results of the AMSE of y
t using the ARIMA controller and the P
I controller for different ARIMA(1,1
,
1) disturbances with varying ϕ
1 and θ
1. It is shown that the ARIMA controller’s performance is better than the P
I controller’s in most of the cases at a significant level. Only when ϕ
1
=
0.95 is the P
I controller’s performance better than the ARIMA controller’s. This is because a ARIMA(1,1
,
1) process will become a ARIMA(0,2
,
1) process when ϕ
1
=
1. As this ARIMA controller is purposely designed to control a ARIMA(1,1
,
1) disturbance, its performance will be contaminated when ϕ
1 is close to 1. The results also imply that if a ARIMA(0,2
,
1) disturbance is misidentified to be a ARIMA(1,1
,
1) disturbance, the ARIMA controller would have a bad performance.For illustration, Figure4 further presents the contour plots for the AMSE as a function of ϕ
1 and θ
1 by using the ARIMA controller and the P
I controller, respectively. It can be generally observed that both controllers that are designed based on underestimated disturbance parameters (i.e., ϕ
^
1
<
ϕ
1, θ
^
1
<
θ
1) will have worse performance than the ones designed based on overestimated disturbance parameters (i.e., ϕ
^
1
>
ϕ
1, θ
^
1
>
θ
1). Specially, as can be seen from Figure 4(a), when ϕ
1 is not greater than 0.9, the AMSE of y
t by using the ARIMA controller changes very slowly with varying ϕ
1 and θ
1. However, when ϕ
1 is greater than 0.9, the AMSE increases very quickly with ϕ
1 increasing. The parameter θ
1 has less effects than ϕ
1 on the performance of the ARIMA controller. In comparison with the ARIMA controller, the P
I controller’s performance is affected by both ϕ
1 and θ
1 in almost the same degree as shown in Figure 4(b). In all the change area of ϕ
1 and θ
1 except the area ϕ
1
>
0.9, the P
I controller’s performance is worse than the ARIMA controller’s. However, the maximum AMSE (the worst case) used by the P
I controller in the whole change area of ϕ
1 and θ
1 is smaller than that used by the ARIMA controller. In this sense, we can say that the P
I controller’s performance is more robust than the ARIMA controller’s when uncertainties exist in the ARIMA parameters.Contour plots of AMSE for the ARIMA controller and theP
I controller when the disturbance parameters (
ϕ
1
,
θ
1
) vary.
(a)
ARIMA controller
(b)
P
I controller


## 5. An Illustrative Example

The studies in the last section are carried out based on an ARIMA(1,1
,
1) disturbance. As aforementioned, the proposed ARIMA controller can be applied to any general ARIMA(p
,
d
,
q) disturbance. For illustration purpose, we assume a process as follows:
(30)
y
t
=
x
t
-
1
+
n
t
and (
1
-
0.2
B
)
(
1
-
B
)
n
t
=
(
1
-
0.1
B
-
0.41
B
2
+
0.105
B
3
)
ε
t, ε
t
~
WN
(
0,0
.
8
2
); y
t is the deviation from target. The disturbance model n
t is the same as one of the ARIMA(1,1
,
3) disturbances presented in [21]. For the sake of simplicity, we further set σ
e
=
σ
w
=
0, y
0
=
0, and assume that all the disturbance parameters are known. That is, we ignore measurement errors, adjustment errors, initial bias uncertainties, and disturbance parameter uncertainties and only show the ARIMA controller’s superiority in controlling higher order ARIMA disturbance models.For the above ARIMA(1,1
,
3) disturbance, we can get m
=
max
⁡
(
1,3
)
=
3 and(31)
θ
z
ϕ
z
=
1
-
0.1
z
-
0.41
z
2
+
0.105
z
3
1
-
0.2
z
=
1
+
0.1
z
-
0.39
z
2
+
0.027
z
3
+
⋯
.
So, in the offline procedure, the ARIMA controller’s parameters are given as follows:
(32)
h
=
1
1
0
0
1
M
=
1
0
0
0
0
0
0.1
0
0
0
0
0
-
0.39
0
0
0
0
0
0.027
0
0
0
0
0
1
L
=
1
0
0
0
0
0
0
1
0
0
0
0
0
1
0
0
0
0
0.2
0
0
1
0
0
1
Σ
=
0
0
0
0
0
0
0.64
0.64
0.64
0.64
0
0.64
0.64
0.64
0.64
0
0.64
0.64
0.64
0.64
0
0.64
0.64
0.64
0.64
m
=
0
0.1
-
0.39
0.027
1
.
The initial values R
^
1
-
=
V
1
-
=
0. Using the same optimization tuning techniques described in Appendix B, we get k
P
=
1.128 and k
I
=
-
0.008.We randomly draw one simulation and show the paths of the process outputy
t in Figure 5 when the two control algorithms are implemented, respectively. The MSE of y
t implemented by the P
I controller is 0.819 while that implemented by the ARIMA controller is 0.646. The paths suggest that the ARIMA controller maintains the process output y
t closer to the target than the P
I controller in much more places in the first 100 simulated runs.Figure 5
One simulated path by using the ARIMA andP
I controllers, respectively, for the ARIMA(1,1,3) disturbance that (
1
-
0.2
B
)
(
1
-
B
)
n
t
=
(
1
-
0.1
B
-
0.41
B
2
+
0.105
B
3
)
ε
t with σ
ε
=
0.8.


## 6. Concluding Remarks

In this paper, we have developed an optimal control algorithm, the ARIMA controller, for a linear system with a general ARIMA(p
,
d
,
q) disturbance, in the presence of measurement errors and adjustment errors together with a random initial bias. We theoretically prove that the ARIMA controller is optimal to both the finite-horizon and infinite-horizon problems. The performances of the ARIMA and P
I controllers have been evaluated and compared via Monte Carlo simulations under multiple scenarios. In almost all the analyzed scenarios, the ARIMA controller outperforms the P
I controller. Only when uncertainties exist in the ARIMA parameters does the P
I controller have more robust performance than the ARIMA controller. Our simulation studies also show that when designing the two controllers, underestimating the process parameters, including the process gain and the ARIMA parameters, will have more negative impact on the controllers’ performance than overestimating the parameters. Although the steps to implement the P
I controller are simple, a tuning process is always needed to determine the P
I parameters k
P and k
I for each given disturbance. It is usually time-consuming. Conversely, the ARIMA controller can automatically determine its algorithm parameters for each given disturbance.The ARIMA controller is a complement to the ARMA controller which is designed to optimally control any general ARMA(p
,
q) disturbance. The ARMA controller can be used to control a large class of weakly stationary disturbances while the ARIMA controller can handle a large class of nonstationary or periodic disturbances. Such disturbances are common in rapid thermal processing, reactive ion etching, I-line lithography, and lapping processes in the semiconductor manufacturing.



---

*Source: 808903-2015-02-25.xml*