# Offset-Free Strategy by Double-Layered Linear Model Predictive Control

Authors: Tao Zou
Journal: Journal of Applied Mathematics
 (2012)

---

## Abstract

In the real applications, the model predictive control (MPC) technology is separated into two layers, that is, a layer of conventional dynamic controller, based on which is an added layer of steady-state target calculation. In the literature, conditions for offset-free linear model predictive control are given for combined estimator (for both the artificial disturbance and system state), steady-state target calculation, and dynamic controller. Usually, the offset-free property of the double-layered MPC is obtained under the assumption that the system is asymptotically stable. This paper considers the dynamic stability property of the double-layered MPC.


---

## Body

## 1. Introduction

The technique model predictive control (MPC) differs from other control methods mainly in its implementation of the control actions. Usually, MPC solves a finite-horizon optimal control problem at each control interval, so that the control moves for the current time and a period of future time (say, totallyN control intervals) are obtained. However, only the current control move is applied to the plant. At the next control interval, the same kind of optimization is repeated with the new measurements [1]. The MPC procedures applied in the industrial processes lack theoretical guarantee of stability. Usually, industrial MPC adopts a finite-horizon optimization, without a special weighting on the output prediction at the end of the prediction horizon.Theoretically, the regulation problem for the nominal MPC can have guarantee of stability by imposing special weight and constraint on the terminal state prediction [2]. The authors in [2] give a comprehensive framework. However, [2] does not solve everything for the stability of MPC. In the past 10 years, the studies on the robust MPC for regulation problem go far beyond [2]. We could say that, for the case of regulation problem when the system state is measurable, the research on MPC is becoming mature (see e.g., [3–8]). For the case of regulation problem when the system state is unmeasurable, and there is no model parametric uncertainty, the research on MPC is becoming mature (see e.g. [9–11]). For other cases (output feedback MPC for the systems with parametric uncertainties, tracking MPC, etc.), there are many undergoing researches (see e.g., [12–16]).A synthesis approach of MPC is that with guaranteed stability. However, the industrial MPC adopts a more complex framework than the existing synthesis approaches of MPC. Its hierarchy is shown in, for example [17]. In other words, the synthesis approaches of MPC have not been sufficiently developed to include the industrial MPC. Today, the separation of the MPC algorithm into steady-state target and dynamic control move calculations is a common part of industrial MPC technology [17]. The use of steady-state target calculation is necessary, since the disturbances entering the systems or new input information from the operator may change the location of the optimal steady-state at any control interval (see e.g., [18]). The goal of the steady-state target calculation is to recalculate the targets from the local optimizer every time the MPC controller executes.In the linear MPC framework, offset-free control is usually achieved by adding step disturbance to the process model. The most widely used industrial MPC implementations assume a constant output disturbance that can lead to sluggish rejections of disturbances that enter the process elsewhere. In [19, 20], some general disturbance models that accommodate unmeasured disturbances entering through the process input, state, or output, have been proposed. In a more general sense, the disturbance model can incorporate any nonlinearity, uncertainty, and physical disturbance (measured or unmeasured). The disturbance can be estimated by the Kalman filter (or the usual observer). The estimated disturbance is assumed to be step-like, that is unchanging in the future, at each control interval (MPC refreshes its solution at each control interval). The estimated disturbance drives the steady-state target calculation, in order to refresh the new target value for the control move optimization.This paper visits some preliminary results for the stability of double-layered MPC or output tracking MPC. These results could be useful for incorporating the industrial MPC into the synthesis approaches of MPC. The preliminary results for this paper can be found in [21, 22].Notations 1.
For any vectorx and positive-definite matrix M, ∥x∥M2:=xTMx. x(k+i∣k) is the value of vector x at time k+i, predicted at time k. I is the identity matrix with appropriate dimension. All vector inequalities are interpreted in an element-wise sense. The symbol ⋆ induces a symmetric structure in the matrix inequalities. An optimal solution to the MPC optimization problem is marked with superscript ⋆. The time-dependence of the MPC decision variables is often omitted for brevity.


## 2. System Description and Observer Design

Consider the following discrete-time model:(2.1)x(k+1)=Ax(k)+Bu(k)+Ed(k),d(k+1)=d(k)+Δd(k),y(k)=Cx(k)+Dd(k),
where u∈ℜm denotes the control input variables, x∈ℜn the state variables, y∈ℜp the output variables, and d∈ℜq the unmeasured signals including all disturbances and plant-model mismatches.Assumption 2.1.
The augmented pair(2.2)([CD],[AE0I])
is detectable, and the following condition holds:
(2.3)rank⁡[I-A-ECD]=n+q.The augmented observer is(2.4)[x^(k+1)d^(k+1)]=[AE0I][x^(k)d^(k)]+[B0]u(k)+[Fs1Fs2](Cx^(k)+Dd^(k)-y(k)),
where Fs=[(Fs1)T,(Fs2)T]T is the prespecified observer gain. Define the estimation error x~(k)=x(k)-x^(k) and d~(k)=d(k)-d^(k); then one has the following observer error dynamic equation:
(2.5)[x~(k+1)d~(k+1)]=([AE0I]+[Fs1Fs2][CD])[x~(k)d~(k)]+[0I]Δd(k).Assumption 2.2.
Δ
d
(
k
) is an asymptotically vanishing item, and the observer error dynamics is asymptotically stable, that is, limk→∞{Δd(k),x~(k),d~(k)}={0,0,0}.


## 3. Double-Layered MPC with Off-Set Property

For the system (2.1), its steady-state state and input target vectors, xt(k) and ut(k), can be determined from the solution of the following quadratic programming (QP) problems (steady-state target calculation, steady-state controller):
(3.1)min⁡xt,ut‖ut-u-r‖Rt2,(3.2)s.t.{[I-A-BC0][xtut]=[Ed^(k)y-r-Dd^(k)]umin⁡≤ut≤umax⁡(3.3)min⁡xt,ut‖yt-y-r‖Qt2,(3.4)s.t.{[I-A-BC0][xtut]=[Ed^(k)yt-Dd^(k)]umin⁡≤ut≤umax⁡,
where y-r is the desired steady-state output (e.g., from the local optimizer), u-r is the desired steady-state input, and (umin,umax) are the input bounds. Problems (3.1) and (3.2) is solved; when (3.1) and (3.2) is feasible, yt=y-r and (3.3) and (3.4) is not solved; when (3.1) and (3.2) is infeasible, (3.3) and (3.4) is solved.When this target generation problem is feasible, one has(3.5)xt(k)=Axt(k)+But(k)+Ed^(k),yt(k)=Cxt(k)+Dd^(k).
Subtracting (3.5) from (2.1) and utilizing (2.5) yield
(3.6)χ^(k+1,k)=Aχ^(k,k)+Bω(k)-Fs1(Cx~(k)+Dd~(k)),
where the shifted variables χ^(·,k):=x^(·)-xt(k), ω:=u-ut. The following nominal model of the transformed system (3.6) is used for prediction
(3.7)χ^(k+i+1∣k)=Aχ^(k+i∣k)+Bω(k+i∣k).
Its infinite horizon predictive control performance cost is defined as
(3.8)J0∞(k)=∑i=0∞‍W(χ^(k+i∣k),ω(k+i∣k)),
where W(χ^(k+i∣k),ω(k+i∣k))=∥χ^(k+i∣k)∥Q2+∥ω(k+i∣k)∥R2.Defining a quadratic functionV(χ^(k+i∣k))=∥χ^(k+i∣k)∥P2, if one can show that
(3.9)V(χ^(k+i+1∣k))-V(χ^(k+i∣k))≤-W(χ^(k+i∣k),ω(k+i∣k)),
then it can be concluded that V(χ^(k+i∣k))→0 as i→∞. Furthermore, summing (3.9) from i=N to ∞ yields the upper bound of JN∞ as
(3.10)∑i=N∞‍W(χ^(k+i∣k),ω(k+i∣k))≤V(χ^(k+N∣k)).
By substituting (3.10) into (3.8), one can get
(3.11)J0∞(k)≤∑i=0N-1‍W(χ^(k+i∣k),ω(k+i∣k))+V(χ^(k+N∣k))=:J-(χ^(k),π(k)).
Here J-(χ^(k),π(k)) gives an upper bound of J0∞(k); so we can formulate the MPC as an equivalent minimization problem on J-(χ^,π) with respect to the optimal control sequence
(3.12)π*(k)=[ω*(k∣k)T,ω*(k+1∣k)T,…,ω*(k+N-1∣k)T]T.
When x^(k+N∣k) lies in the terminal region, ω(k+i∣k)=Kχ^(k+i∣k), i≥N. From the definition of J-(χ^(k),π(k)), at time instant k+1, one has
(3.13)J-(χ^(k+1),π(k+1))=∑i=1N‍W(χ^(k+i∣k+1),ω(k+i∣k+1))+V(χ^(k+N+1∣k+1))
with the shifted control sequence
(3.14)π(k+1)=[(ω*(k+1∣k)+ut(k)-ut(k+1))T,…,aa(ω*(k+N-1k)+ut(k)-ut(k+1))T,(Kχ^(k+Nk)+ut(k)-ut(k+1))T]T.
We can explicitly derive the multi-step-ahead state and output prediction:
(3.15)χ^(k+N∣k)=ANχ^(k)+A~Bπ(k),(3.16)Y~χ(k)=T~Aχ^(k)+T~Bπ(k),
where
(3.17)A~B=[AN-1B,…,AB,B],Y~χ(k)=[χ^(k∣k)χ^(k+1∣k)⋮χ^(k+N-1∣k)],(3.18)T~A=[IA⋮AN-1],T~B=[00⋯0B0⋯0⋮⋱⋱⋮AN-2B⋯B0].Lemma 3.1.
For a quadratic functionW(x,u)=xTQx+uTRu, Q,R>0, there exist finite Lipschitz constants ℒx,ℒu>0 such that
(3.19)‖W(x1,u1)-W(x2,u2)‖≤Lx‖x1-x2‖+Lu‖u1-u2‖
for all x1,x2∈𝒳, u1,u2∈𝒰, where 𝒳, 𝒰 are bounded regions. Similarly, for a quadratic function V(x)=xTPx, P>0, there exists a finite Lipschitz constant ℒV>0 such that
(3.20)‖V(x1)-V(x2)‖≤LV‖x1-x2‖
for all x1,x2∈𝒳.Clearly,ℒx, ℒu, ℒV depend on 𝒳, 𝒰. However, it is unnecessary to specify 𝒳, 𝒰 in the following. Moreover, ℒV depends on P, which is time varying; this paper assumes taking ℒV for all possible P.Lemma 3.2.
Consider the prediction model (3.7). Then, with the shifted control sequence π(k+1),
(3.21)‖χ^(k+i∣k+1)-χ^(k+i∣k)‖≤‖A‖i-1(‖Fs1C‖‖x~(k)‖+‖Fs1D‖‖d~(k)‖+‖xt(k)-xt(k+1)‖)+∑j=0i-2‍‖A‖j‖B‖(‖ut(k)-ut(k+1)‖).Proof .
It is easy to show that(3.22)χ^(k+1,k+1)=x^(k+1)-xt(k+1)=χ^(k+1∣k)-Fs1(Cx~(k)+Dd~(k))+xt(k)-xt(k+1).
Then,
(3.23)‖χ^(k+1∣k+1)-χ^(k+1∣k)‖aaa=‖χ^(k+1)-χ^(k+1∣k)‖aaa≤‖Fs1C‖‖x~(k)‖+‖Fs1D‖‖d~(k)‖+‖xt(k)-xt(k+1)‖,∥χ^(k+2∣k+1)-χ^(k+2∣k)∥aaa=‖A(χ^(k+1∣k+1)-χ^(k+1∣k))+B(ω(k+1∣k+1)-ω(k+1∣k))‖aaa≤‖A‖‖χ^(k+1∣k+1)-χ^(k+1∣k)‖+‖B‖‖ut(k)-ut(k+1)‖.
By induction, one can easily show the claimed result, and thus the proof is completed.Theorem 3.3.
For the system (2.1) subject to the input constraints
(3.24)umin⁡≤u≤umax⁡,
under Assumptions 2.1-2.2, the closed-loop output feedback model predictive control system, with objective function J-(χ^(k),π(k)), augmented observer (2.4), and target generation procedure (3.1)–(3.4), achieves the offset-free reference tracking performance if the following three conditions are satisfied. (a)
There exist feasible solutions(xt(k),ut(k)) to the target generation problem (3.1)–(3.4), at each time k.(b)
There exist feasible solutions, including a control sequenceπ*(k), a positive-definite matrix X^, and any matrix Y^, at each time k, to the dynamic optimization problem (dynamic control move calculation problem)(3.25)min⁡γ1,γ2,π,X^,Y^(γ1+γ2),subject to the linear matrix inequalities(3.26)[γ1⋆⋆T~Aχ^(k)+T~BπQ~-1⋆π0R~-1]≥0,(3.27)[1⋆ANχ^(k)+A~BπX^]≥0,(3.28)[X^⋆⋆⋆AX^+BY^X^⋆⋆X^0γ2Q-1⋆Y^00γ2R-1]≥0,(3.29)[u-j2⋆Y^TUjTX^]≥0,j=1,…,m,(3.30)[Im×N-Im×N]π≤[Πm(umax⁡-ut(k))-Πm(umin⁡-ut(k))],whereUj is the jth row of the m-ordered identity matrix, Q~=IN⊗Q, R~=IN⊗R, Πm=[Im,…,Im]T, and u-j=min{(umax-ut(k))j,(ut(k)-umin)j}.(c)
By applyingu(k)=ut(k)+ω*(k∣k), where ω*(k∣k) is obtained by solving (3.25)—(3.30), the closed-loop system is asymptotically stable.Proof.
The matrix inequality (3.28) implies that
(3.31)(A+BK)TP(A+BK)-P+Q+KTRK≤0.
By referring to [23], it is easy to prove that (3.9) holds for all i≥N. Then, V(χ^(k+N∣k))≤∥χ^(k+N∣k)∥P2. Let ∥χ^(k+N∣k)∥P2≤γ2(k), which is guaranteed by (3.27), where P=γ2X^-1. Meanwhile, it is easy to show that, by applying (3.26), the optimal γ1*(k) is exactly the optimal value of
(3.32)J0N-1*(k)=∑i=0N-1‍W(χ^(k+i∣k),ω*(k+i∣k)).
Now we check if each element of the predictive control inputs satisfies the constraintsuj,min≤uj(k+i∣k)≤uj,max, i≥0, j=1,…,m. For any i within the finite horizon N, the input constraints are satisfied since Πm(umin-ut(k))≤π≤Πm(umax-ut(k)), as shown in (3.30). Otherwise, beyond the finite horizon i≥N, χ^(k+i∣k) belongs to the constraint set ℰ=⋃‍{z∈ℜn∣zTX^-1z≤1}, which is guaranteed by (3.27). In this case, by referring to [23], it is easy to show that, (3.27)–(3.29) guarantee that the feedback control law ω(k+i∣k)=Kχ^(k+i∣k), i≥N, K=Y^X^-1 satisfies the input constraints.
Since point (c) is assumed, the offset-free property can be referred to as in [19, 20, 22].


## 4. Improved Procedure for Double-Layered MPC

At each timek+1≥0, we consider the following constraints:
(4.1)[I-A-BC0][xtut]=[Ed^(k+1)y-r-Dd^(k+1)],umin⁡≤ut≤umax⁡,(4.2)[I-A-BC0][xtut]=[Ed^(k+1)yt-Dd^(k+1)],umin⁡≤ut≤umax⁡,(4.3)[1υ(k)⋆AN[x^(k+1)-xt]+A~Bπ(k+1)X^(k)]≥0,(4.4)(υ(k)UjY^(k)X^(k)-1Y^(k)TUjT)1/2≤(umax⁡-ut)j,(υ(k)UjY^(k)X^(k)-1Y^(k)TUjT)1/2≤(ut-umin⁡)j,j=1,…,m,
where υ(k)=γ2*(k)/(γ2*(k)-W(χ^(k+N∣k),ω*(k+N∣k))+(1-ϱ)W(χ^(k∣k),ω*(k∣k))), with ϱ∈(0,1] being a given design parameter. Equation (4.1) is utilized for (3.1); (4.2) is utilized for (3.3).Theorem 4.1.
For the system (2.1) subject to the input constraints under Assumptions 2.1-2.2, the closed-loop output feedback model predictive control system, with objective function J-(χ^(k),π(k)), augmented observer (2.4), target generation procedure (at k=0, (3.1)–(3.4); at any k+1, (3.1), (3.3), (4.1)–(4.4)), and dynamic optimization problem (3.25)–(3.30), is input-to-state (ISS) stable if the following two conditions are satisfied. (a)
There exist feasible solutions(xt(k),ut(k)) to the target generation problem, at each control interval.(b)
There exist feasible solutions, including a control sequenceπ*(k), a positive-definite matrix X^, and any matrix Y^, at time k=0, to the dynamic optimization problem (3.25)–(3.30).Proof.
By applying the shifted control sequenceπ(k+1), at time k+1, one has
(4.5)γ1(k+1)-γ1*(k)=J0N-1(k+1)-J0N-1*(k)=W(χ^(k+N∣k+1),ω(k+N∣k+1))+∑i=1N-1‍[W(χ^(k+i∣k+1),ω(k+i∣k+1))-W(χ^(k+i∣k),ω*(k+i∣k))]-W(χ^(k∣k),ω*(k∣k)).
By applying Lemmas 3.1-3.2, it is shown that
(4.6)γ1(k+1)-γ1*(k)≤W(χ^(k+N∣k+1),ω(k+N∣k+1))+Lx∑i=1N-1‍[∑j=0i-2‍‖A‖i-1(‖Fs1C‖‖x~(k)‖+‖Fs1D‖‖d~(k)‖+‖xt(k)-xt(k+1)‖)+∑j=0i-2‍‖A‖j‖B‖(‖ut(k)-ut(k+1)‖)]+(N-1)Lu‖ut(k)-ut(k+1)‖-W(χ^(k∣k),ω*(k∣k)).
By further applying
(4.7)χ^(k+N∣k+1)=χ^(k+N∣k)+AN-1[-Fs1(Cx~(k)+Dd~(k))+xt(k)-xt(k+1)]+∑i=0N-2‍AiB[ut(k)-ut(k-1)],
it is shown that
(4.8)γ1(k+1)-γ1*(k)≤W(χ^(k+N∣k),ω*(k+N∣k))+L~x(‖Fs1C‖‖x~(k)‖+‖Fs1D‖‖d~(k)‖+‖xt(k)-xt(k+1)‖)+L~u‖ut(k)-ut(k+1)‖-W(χ^(k∣k),ω*(k∣k)),
where ℒ~x,ℒ~u>0 are appropriate scalars.
On the other hand, at timek+1, since the target generation problem is feasible, it is feasible to choose γ2(k+1)=γ2*(k)-W(χ^(k+N∣k),ω*(k+N∣k)+(1-ϱ)W(χ^(k∣k),ω*(k∣k)).
Then,(4.9)(γ1(k+1)+γ2(k+1))-(γ1*(k)+γ2*(k))≤-ϱW(χ^(k∣k),ω*(k∣k))+L~x(‖Fs1C‖‖x~(k)‖+‖Fs1D‖‖d~(k)‖+‖xt(k)-xt(k+1)‖)+L~u‖ut(k)-ut(k+1)‖≤-ϱλmin⁡(Q)‖χ^(k∣k)‖+L~x(∥Fs1C∥∥x~(k)∥+‖Fs1D‖‖d~(k)‖+‖xt(k)-xt(k+1)‖)+L~u‖ut(k)-ut(k+1)‖.
Hence, γ1*(k)+γ2*(k) can serve as an ISS (for the definition of this term, see [22]) Lyapunov function, and the closed-loop system is input-to-state stable.If we use the terminal equality constraint, rather than the terminal inequality constraint, then (3.27) should be revised as
(4.10)ANχ^(k)+A~Bπ=0
and (3.28), (3.29) should be removed; moreover, (4.3) should be revised as
(4.11)AN(x^(k+1)-xt)+A~Bπ(k+1)=0
with the shifted control sequence
(4.12)π(k+1)=[(ω*(k+N-1∣k)+ut(k)-ut(k+1))T,(ut(k)-ut(k+1))T]T(ω*(k+1∣k)+ut(k)-ut(k+1))T,…,(ω*(k+N-1∣k)+ut(k)-ut(k+1))T,(ut(k)-ut(k+1))T]T,
and (4.4) should be removed.Theorem 4.2.
For the system (2.1) subject to the input constraints under Assumptions 2.1–2.2, the closed-loop output feedback model predictive control system, with objective function J-(χ^(k),π(k)), augmented observer (2.4), target generation procedure (at k=0, (3.1)–(3.4); at any k+1, (3.1), (3.3), (4.1), (4.2), (4.11)), and dynamic optimization problem (3.25), (3.26), (4.10), (3.30), is input-to-state stable if the following two conditions are satisfied. (a)
There exist feasible solutions(xt(k),ut(k)) to the target generation problem, at each time k.(b)
There exist feasible solutionsπ*(k), at time k=0, to the dynamic optimization problem (3.25), (3.26), (4.10), (3.30).Proof.
By applying the shifted control sequenceπ(k+1), at time k+1, one has
(4.13)γ1(k+1)-γ1*(k)=W(χ^(k+N∣k+1),ω(k+N∣k+1))+∑i=1N-1‍[W(χ^(k+i∣k+1),ω(k+i∣k+1))-W(χ^(k+i∣k),ω*(k+i∣k))]-W(χ^(k∣k),ω*(k∣k))=[W(χ¯(k+N∣k+1),ω(k+N∣k+1))-W(χ¯(k+N∣k),ω(k+N∣k))]+∑i=1N-1‍[W(χ^(k+i∣k+1),ω(k+i∣k+1))-W(χ^(k+i∣k),ω*(k+i∣k))]-W(χ^(k∣k),ω*(k∣k)).
By analogy to Theorem 4.1, it is shown that γ1*(k) can serve as an ISS Lyapunov function, and the closed-loop system is input-to-state stable.Assume thatA is nonsingular. Then, applying (4.11) yields
(4.14)xt=x^(k+1)+A-NA~Bπ(k+1).
Further applying (4.3) yields yt=Cx^(k+1)+CA-NA~Bπ(k+1)+Dd^(k+1) and
(4.15)But=(I-A)[x^(k+1)+A-NA~Bπ(k+1)]-Ed^(k+1).
Hence, by applying (4.10)-(4.11), an analytical solution of the steady-state controller may be obtained.


## 5. Numerical Example

Let us consider the heavy fractionator, which is a Shell standard problem, with the following model:(5.1)GU(s)=[4.05e-27s50s+11.77e-28s60s+15.88e-27s50s+15.39e-18s50s+15.72e-14s60s+16.90e-15s40s+14.38e-20s33s+14.42e-22s44s+17.2019s+1],GF(s)=[1.2e-27s45s+11.44e-27s60s+11.52e-18s25s+11.83e-15s20s+11.1427s+11.2632s+1],
where GU(s) is the transfer function matrix between inputs and outputs, and GF(s) between disturbances and outputs. The three inputs of the process are the product draw rates from the top and side of the column (u1, u2), and the reflux heat duty for the bottom of the column (u3). The three outputs of the process represent the draw composition (y1) from the top of the column, the draw composition (y2), and the reflux temperature at the bottom of the column (y3). The two disturbances are the reflux heat duties for the intermediate section and top of the column (d1, d2).The inputs are constrained between-0.5 and 0.5, while the outputs between -0.5 and 0.5. The weighting matrices are identity matrices. N=3. The sampling interval is 3 seconds. With the algorithm as in Theorem 3.3 applied, the simulation results are shown in Figure 1. The steady-state calculation begins running at instant k=20, when the optimizer finds the optimum target yt=[-0.5,0.5,-0.4269]T. The objective value is -0.3538, indicating that -0.3538 unit benefits are obtained. During time k=200–300, the disturbances d1=-1.3 and d2=1 are added. The simulation verifies our theoretical results.Figure 1
The closed-loop output trajectories, the corresponding control input signals, and the disturbances.


## 6. Conclusions

We have given some preliminary results for the stability of double-layered MPC. The results cannot be seen as the strict synthesis approaches; rather, they are endeavors towards this kind of approaches. Instead of asymptotic stability, we obtain the input-to-state stability, as in [22]. The results are inspired by [22]; but they are much different, as shown in Remarks 1–11 of [21].We believe that several works need to be continued. Indeed, assuming feasibility of the target generation problem at each control interval is very restrictive, and overlooking the uncertainties in the prediction model brings difficulties for proving both the asymptotic stability and offset-free property. It may be necessary to develop a whole procedure, where the target generation problem is guaranteed (rather than assumed) to be feasible at each control interval and an augmented system is used for the stability analysis.



---

*Source: 808327-2012-07-29.xml*