# MPC/LQG for Infinite-Dimensional Systems Using Time-Invariant Linearizations

Peter Benner, Sabine Hein

► To cite this version:

Peter Benner, Sabine Hein. MPC/LQG for Infinite-Dimensional Systems Using Time-Invariant Linearizations. 25th System Modeling and Optimization (CSMO), Sep 2011, Berlin, Germany. pp.217-224, 10.1007/978-3-642-36062-6_22. hal-01347540

HAL Id: hal-01347540

https://inria.hal.science/hal-01347540

Submitted on 21 Jul 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
MPC/LQG for infinite-dimensional systems
using time-invariant linearizations

Peter Benner¹ and Sabine Hein²

¹ Max Planck Institute for Dynamics of Complex Technical Systems, Sandtorstr. 1,
39106 Magdeburg, Germany
benner@mpi-magdeburg.mpg.de
http://www.mpi-magdeburg.mpg.de/mpcsc/benner/

² Voith Engineering Services GmbH,
Aue 23–27, 09112 Chemnitz
sabine.hein@mathematik.tu-chemnitz.de

**Abstract.** We provide a theoretical framework for model predictive con-
trol of infinite-dimensional systems, like, e.g., nonlinear parabolic PDEs,
including stochastic disturbances of the input signal, the output mea-
surements, as well as initial states. The necessary theory for implement-
ing the MPC step based on an LQG design for infinite-dimensional linear
time-invariant systems is presented. We also briefly discuss the necessary
ingredients for the numerical computations using the derived theory.

# 1 Introduction

The control of nonlinear processes is a fundamental problem in engineering. A
usual strategy for computer-aided control consists in pre-computing, in an off-
line phase, an optimal trajectory and control input, and in the implementation
(online phase) to endow the system with a feedback controller in order to com-
pensate for external disturbances and deviations from the optimized trajectory.
A successful strategy for designing a nonlinear control scheme for complex dy-
namical systems, whose global optimization is impossible in real time, is model
predictive control (MPC), see, e.g., [10, 14]. In this approach, the behavior of
the dynamical process is predicted on a small (local) time horizon and then
optimized for a certain time interval using an auxiliary problem for which the
computational solution of the local optimization problem is feasible in real-time.
The control strategy is then applied for a small time step, the process is advanced
in time, and for the next time step, prediction and optimization are repeated for
the new state of the system based on new available measurements. Under certain
conditions, this process converges to the optimal solution of the global control
problem if the time steps and prediction/optimization horizons tend to zero,
see, e.g., [10, 14] and references therein. If used as a feedback control scheme, the
"optimization" goal is to minimize the deviation from the desired trajectory so
that stabilization, i.e., convergence to zero, becomes the goal.

If the state is not fully available in the prediction step, one is faced with the
problem of incomplete observations. This requires to include a state estimator in
the prediction/optimization step. If the local optimization problem is solved via an auxiliary linear-quadratic optimal control (LQ) problem (based on a suitable linearization of the nonlinear system) without control and state constraints, the optimal state estimator in a least-squares sense is given by the Kalman(-Bucy) filter [19], and the solution of the local LQ problem is obtained by the linear-quadratic Gaussian (LQG) design, consisting of a combination of the Kalman filter and the linear-quadratic regulator (LQR). This requires the solution of two Riccati equations. Depending on whether the linearization is time-invariant or time-varying, these are the algebraic or differential Riccati equations (ARE or DRE, respectively) — see, e.g., [12] or any other textbook on linear control design. In [18], this MPC/LQG approach was suggested for finite-dimensional problems. Here, we will extend this idea to infinite-dimensional systems.

In the following we consider the control problem

$$ \min \int_{0}^{T_f} \langle y(t), Q(t)y(t) \rangle_Y + \langle u(t), R(t)u(t) \rangle_U dt + G(x(T_f)), \quad (1) $$

subject to the semi-linear stochastic system with additive unmodeled disturbance

$$ \dot{x}(t) = f(x(t)) + Bu(t) + Fv(t), \quad t > 0, \quad x(0) = x_0 + \eta, \quad (2) $$

$u(t) \in \mathcal{U}$, $x(t) \in \mathcal{X}$, where $v(t)$ is an unknown Gaussian disturbance process with covariance $V$ and $\eta$ denotes the noise in the initial condition. Since in many applications the state is not completely available we introduce the output function (simulating measurements)

$$ y(t) = Cx(t) + w(t), \quad y \in \mathcal{Y}, $$

where $w(t)$ is a measurement noise process which will also be assumed to be Gaussian with covariance $W$. If (2) is an ordinary differential equation then we have a finite-dimensional problem with $\mathcal{X} = \mathbb{R}^n$, $\mathcal{Y} = \mathbb{R}^p$ and $\mathcal{U} = \mathbb{R}^m$. In the case of a partial differential equation (PDE), the problem is infinite-dimensional and $\mathcal{X}, \mathcal{Y}, \mathcal{U}$ are appropriate Hilbert spaces. Here, $B, F, Q, R$ are linear operators on these Hilbert spaces, $f$ is a nonlinear map, and $\langle \cdot, \cdot \rangle$ are inner products on the respective Hilbert spaces.

The outline of the paper is as follows: in the next section, we will briefly sketch the MPC/LQG control design. In Section 3, we will then provide the necessary theoretical background to solve the local LQG problems for infinite-dimensional systems and briefly discuss a framework for the numerical approximation of the solution of the AREs to be solved in a practical implementation of the infinite-dimensional controller. Note that we have demonstrated the efficiency of the suggested MPC/LQG approach for the stabilization of the noisy Burgers equation in [4] and for a 3D reaction-diffusion system in [6]. Concluding remarks are provided in Section 4.
## 2 The MPC/LQG controller

Given a reference trajectory and control $(\bar{x}(t), \bar{u}(t))$ obtained, e.g., from an offline optimization procedure, in the following we design a model predictive feedback control strategy. This MPC/LQG approach is based on a linearization of (2) on small intervals to obtain a linear time-invariant (LTI) or time-varying (LTV) problem. Due to space restrictions, we will concentrate here on the LTI case. For the general strategy in the LTV case, we refer to [5, 15] We solve this linear problem on a small interval by using an LQG design. Note that we write $M^*$ to denote the adjoint operator corresponding to the linear operator $M$ and the derivative of $f$ from (2) is to be understood as the Fréchet derivative. With these preliminaries, the strategy is the following:

(1) **Prediction and optimization step on $[t_i, t_i + T_p]$, $t_i + T_p \le T_f$:**
linearize (2) around a given set point $\bar{x}$ to obtain $A = f'(\bar{x}(t_i))$ and the linear state equation

$$\dot{z}(t) = Az(t) + B\tilde{u}(t) + Fv(t), \quad z(t_i) = x(t_i) - \bar{x}(t_i), \quad y(t) = Cx(t) + w(t),$$

with $z(t) = x(t) - \bar{x}(t)$ and $\tilde{u}(t) = u(t) - \bar{u}(t)$. Then solve the ARE

$$0 = XA + A^*X - XBR^{-1}B^*X + C^*QC \quad (3)$$

in order to obtain $X_\star$ and $K = -R^{-1}B^*X_\star$.

(2) **Implementation step on $[t_i, t_i + T_c]$, $T_c \le T_p$:**

solve the filter ARE (FARE)

$$0 = A\Sigma + \Sigma A^* - \Sigma C^*W^{-1}C\Sigma + FVF^*, \quad (4)$$

where $V$, $W$ are the covariance matrices of the noise processes. Feed the real system on $[t_i, t_i + T_c]$ with

$$u(t) = \bar{u}(t) - K(\hat{x}(t) - \bar{x}(t)),$$

and obtain the “measurement” $y(t)$ by solving the nonlinear system on $[t_i, t_i + T_c]$. Estimate the state by $\hat{x}(t)$ by solving

$$\dot{\hat{z}}(t) = A\hat{z}(t) + B\tilde{u}(t) + L(y(t) - C\hat{x}(t)), \quad \hat{z}(t) = \hat{x}(t) - x^*(t),$$

using the estimator gain $L = \Sigma_\star C^*W^{-1}$.

(3) **Receding Horizon Step:**

update $t_{i+1} = t_i + T_c$ and go to the first step.

Note that the solutions of the AREs (3) and (4) are linear selfadjoint operators on $D(A)$, the domain of $A$, and $D(A^*)$, respectively.
Some remarks are in order:

**Remark 1.** For the finite-dimensional case, if $G$ in (1) is selected as a control Lyapunov function, Ito and Kunisch established the asymptotic stability and estimated the performance for the receding horizon synthesis in [16]. Analogous results for the LTV case in the slightly more general MPC setting are obtained in [5, 15].

**Remark 2.** Solving the AREs (3) and (4) yields an LQG controller for an infinite time-horizon. Therefore, we can also consider this scheme as an MPC scheme with infinite prediction and optimization horizon.

In the following, we will discuss an appropriate setting in which this procedure is well-posed and can be approximated using an appropriate numerical scheme. Note that using efficient numerical algorithms, large-scale AREs resulting from discretized PDE control problems can be solved in a reasonable time-scale, see [7]. Whether or not this is real-time feasible depends on the control horizon $T_c$ of the process. Further advances in computer hardware and improvements of the numerical algorithms will certainly allow real-time solution of AREs for moderately fast processes with medium-fine granularity of the discretization in the near future.

## 3 Infinite-dimensional LQG theory

Consider the following nonlinear optimal control problem:

$$ \min \mathcal{J}(u) := \langle x_{T_f}, Gx_{T_f} \rangle_X + \int_0^\infty \langle x(t), C^*QCx(t) \rangle_X + \langle u(t), Ru(t) \rangle_U dt, \quad (5) $$

$$ \begin{array}{ll} \text{subject to} & \dot{x}(t) = f(x(t)) + Bu(t) + Fv(t), \quad t > 0, \\ & y(t) = Cx(t) + w(t), \quad t > 0, \\ & x(0) = x_0 + \eta. \end{array} $$

Following the infinite-dimensional LQG theory derived in [11] and denoting the set of linear maps from $\mathcal{M}$ to $\mathcal{N}$ by $\mathcal{L}(\mathcal{M}, \mathcal{N})$, we will assume the following:

### Assumption 1

- $\mathcal{X}, \mathcal{Y}, \mathcal{U}$ are Hilbert spaces, $f: \mathcal{D}(f) \subseteq \mathcal{X} \to \mathcal{X}$ is a nonlinear map;

- $B \in \mathcal{L}(\mathcal{U}, \mathcal{X})$, $F \in \mathcal{L}(\mathcal{U}, \mathcal{X})$, $C \in \mathcal{L}(\mathcal{X}, \mathcal{Y})$, $G \in \mathcal{L}(\mathcal{X})$;

- $Q \in \mathcal{L}(\mathcal{Y})$, $R, R^{-1} \in \mathcal{L}(\mathcal{U})$, all self-adjoint and nonnegative and $\langle \nu, R\nu \rangle \ge \alpha||\nu||^2$ for all $\nu \in \mathcal{U}$ and some $\alpha > 0$;

- $x_0 \in \mathcal{X}, \eta$ is a zero mean Gaussian random variable on $\mathcal{X}$ with covariance $\Sigma_0$,
- $v(t)$ and $w(t)$ are Wiener processes (Gaussian with zero mean) on the Hilbert spaces $\mathcal{U}$ and $\mathcal{Y}$ with incremental covariance operators $V \in \mathcal{L}(\mathcal{U})$ and $W, W^{-1} \in \mathcal{L}(\mathcal{Y})$, respectively.

Assuming that $f(x)$ is Fréchet-differentiable and linearizing on small intervals $[t_i, t_i + T_p]$ around a stationary operating point $\bar{x}$, we obtain the stochastic LTI problem in differential form on $[t_i, t_i + T_p]$:

$$ 
\begin{aligned}
dz(t) &= Az(t)dt + B\tilde{u}(t)dt + Fdv(t), \quad t_i < t < t_i + T_p, \\
d\tilde{y}(t) &= Cz(t)dt + dw(t), \quad t_i < t < t_i + T_p, \\
z(t_i) &= z_{t_i},
\end{aligned} 
\quad (6) $$

with $z(t) := h(t) = x(t) - \bar{x}(t)$, $\tilde{u}(t) = u(t) - \bar{u}(t)$ and

$$ f(\bar{x} + h)(t) \approx f(\bar{x}(t)) + Ah(t), $$

where $A := f'(\bar{x}(t_i))$ is the Fréchet derivative of $f$, evaluated at $\bar{x}(t_i)$.

Note that the linear system (6) is only a local approximation to the original nonlinear system, but nevertheless we will use it to solve the LQG problem on an infinite horizon. The so obtained control is then only applied locally on the control horizon $[t_i, t_i + T_c]$, then the prediction horizon is shifted by $T_c$, and a new linearization based on $\bar{x}(t_i + T_c)$, leading to a new LQG problem, is used.

To avoid problems of existence and uniqueness of the stochastic evolution equation (6), we use its integral form on $[t_i, t_i + T_p]$:

$$ 
\begin{aligned}
z(t) &= S_{t-t_i}z(t_i) + \int_{t_i}^{t} S_{t-s}B\tilde{u}(s)ds + \int_{t_i}^{t} S_{t-s}Fdv(s), \\
t_i \le s \le t \le t_i + T_p, \\
\tilde{y}(t) &= \int_{t_i}^{t} Cz(s)ds + w(t), \quad t_i < t \le t_i + T_p, \\
z(t_i) &= z_{t_i},
\end{aligned} 
\quad (7) $$

where $S_t$ is a strongly continuous semigroup on $\mathcal{X}$ generated by $A$ on $[t_i, t_i + T_p]$ (see, e.g., [13] for the notion of semigroups and their properties).

A direct consequence of results from [11] is then the following theorem which yields the solution to the MPC/LQG/LTI problem on $[t_i, t_i + T_p]$ for $T_p = \infty$:

**Theorem 1.** Under the Assumptions 1., the optimal control and corresponding estimated state for the minimization problem (5) subject to (7) on $[t_i, t_i + T_p]$ are given by

$$ 
\begin{aligned}
u_*(t) &= u_r(t) - R^{-1}B^*\Pi_\infty(\hat{x}_*(t) - \bar{x}(t)), \\
\hat{x}_*(t) &= S_{t_i}\hat{x}(t_i) + \int_{t_i}^{t} S_{t-s}\Sigma_\infty C^*W^{-1}dy(s) + \int_{t_i}^{t} S_{t-s}(f(\bar{x}(s)) - A\bar{x}(s))ds,
\end{aligned} 
$$
where $S_t$ is the strongly continuous semigroup generated by

$$A - BR^{-1}B^*\Pi_{\infty} - \Sigma_{\infty}C^*W^{-1}C,$$

and $\Pi_{\infty}$ and $\Sigma_{\infty}$ are the unique nonnegative, self-adjoint solutions of the ARE

$$0 = A^*\Pi + \Pi A - \Pi B R^{-1} B^*\Pi + C^*QC,$$

and the FARE

$$0 = A\Sigma + \Sigma A^* - \Sigma C^*W^{-1}C\Sigma + FVF^*.$$

Note again that the global solution of the LQG problem which is computed for $T_p = \infty$ in the above theorem is only used locally on $[t_i, t_i + T_c]$. In this way, we can implement the infinite-dimensional MPC/LQG controller for any infinite dimensional system satisfying the assumptions made in this section.

Checking the Assumptions 1 for a practical problem is tedious. In [15], as an example the control problem for the Burgers equation

$$\begin{align*}x_t(t, \xi) &= \nu x_{\xi\xi}(t, \xi) - x(t, \xi)x_{\xi}(t, \xi) + b(\xi)u(t), \quad \text{on } (0, T_f] \times (0, 1) \\x(t, 0) &= x(t, 1) = 0, \quad t \in (0, T_f], \\x(0, \xi) &= x_0(\xi), \quad \xi \in (0, 1),\end{align*}$$

is considered and it is shown that linearization about a set point leads to an infinite-dimensional LTI system satisfying the Assumptions 1.

For a practical implementation, it is of course necessary to discretize the infinite-dimensional system and to work with a finite-dimensional approximation. If the infinite-dimensional problem is defined via a PDE, this can be achieved using a spatial semi-discretization based on finite differences or finite elements. As both A and its adjoint $A^*$ appear in the formulation of the problem and their discretizations $A_h$ and $A_h^T$ are used to define the finite-dimensional AREs that need to be solved to obtain the approximate feedback and estimator gain matrices $K_h$ and $L_h$, standard convergence results for finite element discretizations are not sufficient. The necessary conditions for dual convergence are stated in [2] for linear parabolic equations with distributed control and are generalized in [8] to boundary control problems. Both papers only consider the LQR problem, the extension to the LQG case is rather straightforward and is executed in [15, Section 8.2.6]. The numerical solution of the resulting large-scale AREs is the computational bottleneck of the suggested control approach. The effective solution of large-scale AREs and associated LQR problems is discussed, e.g., in [1, 3, 7, 20]. The main idea is to apply a Newton-type method to the quadratic nonlinear systems of equations defined by the AREs and to solve the Newton steps by effective iterative methods. It should be noted that for LQG design as discussed here, the actual solution operators/matrices are not necessary as one is only interested in the feedback and estimator gain matrices K and L. Note that the Newton iteration for AREs can be re-written in such a way that one directly iterates on approximations to these operators rather than on approximations to
the ARE solutions. This saves a significant amount of workspace and computa-
tional effort and is thus recommended in the context of the suggested MPC/LQG
scheme. For an efficient variant of the Newton-Kleinman iteration suitable for
large-scale AREs, see [9]. Numerical examples demonstrating the effectiveness
of the proposed MPC/LQG feedback control design for PDE-constrained opti-
mization problems are shown in [4] for the Burgers equation and in [5] for a
bilinear 3D reaction-diffusion system.

4 Conclusions

We have presented a framework for model predictive control of infinite-dimensional nonlinear systems subject to stochastic perturbations based on an LQG design to implement the optimization step. This includes the state estimation using a Kalman filter. Linearization about the set point leads to an LTI system. We have focused here on the necessary theoretical ingredients to render this step well-posed. Sufficient conditions for convergence of a numerical approximation scheme to implement the LQG design in a computational procedure can be derived, but are not detailed here due to space restrictions. Though stabilization properties of the nonlinear infinite-dimensional MPC/LQG controller have not been shown yet, numerical experiments in [4, 5] illustrate the good performance of this control scheme. Further improvements can be obtained if one allows for time-varying linearizations in the optimization step, i.e., linearization around the reference trajectory. The treatment of this case is similar to the LTI case and will be described, together with further numerical experiments, in a forthcoming detailed publication. A convergence and stabilization proof of the infinite-dimensional design based on ideas presented in [16–18] is in progress. Further investigations are necessary in order to make the approach real-time feasible. This may require algorithmic improvements in the Riccati solvers or the inclusion of a model reduction strategy in the prediction and optimization step.

References

1. Banks, H., Ito, K.: A numerical algorithm for optimal feedback gains in high dimensional linear quadratic regulator problems. SIAM J. Cont. Optim. 29(3), 499–515 (1991)

2. Banks, H., Kunisch, K.: The linear regulator problem for parabolic systems. SIAM J. Cont. Optim. 22, 684–698 (1984)

3. Benner, P.: Solving large-scale control problems. IEEE Control Systems Magazine 14(1), 44–59 (2004)

4. Benner, P., Görner, S.: MPC for the Burgers equation based on an LGQ design. Proc. Appl. Math. Mech. 6(1), 781–782 (2006)

5. Benner, P., Hein, S.: Model predictive control based on an LQG design for time-varying linearizations. Preprint CSC/09-07, Chemnitz Scientific Computing Preprints, Fakultät für Mathematik, TU Chemnitz, http://nbn-resolving.de/urn:nbn:de:bsz:ch1-201000221 (2009)
6. Benner, P., Hein, S.: Model predictive control for nonlinear parabolic differential equations based on a linear quadratic Gaussian design. Proc. Appl. Math. Mech. 9(1), 613–614 (2009)

7. Benner, P., Li, J.R., Penzl, T.: Numerical solution of large Lyapunov equations, Riccati equations, and linear-quadratic control problems. Numer. Lin. Alg. Appl. 15(9), 755–777 (2008)

8. Benner, P., Saak, J.: Linear-quadratic regulator design for optimal cooling of steel profiles. Tech. Rep. SFB393/05-05, Sonderforschungsbereich 393 *Paralleleu Nummerische Simulation für Physik und Kontinuumsmechanik*, TU Chemnitz, 09107 Chemnitz, FRG (2005), available from http://www.tu-chemnitz.de/sfb393.

9. Benner, P., Saak, J.: A Galerkin-Newton-ADI Method for Solving Large-Scale Algebraic Riccati Equations. Preprint SPP1253-090, DFG Priority Program "Optimization with Partial Differential Equations" (SPP1253) (January 2010), http://www.am.uni-erlangen.de/home/spp1253/wiki/index.php/Preprints

10. Camacho, E., Bordons, C.: Model Predictive Control. Advanced Textbooks in Control and Signal Processing, Springer-Verlag, London, 2nd edn. (2004)

11. Curtain, R., Pritchard, T.: Infinite Dimensional Linear System Theory, Lecture Notes in Control and Information Sciences, vol. 8. Springer-Verlag, New York (1978)

12. Datta, B.: Numerical Methods for Linear Control Systems. Elsevier Academic Press (2004)

13. Engel, K.L., Nagel, R.: One-Parameter Semigroups for Linear Evolution Equations. Springer-Verlag, New York (2000)

14. Grüne, L., Pannek, J.: Nonlinear Model Predictive Control: Theory and Algorithms. Springer-Verlag, London (2011)

15. Hein, S.: MPC/LQG-Based Optimal Control of Nonlinear Parabolic PDEs. Ph.D. thesis, TU Chemnitz, Department of Mathematics (March 2010), available from http://nbn-resolving.de/urn:nbn:de:bsz:ch1-201000134

16. Ito, K., Kunisch, K.: On asymptotic properties of receding horizon optimal control. SIAM J. Cont. Optim. 40, 1455–1472 (2001)

17. Ito, K., Kunisch, K.: Receding horizon optimal control for infinite dimensional systems. ESAIM: Control Optim. Calc. Var. 8, 741–760 (2002)

18. Ito, K., Kunisch, K.: Receding horizon control with incomplete observations. SIAM J. Cont. Optim. 45(1), 207–225 (2006)

19. Kalman, R., Bucy, R.: New results in linear filtering and prediction theory. Trans. ASME, Series D 83, 95–108 (1961)

20. Morris, K., Navasca, C.: Solution of algebraic Riccati equations arising in control of partial differential equations. In: Control and boundary analysis, Lect. Notes Pure Appl. Math., vol. 240, pp. 257–280. Chapman & Hall/CRC, Boca Raton, FL (2005)