# Learning Output Reference Model Tracking for Higher-Order Nonlinear Systems with Unknown Dynamics +
## 2 Output Model Reference Control for Unknown Dynamics Nonlinear Processes
### Output Reference Model Control Problem Definition  
Let the discrete-time known open-loop stable minimum-phase (MP) state-space deterministic strictly causal ORM be [12; 46]  
\[M:\{\mathbf{x}_{k+1}^{m}=\mathbf{f}^{m}(\mathbf{x}_{k}^{m},\mathbf{r}_{k}), \mathbf{y}_{k}^{m}=\mathbf{g}^{m}(\mathbf{x}_{k}^{m})\}, \tag{2}\]  
where \(\mathbf{x}_{k}^{m}=[x_{k,1}^{m},...,x_{k,n}^{m}]^{\top}\in\Omega_{\mathbf{x}_ {m}}\subset\mathbb{R}_{m}^{m}\) is the ORM state, \(\mathbf{r}_{k}=[r_{k,1},...,r_{k,p}]^{\top}\in\Omega_{R_{m}}\subset\mathbb{R} ^{p}\) is the reference input signal, \(\mathbf{y}_{k}^{m}=[y_{k,1}^{m},...,y_{k,p}^{m}]^{\top}\in\Omega_{\mathbf{y}_ {m}}\subset\mathbb{R}^{p}\) is the ORM output, \(\mathbf{f}^{m}:\Omega_{X_{m}}\times\Omega_{R_{m}}\mapsto\Omega_{X_{m}}\), \(\mathbf{g}^{m}:\Omega_{X_{m}}\mapsto\Omega_{Y_{m}}\) are _known_ nonlinear mappings. Initial conditions are zero unless otherwise stated. Notice that \(\mathbf{r}_{m},\mathbf{y}_{k},\mathbf{y}_{k}^{m}\) are size \(p\) for square feedback control systems (CSs). If the ORM (2) is LTI, it is always possible to express the ORM as an IO LTI transfer function (t.f.) \(\mathbf{M}(z)\) ensuring \(\mathbf{y}_{k}^{m}=\mathbf{M}(z)\mathbf{r}_{k}\), where \(\mathbf{M}(z)\) is commonly an asymptotically stable unit-gain rational t.f. and \(\mathbf{r}_{k}\) is the reference input that drives both the feedback CS and the ORM. We introduce an extended process comprising of the process (1) coupled with the ORM (2). For this, we consider the reference input \(\mathbf{r}_{k}\) as a set of measurable exogenous signals (possibly interpreted as a disturbance) that evolve according to \(\mathbf{r}_{k+1}=\mathbf{h}^{m}(\mathbf{r}_{k})\), with known nonlinear \(\mathbf{h}^{m}:\mathbb{R}^{m}\mapsto\mathbb{R}^{m}\), where \(\mathbf{r}_{k}\) is measurable. Herein, \(\mathbf{h}^{m}(. )\) is a generative model for the reference input (\(\copyright\) 2019 IEEE [12]). The class of LTI generative models \(\mathbf{h}^{m}(. )\) has been studied before in [9] but it is a rather restrictive one. For example, reference inputs signals modeled as a sequence of steps of constant amplitude cannot be modeled by LTI generative models. A step reference input signal with constant amplitude over time can be modeled as \(\mathbf{r}_{k+1}=\mathbf{r}_{k}\) with some initial condition \(\mathbf{r}_{0}\). On the other hand, a sinusoidal scalar reference input signal \(r_{k}\) can be modeled only through a second order state-space model. To see this, let the Laplace transform of \(\cos(\omega t)\sigma(t)\) (\(\sigma(t)\) is the unit step function) be \(H(s)=\ell\{cos(\omega t)\sigma(t)\}\) with the complex Laplace variable \(s\). If \(sH(s)\) is considered a t.f. driven by the unit step function with Laplace transform \(\ell\{\sigma(t)\}=1/s\), then the LTI discrete-time state-space associated with \(sH(s)\) acting as a generative model for \(r_{k}\) is of the form  
\[\mathbf{o}_{k+1} =\mathbf{A}\mathbf{o}_{k}+\mathbf{B}\sigma_{k}, \tag{3}\] \[r_{k} =\mathbf{C}\mathbf{o}_{k}+\mathbf{D}\sigma_{k},\]  
with known \(\mathbf{A}\in\mathbb{R}^{2\times 2},\mathbf{B}\in\mathbb{R}^{2\times 1}, \mathbf{C}\in\mathbb{R}^{1\times 2},D\in\mathbb{R},\mathbf{o}_{0}=[0,0\[\mathbf{x}_{k+1}^{E}= \begin{bmatrix}\mathbf{x}_{k+1}\\ \mathbf{x}_{k+1}^{m}\\ \mathbf{r}_{k+1}\end{bmatrix} = \begin{bmatrix}\mathbf{f}(\mathbf{x}_{k},\mathbf{ u}_{k})\\ \mathbf{f}^{m}(\mathbf{x}_{k}^{m},\mathbf{r}_{k})\\ \mathbf{h}^{m}(\mathbf{r}_{k})\end{bmatrix} =\mathbf{E}(\mathbf{x}_{k}^{E}, \mathbf{u}_{k}),\mathbf{x}_{k}^{E}\in\Omega_{\chi^{E}}, \tag{4}\]  
where \(\mathbf{x}_{k}^{E}\) is called the extended state vector. Note that the extended state-space fulfils the Markov property. The ORM tracking control problem is formulated in an optimal control framework. Let the infinite horizon cost function (c.f.)