## Appendix A Proof of the Proposition 1  
To start the synthesis of the control sequence from the time horizon \(t_{f}=T_{s}N\) backward, define the value of the performance index in the discrete time \(N\):  
\[J_{N,N}^{*}=\frac{1}{2}\tilde{x}_{m}^{\intercal}(N)H\tilde{x}_{m}(N).\]  
In this stage, the optimal value for the control \(\tilde{a}_{m}^{*}(N)\) is zero because the expression (7) does not explicitly depend on the control law \(\tilde{a}_{m}(N)\). According to the system (4), \(\tilde{x}_{m}(N)\) depends only on the control \(\tilde{a}_{m}(N-1)\). Then, \(J_{N,N}^{*}\) is the optimal value of \(J\) in \(N\). Now, the summation given by (7) is developed from \(N-1\) up to \(N\), and an optimization with respect to the control \(\tilde{a}_{m}(N-1)\) is made as follows:  
\[J_{N-1,N}^{*} = \min_{\tilde{a}(N-1)}\biggl{\{}\frac{1}{2}\tilde{x}_{m}^{ \intercal}(N)H\tilde{x}_{m}(N)+\ldots\] \[\ldots+\frac{1}{2}\tilde{x}_{m}^{\intercal}(N-1)Qx_{m}(N-1)+ \frac{1}{2}\tilde{a}_{m}^{\intercal}(N-1)R\tilde{a}_{m}(N-1)\biggr{\}}\] \[= \min_{\tilde{a}_{m}(N-1)}\biggl{\{}J_{N,N}^{*}+\frac{1}{2} \tilde{x}_{m}^{\intercal}(N-1)Qx_{m}(N-1)+\ldots\] \[\frac{1}{2}\tilde{a}_{m}^{\intercal}(N-1)R\tilde{a}_{m}(N-1) \biggr{\}}.\]  
According to (4), the state \(\tilde{x}_{m}(N)\) depends on the control \(\tilde{u}_{m}(N-1)\), so one substitutes the right side of the expression given by (4) as follows:  
\[J_{N-1,N}^{*}(\tilde{x}_{m}(N-1),\tilde{u}_{m}(N-1))=\min_{\tilde{a}_{m}(N-1 )}\biggl{\{}\frac{1}{2}[f_{m}(\tilde{x}_{m}(N-1))\]  
\[\ldots+g_{m}\tilde{u}_{m}(N-1)]^{T}\times H[f(\tilde{x}_{m}(N-1))+g_{m}\tilde {u}_{m}(N-1)]\]  
\[\ldots+\frac{1}{2}\tilde{x}_{m}^{\intercal}(N-1)Qx_{m}(N-1)+\frac{1}{2} \tilde{a}_{m}^{\intercal}(N-1)R\tilde{u}_{m}(N-1)]\biggr{\}}. \tag{11}\]Observe that index \(J_{N-1,N}^{*}\) only depends on \(\bar{x}_{m}(N-1)\) and \(\bar{u}_{m}(N-1)\). As \(\bar{x}_{m}(N-1)\) depends on \(\bar{u}_{m}(N-2)\), the expression given by (A1) can be minimized with respect to \(\bar{u}_{m}(N-1)\). Please notice that the expression of \(J_{N-1,N}\) is strongly convex; it guarantees the existence of a local minimum with respect to \(\bar{u}_{m}(N-1)\). Indeed, consider the following sufficient conditions [33]:  
\[\frac{\partial J_{N-1,N}}{\partial\bar{u}_{m}(N-1)}=0,\] (A2)  
\[\frac{\partial^{2}J_{N-1,N}(\bar{x}_{(N-1)},\bar{u}_{m}(N-1))}{\partial^{2} \bar{u}_{m}(N-1)}>0.\] (A3)  
Based on the first necessary condition given by (A2), it follows that:  
\[J_{N-1,N}^{*} = \min_{\tilde{a}(N-1)}\biggl{\{}\frac{1}{2}\tilde{x}_{m}^{ \intercal}(N)H\tilde{x}_{m}(N)+\ldots\]0 (A4)  
The inverse of \([g_{m}^{T}Hg_{m}+R]\) exists due to matrix \(R\) being positive definite. Now, based on the sufficient condition given by (A3), it follows that:  
\[J_{N-1,N}^{*} = \min_{\tilde{a}(N-1)}\biggl{\{}\frac{1}{2}\tilde{x}_{m}^{ \intercal}(N)H\tilde{x}_{m}(N)+\ldots\]1  
Then, the controller given by (A4) is optimal in the local sense. For the discrete instant \(N-2\), the control law \(\bar{u}_{m}^{*}(N-1)\) is the optimal value in step \(N-1\), and according to the Bellman optimality principle, this control generates the optimal value for \(\bar{x}_{m}(N-1)\). It follows that for \(N-2\), this yields the following:  
\[J_{N-1,N}^{*} = \min_{\tilde{a}(N-1)}\biggl{\{}\frac{1}{2}\tilde{x}_{m}^{ \intercal}(N)H\tilde{x}_{m}(N)+\ldots\]2  
\[J_{N-1,N}^{*} = \min_{\tilde{a}(N-1)}\biggl{\{}\frac{1}{2}\tilde{x}_{m}^{ \intercal}(N)H\tilde{x}_{m}(N)+\ldots\]3  
\[J_{N-1,N}^{*} = \min_{\tilde{a}(N-1)}\biggl{\{}\frac{1}{2}\tilde{x}_{m}^{ \intercal}(N)H\tilde{x}_{m}(N)+\ldots\]4  
\[J_{N-1,N}^{*} = \min_{\tilde{a}(N-1)}\biggl{\{}\frac{1}{2}\tilde{x}_{m}^{ \intercal}(N)H\tilde{x}_{m}(N)+\ldots\]5  
Substituting the right side of the state Equation (4) into the term with \(\bar{x}_{m}(N-1)\), the control \(\bar{u}_{m}(N-2)\) appears explicitly. In the rest of the terms, it is not necessary because they depend on the prior or subsequent control values.