## 2 Analytic Solution of State-Constrained Optimal Tracking Problems
### Variational Approach  
By using the Lagrange multiplier method and the penalty function method [23; 24], Problem (7) can be transformed into the following problem:  
\[\underset{\mathbf{x},\mathbf{u}}{\text{minimize}}\ J_{a}=\int_{t_{0}}^{t_{f}}L( \mathbf{x},\dot{\mathbf{x}},\mathbf{u})dt, \tag{17}\]  
\[L(\mathbf{x},\dot{\mathbf{x}},\mathbf{u})=\frac{1}{2}\Big{(}\mathbf{r}-\mathbf{ C}_{y}\mathbf{x}\Big{)}^{T}\mathbf{Q}\Big{(}\mathbf{r}-\mathbf{C}_{y}\mathbf{x} \Big{)}+\frac{1}{2}\mathbf{u}^{T}\mathbf{R}\mathbf{u}+\mathbf{\lambda}^{T}( \mathbf{A}\mathbf{x}+\mathbf{B}\mathbf{u}-\dot{\mathbf{x}})+\frac{1}{2} \mathbf{h}^{T}\text{diag}\Big{(}\mathbf{\alpha}^{T}\Big{)}\,\mathbf{h}, \tag{18}\]  
where \(\mathbf{\lambda}(t)\) is the Lagrange multiplier. Let the Hamiltonian function of (17) be  
\[H(\mathbf{x},\mathbf{u})=\frac{1}{2}\Big{(}\mathbf{r}-\mathbf{C}_{y}\mathbf{x }\Big{)}^{T}\mathbf{Q}\Big{(}\mathbf{r}-\mathbf{C}_{y}\mathbf{x}\Big{)}+ \frac{1}{2}\mathbf{u}^{T}\mathbf{R}\mathbf{u}+\mathbf{\lambda}^{T}(\mathbf{A} \mathbf{x}+\mathbf{B}\mathbf{u})+\frac{1}{2}\mathbf{h}^{T}\text{diag}\Big{(} \mathbf{\alpha}^{T}\Big{)}\,\mathbf{h}, \tag{19}\]  
and \(\mathbf{Q}_{x}=\text{diag}\Big{(}\mathbf{\alpha}_{x}{}^{T}\Big{)}\), \(\mathbf{Q}_{\tau}=\text{diag}\Big{(}\mathbf{\alpha}_{\tau}{}^{T}\Big{)}\). Then, substituting (13) into (19) yields  
\[\begin{array}{rl}H(\mathbf{x},\mathbf{u})&=\frac{1}{2}\Big{(}\mathbf{r}- \mathbf{C}_{y}\mathbf{x}\Big{)}^{T}\mathbf{Q}\Big{(}\mathbf{r}-\mathbf{C}_{y} \mathbf{x}\Big{)}+\frac{1}{2}\mathbf{u}^{T}\mathbf{R}\mathbf{u}+\mathbf{\lambda} ^{T}(\mathbf{A}\mathbf{x}+\mathbf{B}\mathbf{u})+\frac{1}{2}(\mathbf{C}_{h} \mathbf{x}+\mathbf{w})^{T}\mathbf{Q}_{x}(\mathbf{C}_{h}\mathbf{x}+\mathbf{w}) \\ &+\frac{1}{2}(\mathbf{C}_{h}\mathbf{A}_{d}\mathbf{x}+\mathbf{C}_{h}\mathbf{B}_ {d}\mathbf{u}+\mathbf{w})^{T}\mathbf{Q}_{\tau}(\mathbf{C}_{h}\mathbf{A}_{d} \mathbf{x}+\mathbf{C}_{h}\mathbf{B}_{d}\mathbf{u}+\mathbf{w}).\end{array} \tag{20}\]  
By Corollary A1, the following equations hold:  
\[\begin{array}{rl}\dot{\mathbf{\lambda}}&=\mathbf{C}_{y}{}^{T}\mathbf{Q}\Big{(} \mathbf{r}-\mathbf{C}_{y}\mathbf{x}\Big{)}-\mathbf{A}^{T}\mathbf{\lambda}-\mathbf{ C}_{h}{}^{T}\mathbf{Q}_{x}(\mathbf{C}_{h}\mathbf{x}+\mathbf{w})-\mathbf{A}_{d} {}^{T}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}(\mathbf{C}_{h}\mathbf{A}_{d} \mathbf{x}+\mathbf{C}_{h}\mathbf{B}_{d}\mathbf{u}+\mathbf{w})\\ &=-\mathbf{A}^{T}\mathbf{\lambda}-\Big{(}\mathbf{Q}_{1}+\mathbf{A}_{d}{}^{T} \mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}\mathbf{C}_{h}\mathbf{A}_{d}\Big{)}\mathbf{ k}-\mathbf{A}_{d}{}^{T}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}\mathbf{C}_{h} \mathbf{B}_{d}\mathbf{u}+\mathbf{C}_{y}{}^{T}\mathbf{Q}\mathbf{r}\\ &-\Big{(}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{x}+\mathbf{A}_{d}{}^{T}\mathbf{C}_{h}{} ^{T}\mathbf{Q}_{\tau}\Big{)}\mathbf{w},\end{array} \tag{21}\]  
\[0=\mathbf{u}^{T}\mathbf{R}+\mathbf{\lambda}^{T}\mathbf{B}+(\mathbf{C}_{h}\mathbf{ A}_{d}\mathbf{x}+\mathbf{C}_{h}\mathbf{B}_{d}\mathbf{u}+\mathbf{w})^{T}\mathbf{Q}_{ \tau}\mathbf{C}_{h}\mathbf{B}_{d}, \tag{22}\]  
\[\mathbf{Q}_{1}=\mathbf{C}_{y}{}^{T}\mathbf{Q}\mathbf{C}_{y}+\mathbf{C}_{h}{}^{T }\mathbf{Q}_{x}\mathbf{C}_{h}. \tag{23}\]  
Let  
\[\mathbf{R}_{\tau}=\mathbf{R}+\mathbf{B}_{d}{}^{T}\mathbf{C}_{h}{}^{T}\mathbf{ Q}_{\tau}\mathbf{C}_{h}\mathbf{B}_{d}. \tag{24}\]  
Then, the optimal input is  
\[\mathbf{u}=-\mathbf{R}_{\tau}{}^{-1}\mathbf{B}^{T}\mathbf{\lambda}-\mathbf{R}_{ \tau}{}^{-1}\mathbf{B}_{d}{}^{T}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}\mathbf{C} _{h}\mathbf{A}_{d}\mathbf{x}-\mathbf{R}_{\tau}{}^{-1}\mathbf{B}_{d}{}^{T} \mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}\mathbf{w}. \tag{25}\]