## 2 Analytic Solution of State-Constrained Optimal Tracking Problems
### Analytical Solution of the Problem  
The following procedure is the same method used in the derivation of the Riccati Equation (5) and the auxiliary dynamic Equation (6) [7; 8]. **Theorem 1**. : _Assume that the costate is proportional to the state. Then, the costate can be written as_  
\[\mathbf{\lambda}=\mathbf{P}\mathbf{x}-\mathbf{g}, \tag{26}\]  
_where \(\mathbf{P}(t)\) and \(\mathbf{g}(t)\) are values to be determined. Then, the following dynamic equations hold:_  
\[\dot{\mathbf{P}}=-\mathbf{P}\mathbf{A}_{z}-\mathbf{A}_{z}{}^{T}\mathbf{P}+ \mathbf{P}\mathbf{B}\mathbf{R}_{\tau}{}^{-1}\mathbf{B}^{T}\mathbf{P}-\mathbf{ Q}_{2}, \tag{27}\]  
\[\dot{\mathbf{g}} =-\Big{(}\mathbf{A}_{z}{}^{T}-\mathbf{P}\mathbf{B}\mathbf{R}_{ \tau}{}^{-1}\mathbf{B}^{T}\Big{)}\mathbf{g}-\mathbf{C}_{y}{}^{T}\mathbf{Q} \mathbf{r} \tag{28}\] \[+\Big{[}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{x}-\mathbf{P}\mathbf{B} \mathbf{R}_{\tau}{}^{-1}\mathbf{B}_{d}{}^{T}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{ \tau}+\mathbf{A}_{d}{}^{T}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}\Big{(}\mathbf{ I}-\mathbf{C}_{h}\mathbf{B}_{d}\mathbf{R}_{\tau}{}^{-1}\mathbf{B}_{d}{}^{T} \mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}\Big{)}\Big{]}\mathbf{w},\]  
_where_  
\[\mathbf{Q}_{2}=\mathbf{Q}_{1}+\mathbf{A}_{d}{}^{T}\mathbf{C}_{h}{}^{T} \mathbf{Q}_{\tau}\Big{(}\mathbf{I}-\mathbf{C}_{h}\mathbf{B}_{d}\mathbf{R}_{ \tau}{}^{-1}\mathbf{B}_{d}{}^{T}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}\Big{)} \mathbf{C}_{h}\mathbf{A}_{d}, \tag{29}\]  
\[\mathbf{A}_{z}=\mathbf{A}-\mathbf{B}\mathbf{R}_{\tau}{}^{-1}\mathbf{B}_{d}{}^ {T}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}\mathbf{C}_{h}\mathbf{A}_{d}. \tag{30}\]  
**Proof. ** By differentiating (26),  
\[\dot{\mathbf{\lambda}}=\dot{\mathbf{P}}\mathbf{x}+\mathbf{P}\dot{\mathbf{x}}- \dot{\mathbf{g}}. \tag{31}\]  
Substituting (1), (25), and (26) into (31) yields  
\[ \begin{array}{ll}\dot{\mathbf{\lambda}}&=(\dot{\mathbf{P}}+\mathbf{P}\mathbf{A} )\mathbf{x}+\mathbf{P}\mathbf{B}\mathbf{u}-\dot{\mathbf{g}}\\ &=(\dot{\mathbf{P}}+\mathbf{P}\mathbf{A}_{z})\mathbf{x}-\mathbf{P}\mathbf{B} \mathbf{R}_{\tau}{}^{-1}\mathbf{B}^{T}\mathbf{\lambda}-\mathbf{P}\mathbf{B} \mathbf{R}_{\tau}{}^{-1}\mathbf{B}_{d}{}^{T}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{ \tau}\mathbf{w}-\dot{\mathbf{g}}\\ &=\Big{(}\dot{\mathbf{P}}+\mathbf{P}\mathbf{A}_{z}-\mathbf{P}\mathbf{B} \mathbf{R}_{\tau}{}^{-1}\mathbf{B}^{T}\mathbf{P}\Big{)}\mathbf{x}-\dot{ \mathbf{g}}+\mathbf{P}\mathbf{B}\mathbf{R}_{\tau}{}^{-1}\mathbf{B}^{T} \mathbf{g}-\mathbf{P}\mathbf{B}\mathbf{R}_{\tau}{}^{-1}\mathbf{B}_{d}{}^{T} \mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}\mathbf{w},\end{array}  \tag{32}\]  
Since the left sides of (21) and (32) are equal, the right sides of (21) and (32) are also equal. Therefore,  
\[ \begin{array}{ll}0&=\Big{(}\dot{\mathbf{P}}+\mathbf{P}\mathbf{A}_{z}- \mathbf{P}\mathbf{B}\mathbf{R}_{\tau}{}^{-1}\mathbf{B}^{T}\mathbf{P}+\mathbf{ Q}_{1}+\mathbf{A}_{d}{}^{T}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}\mathbf{C}_{h} \mathbf{A}_{d}\Big{)}\mathbf{x}\\ &-\dot{\mathbf{g}}+\mathbf{P}\mathbf{B}\mathbf{R}_{\tau}{}^{-1}\mathbf{B}_{d}{ }^{T}\mathbf{g}+\Big{(}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{x}+\mathbf{A}_{d}{}^{T} \mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}-\mathbf{P}\mathbf{B}\mathbf{R}_{\tau}{}^ {-1}\mathbf{B}_{d}{}^{T}\mathbf{C}_{h}{}^{T}\mathbf{Q}_{\tau}\Big{)}\mathbf{w }-\mathbf{C}_{y}{}^{T}\mathbf{Q}\mathbf{r}\\ &+\mathbf{A}^{T}\mathbf{\lambda}+\mathbf{A}_{d}{}^{T}\mathbf{C}_{h}{}^{T}\mathbf{Q }_{\tau}\mathbf{C}_{h}\mathbf{B}_{d}\mathbf{u}.\end{array}  \tag{33}\]  
Substituting (25) and (26) into (33) yields  
\[