## 3 The OPTCON Algorithm
### Computational Details for the AL Procedure
\begin{array}{l}\Delta J_{T-j}^{*}=g_{j+1}-\frac{1}{2}\lambda^{u}(\Lambda^{ uu})^{-1}\lambda^{u}+\frac{1}{2}tr(K_{j}\Sigma^{\xi\xi})+E\{(\lambda^{x}+ \Lambda^{xu}(\Lambda^{uu})^{-1}\lambda^{u})\delta x_{j-1}\\ \qquad+\frac{1}{2}\delta x_{j-1}^{{}^{\prime}}\Lambda^{xx}\delta x_{j-1}\}- \frac{1}{2}(E\{\delta x_{j-1}\Omega_{xx}\delta x_{j-1}\}-tr(\Omega_{xx} \Sigma^{xx}))\\ =g_{j+1}-\frac{1}{2}\lambda^{u}(\Lambda^{uu})^{-1}\lambda^{u}+\frac{1}{2}tr(K _{j}\Sigma^{\xi\xi})+tr(\Omega_{xx}\Sigma^{xx})\\ \qquad+E\{(\lambda^{x}+\Lambda^{xu}(\Lambda^{uu})^{-1}\lambda^{u})\delta x_{j -1}+\frac{1}{2}\delta x_{j-1}^{{}^{\prime}}(\Lambda^{xx}-\Omega_{xx})\delta x _{j-1}\}.\end{array} \]  
Defining  
\[h_{j}^{x}:=\lambda^{x}+\Lambda^{xu}(\Lambda^{uu})^{-1}\lambda^{u},\quad H_{j}: =\Lambda^{xx}-\Omega_{xx}:=\Lambda^{xx}-\Lambda^{xu}(\Lambda^{uu})^{-1} \Lambda^{ux} \tag{31}\]  
and  
\[g_{j}=g_{j+1}-\frac{1}{2}\lambda_{j}^{u}(\Lambda^{uu})^{-1}\lambda_{j}^{u}+ \frac{1}{2}tr(K_{j}\Sigma_{j/j}^{\xi\xi}+\Omega_{xx}\Sigma_{j}^{xx}),\]  
we get  
\[\Delta J_{T-j}^{*}=g_{j}+E\{h_{j}^{x}\delta x_{j-1}+\frac{1}{2}\delta x_{j-1}^ {{}^{\prime}}H_{j}\delta x_{j-1}\}. \tag{32}\]  
This shows that \(\Delta J_{T-j}^{*}\) is a quadratic function of \(\delta x_{j-1}\). Next, using the results of [31] we find the solution of the \(g\) recursion:  
\[g_{j}=g_{j+1}-\frac{1}{2}\lambda_{j}^{u}(\Lambda^{uu})^{-1}\lambda_{j}^{u}+ \frac{1}{2}tr(K_{j}\Sigma_{j/j}^{\xi\xi}+\Omega_{xx}\Sigma_{j}^{xx}).\]  
Let us define a deterministic term \(M_{j}=\frac{1}{2}\lambda_{j}^{u}(\Lambda^{uu})^{-1}\lambda_{j}^{u}\) and a stochastic term \(N_{j}=\frac{1}{2}tr(K_{j}\Sigma_{j/j}^{\xi\xi}+\Omega_{xx}\Sigma_{j}^{xx})\), so that \(g_{j}=g_{j+1}-M_{j}+N_{j}\). By working backwards from period \(T\) the following can be shown:  
\[g_{T-j}=g_{T-j+1}-\sum_{i=T-j}^{T}M_{i}+\sum_{s=T-j}^{T}N_{i}\quad\text{and} \quad g_{T+1}=0. \tag{33}\]  
Next, solve it partially for the stochastic term \(N_{j}\). For that define \(\gamma_{j}=\gamma_{j+1}-M_{j}\) with \(\gamma_{T+1}=0\). Using again the backward recursion get the equation:  
\[\gamma_{T-j}=\gamma_{T-j+1}-\sum_{i=T-j}^{T}M_{i}=-\sum_{i=T-j}^{T}M_{i}.\]Put it into (33) and get  
\[g_{T-j}=\gamma_{T-j}+\sum_{i=T-j}^{T}N_{i}\]  
or in general  
\[g_{j+1}=\gamma_{j+1}+\sum_{i=j+1}^{T}N_{i}.\]  
Thus  
\[g_{j+1}=\gamma_{j+1}+\frac{1}{2}\sum_{i=j+1}^{T}tr(K_{i}\Sigma_{i-1/i-1}^{ \xi\xi}+\Omega_{xx}\Sigma_{i-1}^{xx}) \tag{34}\]  
and  
\[\gamma_{j}=\gamma_{j+1}-\frac{1}{2}\lambda_{j-1}^{u}(\Lambda^{uu})^{-1}\lambda _{j-1}^{u}.\]  
Use the fact that \(\Delta J_{T-(t+1)}^{*}\) and \(\Delta J_{t+1}^{*}\) are equivalent; thus put (34) into (32) for \(t+1\):  
\[g_{j}=g_{j+1}-\frac{1}{2}\lambda_{j}^{u}(\Lambda^{uu})^{-1}\lambda_{j}^{u}+ \frac{1}{2}tr(K_{j}\Sigma_{j/j}^{\xi\xi}+\Omega_{xx}\Sigma_{j}^{xx}),\]0  
Then  
\[ \begin{array}{l}J_{t+1}^{*}(x_{t})=J_{o,t+1}^{*}+\Delta J_{t+1}^{*}=C_{o,t+ 1}+\gamma_{j+1}+\frac{1}{2}(\sum_{i=l+1}^{T}tr(K_{i}\Sigma_{i}^{\xi\xi}+ \Omega_{xx}\Sigma^{xx}))\\ \hskip 142.26378 pt+E\{h_{t+1}^{x}\delta x_{t}+\frac{1}{2}\delta x_{t}^{\prime }H_{t+1}\delta x_{t}/P^{t}\},\end{array}  \tag{36}\]  
where \(C_{o,t+1}=\sum_{i=l+1}^{T}L_{t+1}(x_{o,t+1},u_{o,t+1})\). \[ \begin{array}{l}J_{t}^{*}=\min_{u_{t}}E_{t-1}\{L_{t}(x_{t},u_{t})+J_{t+1}^{* }(x_{t})\}\\ \hskip 142.26378 pt=\min_{u_{t}}E\{L_{t}(x_{t},u_{t})+C_{o,t+1}+\gamma_{t+1}+ \frac{1}{2}(\sum_{i=l+1}^{T}tr(K_{i}\Sigma_{i}^{\xi\xi}+\Omega_{xx}\Sigma^{xx} ))\\ \hskip 142.26378 pt+E\{h_{t+1}^{x}\delta x_{t}+\frac{1}{2}\delta x_{t}^{\prime }H_{t+1}\delta x_{t}/P^{t}/P^{t-1}\}\}\\ \hskip 142.26378 ptE\{h_{t+1}^{x}\delta x_{t}/P^{t-1}\}=0\text{ and }\\ \hskip 142.26378 ptE\{\frac{1}{2}\delta x_{t}^{\prime}H_{t+1}\delta x_{t}/P^{t-1 }\}=\frac{1}{2}\delta\delta_{t/t-1}^{x}H_{t+1}\delta\delta_{t/t-1}+\frac{1}{2} tr(H_{t+1}\Sigma_{t/t-1}^{xx})=\frac{1}{2}tr(H_{t+1}\Sigma_{t/t-1}^{xx}).\end{array}