## 2 Theoretical Background  
### Optimal Control Problem  
We consider optimal control problems with a quadratic objective function and a nonlinear multivariate discrete-time dynamic system under additive and parameter uncertainties. The basis for an optimal control problem is a deterministic dynamic system to be controlled (plant, firm, economy, etc.) in discrete time in the form:  
\[x_{t}=f(x_{t-1},x_{t},u_{t},z_{t}),\;\;t=1,...,T, \tag{1}\]  
where:  
* \(x_{t}\in R^{n}\) is a vector of state variables that describes the state of the system at any point in time \(t\),
* \(u_{t}\in R^{m}\) is a vector of control variables; we assume that the decision maker determines the values of the control variables exactly (without randomization) according to the approximately optimal solution of the problem,
* \(z_{t}\in R^{l}\) denotes a vector of non-controlled deterministic exogenous variables, whose values are known to the decision maker at time \(t\),
* \(T\) denotes the terminal time period of the finite planning horizon.  
The function \(f\) is assumed to be twice continuously differentiable with respect to all of its arguments.  
We assume that there is some true law of motion given by Equation (1) in the background which is at least partially unknown to the policy maker while the function \(f\) is known to him/her. The policy maker faces two sources of uncertainty, additive and parameter uncertainties where:* \(\theta\in R^{p}\) denotes a vector of stochastic variables of system parameters (parameter uncertainty),
* \(\overline{\theta}\in R^{p}\) is a vector of true parameters whose values are assumed to be constant but unknown to the policy maker,
* \(\epsilon_{t}\in R^{n}\) is a vector of additive stochastic disturbances (system error).
* \(\theta_{t}\) and \(\epsilon_{t}\) are assumed to be independent Gaussian random vectors with expectations \(\hat{\theta}\) and \(O_{n}\) respectively and covariance matrices \(\Sigma^{\theta\theta}\) and \(\Sigma^{\epsilon x}\) respectively.  
Including uncertainty, the system (1) can be written as:  
\[x_{t}=f(x_{t-1},x_{t},u_{t},\theta_{t},z_{t})+\epsilon_{t},\;\;t=1,...,T. \tag{2}\]  
The optimal control problem to be solved approximately assumes that there is a modeler who wishes to control the system (2). It means that the controller wishes to bring the state variables, using the control variables, as close as possible to some pre-defined desired time path, according to a given optimality criterion. The OPTCON algorithm allows for the optimal control of the system (2) using a quadratic objective function. To this end the modeler needs to define the following variables:  
* \(\tilde{x}_{t}\in R^{n}\) are given target ('ideal') values (for \(t=1,...,T\)) of the state variables,
* \(\tilde{u}_{t}\in R^{m}\) are given target ('ideal') values (for \(t=1,...,T\)) of the control variables,
* \(W_{t}\) is an \(((n+m)\times(n+m))\) symmetric matrix defining the relative weights of the state and control variables in the objective function. In a frequent special case, \(W_{t}\) includes a discount factor \(\alpha\) with \(W_{t}=\alpha^{t-1}W\).  
The resulting intertemporal objective function is given in quadratic tracking form:  
\[J=E\left[\sum_{t=1}^{T}L_{t}(x_{t},u_{t})\right], \tag{3}\]  
with  
\[L_{t}(x_{t},u_{t})=\frac{1}{2}\left(\begin{array}{c}x_{t}-\tilde{x}_{t}\\ u_{t}-\tilde{u}_{t}\end{array}\right)^{\prime}W_{t}\left(\begin{array}{c}x_{ t}-\tilde{x}_{t}\\ u_{t}-\tilde{u}_{t}\end{array}\right). \tag{4}\]  
The OPTCON algorithm allows us to find approximate optimal control solutions which minimize the functions (3) and (4) and satisfy the system dynamics (2).