# A Non-autonomous Stochastic Discrete Time System with Uniform Disturbances

Ioannis K. Dassios, Krzysztof J. Szajowski

► To cite this version:

Ioannis K. Dassios, Krzysztof J. Szajowski. A Non-autonomous Stochastic Discrete Time System with Uniform Disturbances. 27th IFIP Conference on System Modeling and Optimization (CSMO), Jun 2015, Sophia Antipolis, France. pp.220-229, 10.1007/978-3-319-55795-3_20. http-01626897

HAL Id: hal-01626897

https://inria.hal.science/hal-01626897

Submitted on 31 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A non-autonomous stochastic discrete time system with uniform disturbances

Ioannis K. Dassios¹² and Krzysztof J. Szajowski³

¹ MACSI, Department of Mathematics & Statistics, University of Limerick, Ireland,

² ERC, Electricity Research Centre, University College Dublin, Ireland,
jdasios@math.uoa.gr*^*^,

³ Wrocław University of Technology, Faculty of Pure and Applied Mathematics
Wybrzeże Wyspiańskiego 27, 50-370 Wrocław, Poland
Krzysztof.Szajowski@pwr.edu.pl†

**Abstract.** The main objective of this article is to present Bayesian optimal control over a class of non-autonomous linear stochastic discrete time systems with disturbances belonging to a family of the one parameter uniform distributions. It is proved that the Bayes control for the Pareto priors is the solution of a linear system of algebraic equations. For the case that this linear system is singular, we apply optimization techniques to gain the Bayesian optimal control. These results are extended to generalized linear stochastic systems of difference equations and provide the Bayesian optimal control for the case where the coefficients of these type of systems are non-square matrices. The paper extends the results of the authors developed for system with disturbances belonging to the exponential family.

**Keywords:** Bayes control, optimal, singular system, disturbances, Pareto distribution

## 1 Introduction

Linear stochastic discrete time systems (or linear matrix stochastic difference equations), are systems in which the variables take their value at instantaneous time points. The horizon of control depends on the problem. The state at instance $n$ depends on random disturbance and the chosen controls. Discrete time systems differ from continuous time ones in that their signals are in the form of sampled data. With the development of the digital computer, the stochastic discrete time system theory plays an important role in the control theory. In real systems, the discrete time system often appears when it is the result of sampling the continuous-time system or when only discrete data are available for use. The investigation aims are, when such system is under consideration, determining the control goals, performance measures and the information available at moments of controls' specification. The small deviations of the parameters can be

<footnote>*^*^ WWW home page: https://ioannisdassios.wordpress.com/research-visits-invited-talks/</footnote>

<footnote>† WWW home page: http://neyman.im.pwr.edu.pl/~szajow/</footnote>
treated as disturbances. As the random disturbance is admitted the performance
measure will be the mean value of the deviation of the states from the required
behavior of the system. When all the parameters of the system are known and
the distribution of disturbances is well defined then the optimal control can be
determined at least for the finite horizon case. The extension of the model to the
adaptive one means that the disturbances are not precisely described. Adaptive
control is the control method used by a controller which must adapt to a con-
trolled system with parameters which vary, or are initially uncertain (c.f. Black
et al. [2] or Tesfatsion [18] for the history of the adaptive control). Under some
unification the model of adaptive control of the linear system is formulated as a
control of the discrete time Markov process (cf. [5]).

It is assumed that the disturbance has a fixed probabilistic description which
is determined by the assumption. In this paper it is assumed that the distribu-
tion function is known to be an accuracy of parameters and the disturbances
additionally change the state of the system. It resembles the statistical problem
of estimation. It was seminal paper by Wald [21] where the background of the
modern decision theory was established (cf. [22, Chapt. 7]). The decision the-
ory approach to the control problems were immediately applied (see books by
Sworder [16], Aoki [1], Sage and Melsa [14]). The new class of control systems
under uncertainty was called adaptive (cf. [18], [2]). In these adaptive control
problems the important role have Bayesian systems. In this class of control mod-
els it is assumed that the preliminary knowledge of the disturbances is given by
a priori distributions of their parameters. The aim is to construct the controls
in a close form. The construction of the Bayes control is also auxiliary for the
construction of minimax controls (see Szajowski and Trybuła [17], Porosiński
and Szajowski [11], Grzybowski [8], González-Trejo et al. [7]). It is observed
the interested in various models of disturbance structure (cf. Duncan & Pasik-
Duncan [6]) and the disturbance distributions (cf. Walczak [19,20]). Stochastic
discrete time systems have many applications which we have described in [3]
where the Bayes control of the linear system with quadratic cost function and
the disturbances having the distribution belonging to the exponential family
with conjugate priors is solved.

The paper is organized as follows: the description of the stochastic discrete
time systems is subject of the section 1.1 and some remarks on disturbances
are given in the section 1.2. In the section 2 we determine the Bayes control
for the conjugate prior distribution $\pi$ of the parameter $\bar{\lambda}$ as the solution of a
singular linear system and provide optimal Bayesian control. We close the paper
by studying the Bayes control of a class of generalized linear stochastic discrete
time systems.

## 1.1 Stochastic discrete time systems

Let $\bar{x}_n \in \mathbb{R}^m$ be the state of the system, $\bar{u}_n \in \mathbb{R}^m$ be the control. Assume
that $\bar{v}_n \in V \subset \mathbb{R}^m$, with $\bar{v}_n = (v_1^n, v_2^n, \dots, v_n^n, 0, \dots, 0)^T$, is the disturbance at
time $n$ and $\alpha_n, b_n, c_n \in \mathbb{R}^{m \times m}$. Consider a stochastic discrete time system (cf.
Kushner [9])

$$ \bar{x}_{n+1} = \alpha_n \bar{x}_n + b_n \bar{u}_n + c_n \bar{v}_n, \quad \forall n = 0, 1, \dots, N-1. \quad (1) $$

The horizon N of the control, the time up to which the system is controlled, is a random variable, independent of the disturbances $\bar{v}_0, \bar{v}_1, \dots$, and has the following known distribution

$$ P\{N = k\} = p_k, \quad \forall k = 0, 1, \dots, M, \quad \sum_{i=0}^{M} p_k = 1, \quad p_M \neq 0. \quad (2) $$

In the authors paper [3] it was considered the family of the exponentially distributed disturbances. Let us assume here that the disturbances $v_n^i$ have the uniform distributions on $[0, \lambda_i]$ with parameter $\lambda_i \in \mathbb{R}^+$, $i = 1, 2, \dots, k$ and

$$ X_n = (\bar{x}_0, \bar{x}_1, \dots, \bar{x}_n), \quad U_n = (\bar{u}_0, \bar{u}_1, \dots, \bar{u}_n), \quad \bar{\lambda} = (\lambda_1, \lambda_2, \dots, \lambda_k, 0, \dots, 0)^T. $$

For convenience $U_M$ will be denoted by U and called a control policy.

**Definition 1.** The control cost for a given policy U (the loss function) is the following

$$ L(U, X_N) = \sum_{i=0}^{N} (\bar{y}_i^T s_i \bar{y}_i + \bar{u}_i^T k_i \bar{u}_i), \quad (3) $$

where $k_i \in \mathbb{R}^{m \times m} \ge 0_{m,m}$, are symmetric matrices, $s_i \in \mathbb{R}^{2m \times 2m} \ge 0_{2m,2m}$ and $\bar{y}_i = \begin{pmatrix} \bar{x}_i \\ \vdots \\ \bar{\lambda} \end{pmatrix} \in \mathbb{R}^{2m}, \forall i = 0, 1, \dots, M$. With $0_{i,j}$ we will denote the zero matrix $i \times j$.

Let the prior distribution $\pi$ of the parameter $\bar{\lambda}$ be given. It is considered the Pareto priors (see [4, Ch. 9.7], [10]) with parameters $r_i > 0, \beta_i > 2$

$$ g(\lambda_i | \beta_i, r_i) = \frac{\beta_i r_i^{\beta_i}}{\lambda_i^{\beta_i+1} \mathbb{I}_{[r_i, \infty)}(\lambda_i)}. \quad (4) $$

Denote $E_N, E_{\bar{\lambda}}$ the expectations with respect to the distributions of N and random vectors $\bar{v}_0, \bar{v}_1, \dots$ (when $\bar{\lambda}$ is the parameter), $E_\pi$ and $E$ are the expectations with respect to the distribution $\pi$ and to the joint distribution $\bar{v}_n$ and $\bar{\lambda}$, respectively.

**Definition 2.** (see [9], [12], [15], [19,20]) Let $L(\cdot, \cdot)$ be the loss function given by (3).

(a) The risk connected with the control policy U, when the parameter $\bar{\lambda}$ is given, is defined as follows

$$ R(\bar{\lambda}, U) = E_N \left[ E_{\bar{\lambda}}[L(U, X_N) | X_0] \right] = E_N \left[ E_{\bar{\lambda}} \left[ \sum_{i=0}^{N} \bar{y}_i^T s_i \bar{y}_i + \bar{u}_i^T k_i \bar{u}_i \mid X_0 \right] \right]. $$
(b) The expected risk $r$, associated with $\pi$ and the control policy $U$, is equal to

$$r(\pi, U) = E_{\pi}[R(\bar{\lambda}, U)] = E_{N}\left[E\left[\sum_{i=0}^{N} \bar{y}_{i}^{T} s_{i} \bar{y}_{i} + \bar{u}_{i}^{T} k_{i} \bar{u}_{i} \mid X_{0}\right]\right].$$

(c) The expected risk $r$, associated with $\pi$ and the control policy $U$, is equal to

$$r_n(\pi, U^n) = E_N[E_\pi[R(\bar{\lambda}, U)] = E_N\left[E\left[\sum_{i=n}^{N} \bar{y}_i^T s_i \bar{y}_i + \bar{u}_i^T k_i \bar{u}_i \mid X_0\right]\right].$$

Let the initial state $\bar{x}_0$ and the distribution $\pi$ of the parameter $\bar{\lambda}$ be given.

**Definition 3.** A control policy $U^*$ is called the Bayes policy when $r(\pi, U^*) = \inf_{U \in \wp_\pi} r(\pi, U)$, where $\wp_\pi$ is the class of the control policies $U$ for which exists $r(\pi, U)$.

## 1.2 Filtering

Let us assume that the random variables $\bar{v}_n$ have the density $p(\bar{v}_n, \bar{\lambda})$ with respect to a $\sigma$-finite measure $\mu$ on $\mathbb{R}$. The consideration is focused on the special case when each coordinate has the uniform distribution, i.e. the density $p(\bar{v}_n, \bar{\lambda})$ has the following representation:

$$p(\bar{v}_n, \bar{\lambda}) = \prod_{i=1}^{k} p(v_n^i, \lambda_i), \qquad (5)$$

where $p(v_n^i, \lambda_i) = \frac{1}{\lambda_i} \mathbb{I}_{[0, \lambda_i]}(v_n^i)$, for all $i = 1, 2, ..., k$. $V_i^*$ is the set of the random variables $v_n^i$. We have:

$$E_{\lambda_i}[v_n^i] = \frac{\lambda_i}{2} = q_i \lambda_i \text{ and, } E_{\lambda_i}[(v_n^i)^2] = \frac{\lambda_i^2}{3} = q_{1,i} \lambda_i^2,$$

where $q_i, q_{1,i}$ are constants. Let $\bar{\lambda}$ have the a priori distribution $\pi$ with density

$$g(\bar{\lambda} \mid \bar{\beta}, \bar{r}) = \prod_{i=1}^{k} g_i(\lambda_i; \beta^i, r^i), \qquad (6)$$

where $g_i(\lambda_i | \beta^i, r^i)$ is given by (4) where $\bar{\beta} \in S_k^\beta \subset \mathbb{R}^m, \bar{r} \in S_k^r \subset \mathbb{R}^m$ with

$$\bar{\beta} = (\beta^1, \beta^2, \dots, \beta^k, 0, \dots, 0)^T,$$

and $\bar{r} = (r^1, r^2, \dots, r^k, 0, \dots, 0)^T$. When such the a priori density is assigned to $\lambda_i$ and then the object of filtering, to determine the Bayes control, is to produce a posteriori density for $\lambda_i$ after any new observations of the state of the system. We change the control after obtaining the new data. Hence, to determine the Bayes control, a posteriori density for $\bar{\lambda}$ must be obtained after any new observations.
This is possible if for $n = 0, 1, ..., N-1$ and a given $\bar{x}_0$, we can derive $\bar{v}_n$ from (1), i.e. the equations

$$ \bar{v}_n = c_n^{-1}[\bar{x}_{n+1} - \alpha_n \bar{x}_n - b_n \bar{u}_n]. $$

If for a value of $n$, the matrix $c_n$ is singular, we will have to compute the Moore-Penrose Pseudoinverse $c_n^\dagger$ and then use the following expression

$$ \bar{v}_n = c_n^\dagger[\bar{x}_{n+1} - \alpha_n \bar{x}_n - b_n \bar{u}_n]. $$

The Moore-Penrose pseudo-inverse can be calculated via the singular value decomposition of $c_n$ (see [13]). In these cases a posteriori density $f(\bar{\lambda} | X_n, U_{n-1})$ of the parameter $\bar{\lambda}$, after having observed $X_n$ and chosen $U_{n-1}$, has the same form as (6) i.e.

$$ f(\bar{\lambda} | X_n, U_{n-1}) = f(\bar{\lambda} | V_{n-1}) = g(\bar{\lambda} | \bar{\beta}_n, \bar{r}_n), $$

where $V_{n-1} = (\bar{v}_0, \bar{v}_1, ..., \bar{v}_{n-1})$, $\bar{\beta}_n = \bar{\beta}_{n-1} + 2\bar{q}$, $\bar{q} = (q_1, q_2, ..., q_k, 0, ..., 0)^T \in Q_i^* \subset \mathbb{R}^m$ and $\bar{r}_n = \bar{r}_{n-1} \lor \bar{v}_n$ ($\bar{r}_0 = \bar{r}$). Under these denotations we have $E(\lambda_i | X_n, U_{n-1}) = T^{n,i} r_n^i = \frac{\beta_n}{\beta_{n-1}^i} r_n^i$ and $E(\lambda_i^2 | X_n, U_{n-1}) = T_1^{n,i}(r_n^i)^2$. For known $X_n$ and $U_{n-1}$, the conditional distribution of $\bar{v}_n$ has the density

$$ h(\bar{v}_n | X_n, U_{n-1}) = \prod_{i=1}^{k} h_i(v_n^i | X_n, U_{n-1}), $$

where

$$ \begin{aligned} h_i(v_n^i | X_n, U_{n-1}) &= \int_0^\infty p(v_n^i, \lambda)g(\lambda|\beta_n^i, r_n^i)d\lambda \\ &= \frac{\beta_n^i(r_n^i)^{\beta_n^i}}{\beta_{n+1}^i(r_{n+1}^i)^{\beta_{n+1}}} \mathbb{I}_{[0,\infty)}(v), \end{aligned} $$

for $n = 0, 1, ..., M-1$, $i = 1, 2, ..., k$. In addition (see [19,20]) by direct calculation we get

**Lemma 1.** The following equations are fulfilled:

$$ E(v_n^i | X_n, U_{n-1}) = \frac{1}{2} \frac{\beta_n^i}{\beta_{n+1}^i} r_n^i = Q^{n,i} r_n \quad (7) $$

$$ E((v_n^i)^2 | X_n, U_{n-1}) = Q_1^{n,i} (r_n^i)^2 \text{ where } Q_1^{n,i} = \frac{\beta_n^i}{3(\beta_n^i - 2)}. \quad (8) $$

$$ E(r_{n+1}^i | X_n, U_{n-1}) = Q_2^{n,i} r_n^i \text{ where } Q_2^{n,i} = \frac{(\beta_n^i)^2}{(\beta_n^i)^2 - 1}, \quad (9) $$

$$ E((r_{n+1}^i)^2 | X_n, U_{n-1}) = Q_3^{n,i} (r_n^i)^2 \text{ where } Q_3^{n,i} = \frac{\beta_n^i (\beta_n^i - 1)}{(\beta_n^i + 1)(\beta_n^i + 2)}. \quad (10) $$

$$ E(x_{n+1}^i | X_n, U_{n-1}) = \alpha_n x_n + u_n + \gamma_n Q^{n,i} r_n^i, \quad (11) $$

$$ E((x_{n+1}^i)^2 | X_n, U_{n-1}) = (\alpha_n x_n + u_n)^2 + 2(\alpha_n x_n + u_n)\gamma_n^i Q^{n,i} r_n^i + \gamma_n^2 Q_1^{n,i} (r_n^i)^2, \quad (12) $$

$$ E(x_{n+1} r_{n+1}^i | X_n, U_{n-1}) = (\alpha_n x_n + u_n) Q_2^{n,i} r_n^i + \gamma_n^i Q_4^{n,i} r_n^i, \quad (13) $$

$$ \text{where } Q_4^{n,i} = \frac{(\beta_n^i)^2}{(\beta_{n+1}^i)(\beta_n^i - 2)}. $$
## 2 The Bayes Control

Suppose the initial state $\bar{x}_0$ is given, the disturbances have the distribution with the density given by (5) and the prior distribution $\pi$ of the parameter $\bar{\lambda}$ is given by (6). Let the distribution of the random horizon $N$ be given by (2). Consider the problem of the Bayes control for the system (1) with the starting point at the moment $n$, when $X_n, U_{n-1}$ are given. The expected risk is then given by (c.f Definition 2 (c); see [12], [3])

$$r_n(\pi, U^n) = E \left[ \sum_{i=n}^{M} (\bar{y}_i^T s_i \bar{y}_i + \bar{u}_i^T k_i \bar{u}_i) \mid X_n, U_{n-1} \right]. \quad (14)$$

Let us denote $\varphi_k = \sum_{i=k}^{M} p_i$. We have

$$r_n = E\left[\sum_{i=n}^{M} \frac{\varphi_i}{\varphi_n} (\bar{y}_i^T s_i \bar{y}_i + \bar{u}_i^T k_i \bar{u}_i) \mid X_n, U_{n-1}\right].$$

For the above truncated problem we provide the following definitions:

**Definition 4.** The Bayes risk is defined as

$$W_n = \inf_{U^n} r_n(\pi, U^n), \quad (15)$$

where $r(\pi, U^n)$ is the expected risk defined in the definition 2 (c) and the formulae (14).

**Definition 5.** If there exists $U^{n*} = (\bar{u}_n^*, \bar{u}_{n+1}^*, \dots, \bar{u}_N^*)$ such that $W_n = r(\pi, U^{n*})$, then $U^{n*}$ will be called the Bayes policy and $\bar{u}_i^*$, $i = n, n+1, \dots, N$ the Bayes controls for truncated control problem.

Obviously, $r(\pi, U^0) = r(\pi, U)$, $W_0 = r(\pi, U^*)$. For the solution of the Bayes control problem we derive the Bayes controls $\bar{u}_n^*$ for $n = N, N-1, \dots, 1, 0$ recursively. Then $U^{0*}$ is the solution of the problem. From the Bellman's dynamic programming optimality principle we obtain the following Lemma, see [12].

**Lemma 2.** Assume the stochastic discrete time system (1). Then the Bayes risk $W_n$ has the form

$$W_n = \bar{x}_n^T A_n \bar{x}_n + 2 \bar{r}_n^T B_n \bar{x}_n + 2 \bar{r}_n^T C_n \bar{r}_n, \quad (16)$$

where $A_n, B_n, C_n \in \mathbb{R}^{m \times m}$, $\bar{D}_n \in \mathbb{R}^m$ with $A_n = f_1(s_n), B_n = f_2(Q^n, Q_2^n, s_n), C_n = f_3(Q^n, Q_1^n, Q_3^n, Q_4^n, s_n)$. The functions $f_j, j = 1, 2, 3$ are strictly monotonic, differentiable. The constants $Q^{n,i}, Q_j^{n,i}, j = 1, 2, 3, 4, n = 0, 1, \dots, N$ are given by (7), (8) and $s_n$ is defined in (3).
## 2.1 Bayesian optimal control for stochastic discrete time systems

We can now prove the following theorem

**Theorem 1.** Assume the stochastic discrete time system (1). Then, the Bayes control $\bar{u}_n^*$ is given by the solution of the linear system

$$K_n \bar{u}_n^* = L_n, \quad (17)$$

where

$$K_n = k_n + b_n^T A_{n+1} b_n \quad (18)$$

and

$$L_n = -b_n^T [A_{n+1} \alpha_n \bar{x}_n + (A_{n+1} c_n Q^n + B_{n+1} Q_2^n) \bar{r}_n]. \quad (19)$$

The matrices $k_n$, $A_n$, $Q^n$, are defined in (3), (16), the lemma 1, respectively and $\bar{e} = \sum_{j=0}^{n-1} Q^j \bar{r}_j$.

**PROOF.** From (15), the Bayes risk is given by $W_n = \inf_{U^n} r(\pi, U^n)$. It is, equivalently,

$$W_n = \min_{U^n} E \left[ \sum_{i=n}^{M} (\bar{y}_i^T s_i \bar{y}_i + \bar{u}_i^T k_i \bar{u}_i) \mid X_n, U_{n-1} \right].$$

We have

$$W_n = \min_{\bar{u}_n} \left\{ \bar{u}_n^T k_n \bar{u}_n + E \left[ \bar{y}_n^T s_n \bar{y}_n \mid X_n, U_{n-1} \right] \right. \\ \left. + \min_{U^{n+1}} E \left[ E \left[ \sum_{i=n+1}^{k} (\bar{y}_i^T s_i \bar{y}_i + \bar{u}_i^T k_i \bar{u}_i) \mid X_n, U_{n-1} \right] \right] \right\}.$$

It means $W_n = \min_{\bar{u}_n} \{\bar{u}_n^T k_n \bar{u}_n + E[\bar{y}_n^T s_n \bar{y}_n \mid X_n, U_{n-1}] + E[W_{n+1} \mid X_n, U_{n-1}]\}$. Hence, the Bayes control $\bar{u}_n^*$ satisfies the equation ($\nabla$ is the gradient):

$$\nabla_{\bar{u}_n} \{ \bar{u}_n^T k_n \bar{u}_n + E[\bar{y}_n^T s_n \bar{y}_n \mid X_n, U_{n-1}] + E[W_{n+1} \mid X_n, U_{n-1}] \}_{\bar{u}_n = \bar{u}_n^*} = 0_{m,1}.$$

By using (16) we get

$$k_n \bar{u}_n^* + b_n^T A_{n+1} (\alpha_n \bar{x}_n + b_n \bar{u}_n + c_n E(\bar{u}_n \mid X_n, U_{n-1})) + E \left[ \{b_n^T B_{n+1} \bar{r}_{n+1}\}_{\bar{u}_n = \bar{u}_n^*} \mid X_n, U_{n-1} \right] = 0_{m,1}.$$

By the properties of conjugate priors for the uniform distribution (see the lemma 1 we have

$$k_n \bar{u}_n^* + b_n^T A_{n+1} (\alpha_n \bar{x}_n + b_n \bar{u}_n + c_n Q^n \bar{r}_n) + E \left[ \{b_n^T B_{n+1} \bar{r}_{n+1}\}_{\bar{u}_n = \bar{u}_n^*} \mid X_n, U_{n-1} \right] = 0_{m,1},$$

and at the end $(k_n + b_n^T A_{n+1} b_n) \bar{u}_n^* = -b_n^T [A_{n+1} \alpha_n \bar{x}_n + (A_{n+1} c_n Q^n + B_{n+1} Q_2^n) \bar{r}_n]$. The proof is completed.

<figure><img src="image_2.png" /></figure>
Similarly like for the system with the disturbances belonging to the exponential family (see [3]) we get

**Theorem 2.** Consider the system (1) and the matrices $K_n, L_n$ as defined in (18), (19) respectively. Then

(a) $\forall n$ such that $K_n$ is full rank, the Bayes control $\bar{u}_n^*$, is given by

$$ \bar{u}_n^* = K_n^{-1} L_n. \qquad (20) $$

(b) $\forall n$ such that $K_n$ is rank deficient, the Bayesian optimal control $\hat{u}_n^*$ is given by

$$ \hat{u}_n^* = (K_n^T K_n + E^T E)^{-1} K_n L_n. \qquad (21) $$

Where $E$ is a matrix such that $K_n^T K_n + E^T E$ is invertible and $\|E\|_2 = \theta$, $0 < \theta \ll 1$. Where $\|\cdot\|_2$ is the Euclidean norm.

## 2.2 Bayesian optimal control for generalized stochastic discrete time systems

In this subsection we will expand the results of the section 2.1 by studying Bayesian optimal control for a class of linear stochastic discrete time systems with non-square coefficients. We consider the following non-autonomous linear stochastic discrete time system

$$ I_{r,m} \bar{x}_{n+1} = \alpha_n \bar{x}_n + b_n \bar{u}_n + c_n \bar{v}_n, \quad \forall n = 0, 1, \dots, N-1. \qquad (22) $$

Where $\bar{x}_n \in \mathbb{R}^m$ is the state of the system, $\bar{u}_n \in \mathbb{R}^m$ is the control, $\bar{v}_n \in V \subset \mathbb{R}^m$, with $\bar{v}_n = (v_n^1, v_n^2, \dots, v_n^k, 0, \dots, 0)^T$, is the disturbance at time $n$ and $\alpha_n, b_n, c_n \in \mathbb{R}^{r \times m}$. The horizon $N$ of the control is fixed and independent of the disturbances

$\bar{v}_n, n \ge 0$. If $r = m$, then $I_{r,m} = I_m$. If $r > m$, then $I_{r,m} = \begin{bmatrix} I_m \\ 0_{r-m,m} \end{bmatrix}$ and if $r < m$, then $I_{r,m} = \begin{bmatrix} I_r & 0_{r,m-r} \end{bmatrix}$ with $I_m, I_r$ identity matrices.

**Definition 6.** We will refer to system (22) as a generalized stochastic linear discrete time system.

In the above definition we use the term "generalized" because the coefficients in the system (22) can be either square or non-square matrices.

**Theorem 3.** Consider the system (22) for $r \ne m$ and assume the matrices $K_n, L_n$ as defined in (18), (19) respectively. Then, $\forall n$ such that

(a) $m < r$, $\text{rank}(K_n) = m$ and $L_n \in \text{colspan} K_n$, the Bayes control $\bar{u}_n^*$, is given by

$$ \bar{u}_n^* = K_n^{-1} L_n. \qquad (23) $$

(b) $m < r$, $\text{rank}(K_n) = m$ and $L_n \notin \text{colspan}(K_n)$, a Bayesian optimal control is given by

$$ \hat{u}_n^* = (K_n^T K_n)^{-1} K_n L_n. \qquad (24) $$
(c) $L_n \notin \operatorname{colspan} K_n$ and $K_n$ is rank deficient, a Bayesian optimal control is given by

$$ \hat{u}_n^* = (K_n^T K_n + E^T E)^{-1} K_n L_n. \quad (25) $$

Where $E$ is a matrix such that $K_n^T K_n + E^T E$ is invertible and $\|E\|_2 = \theta$, $0 < \theta \ll 1$.

(d) $m > r$, $K_n$ is full rank, a Bayesian optimal control is given by

$$ \hat{u}_n^* = K_n^T (K_n K_n^T)^{-1} L_n. \quad (26) $$

(e) $L_n \in \operatorname{colspan} K_n$ and $K_n$ is rank deficient, a Bayesian optimal control is given by (25).

The proof is based on ideas similar to those used in prove [3, Th. 3] and is omitted here.

## 3 Conclusions

In this article we focused on developing the Bayesian optimal control for a class of non-autonomous linear stochastic discrete time systems of type (1). Firstly, we proved that the Bayes control of these type of systems is the solution of a linear system of algebraic equations which can also be singular. For this case we used optimization techniques to derive the optimal Bayes control for (1). In addition, we used these methods to obtain the Bayesian optimal control of the non-autonomous linear stochastic discrete time system of type (2), where the coefficients of this system are non-square matrices.

The further extension of this paper is to study to Bayes control problem of stochastic fractional discrete time systems. The fractional nabla operator is a very interesting tool when applied to systems of difference equations and has many applications especially in macroeconomics, since it succeeds to provide information from a specific year in the past until the current year. For all these there is some research in progress.

## Acknowledgments

I. Dassios is supported by Science Foundation Ireland (award 09/SRC/E1780).

## References

1. Aoki, M.: Optimization of stochastic systems. Topics in discrete-time systems. Mathematics in Science and Engineering, Vol. 32, Academic Press, New York-London (1967)

2. Black, W.S., Haghi, P., Ariyur, K.B.: Adaptive systems: History, techniques, problems, and perspectives. Systems 2, 606–660 (2014), http://dx.doi.org/10.3390/systems2040606

3. Dassios, I.K., Szajowski, K.J.: Bayesian optimal control for a non-autonomous stochastic discrete time system. Technical report, Faculty of Pure and Applied Mathematicas, Wybrzee Wyspiańskiego 27, 50-370 Wrocław, Poland (2015), to appear
4. DeGroot, M.: Optimal Statistical Decision. McGraw Hill Book Comp., New York (1970)

5. Duncan, T.E., Pasik-Duncan, B., Stettner, L.: Adaptive control of a partially observed discrete time Markov process. Appl. Math. Optim. 37(3), 269–293 (1998), http://dx.doi.org/10.1007/s002459900077

6. Duncan, T.E., Pasik-Duncan, B.: Discrete time linear quadratic control with arbitrary correlated noise. IEEE Trans. Automat. Control 58(5), 1290–1293 (2013), http://dx.doi.org/10.1109/TAC.2012.2220444

7. González-Trejo, J.I., Hernández-Lerma, O., Hoyos-Reyes, L.F.: Minimax control of discrete-time stochastic systems. SIAM J. Control Optim. 41(5), 1626–1659 (electronic) (2002), http://dx.doi.org/10.1137/S0363012901383837

8. Grzybowski, A.: Minimax control of a system with actuation errors. Zastos. Mat. 21(2), 235–252 (1991)

9. Kushner, H.: Introduction to stochastic control. Holt, Rinehart and Winston, Inc., New York-Montreal, Que.-London (1971)

10. Philbrick, S.: A practical guide to the single parameter pareto distribution. In: Proceedings of the Casualty Actuarial Society. vol. LXXII, pp. 44–84. Casualty Actuarial Society, 4350 N. Fairfax Drive, Suite 250 Arlington, Virginia 22203, U.S.A. (1985)

11. Porosiński, Z., Szajowski, K.: A minimax control of linear systems. In: J.Zabczyk (ed.) Stochastic Systems and Optimization, Proceedings of the 6th IFIP WG 7.1 Working Conference, Jablonna, 12-16.09.1988. Lecture Notes in Control and Inform. Sci., vol. 136, pp. 344–355. Springer, Berlin (1989), http://dx.doi.org/10.1007/BFb0002694, MR1180792; Zbl:0711.93095

12. Porosiński, Z., Szajowski, K., Trybula, S.: Bayes control for a multidimensional stochastic system. Systems Sci. 11(2), 51–64 (1987) (1985)

13. Rugh, W.J.: Linear system theory. Prentice Hall Information and System Sciences Series, Prentice Hall, Inc., Englewood Cliffs, NJ (1993)

14. Sage, A.P., Melsa, J.L.: Estimation theory with applications to communications and control. McGraw-Hill Book Co., New York-Düsseldorf-London (1971), mcGraw-Hill Series in Systems Science

15. Sawitzki, G.: Exact filtering in exponential families: discrete time. Math. Operationsforsch. Statist. Ser. Statist. 12(3), 393–401 (1981), http://dx.doi.org/10.1080/02331888108801598

16. Sworder, D.: Optimal adaptive control systems. Mathematics in Science and Engineering. Vol. 25, Academic Press, New York-London (1966)

17. Szajowski, K., Trybula, S.: Minimax control of a stochastic system with the loss function dependent on parameter of disturbances. Statistics 18(1), 151–165 (1987), http://dx.doi.org/10.1080/02331888708802005

18. Tesfatsion, L.: A dual approach to Bayesian inference and adaptive control. Theory and Decision 14(2), 177–194 (1982), http://dx.doi.org/10.1007/BF00133976

19. Walczak, D.: Bayes and minimax control of discrete time linear dynamical systems. Technical Report, TU Wrocław (1986), master Theses (in Polish)

20. Walczak, D.: Bayesian control of a discrete-time linear system with uniformly distributed disturbances. Mathematica Applicanda 43(2), 173–186 (2016), http://dx.doi.org/10.1016/j.amc.2013.10.090

21. Wald, A.: Contributions to the theory of statistical estimation and testing hypotheses. Ann. Math. Statistics 10, 299–326 (1939)

22. Wald, A.: Statistical Decision Functions. John Wiley & Sons, Inc., New York, N. Y.; Chapman & Hall, Ltd., London (1950)