---
abstract: 'In this article we construct control policies that ensure bounded variance of a noisy marginally stable linear system in closed-loop.  It is assumed that the noise sequence is a mutually independent sequence of random vectors, enters the dynamics affinely, and has bounded fourth moment.  The magnitude of the control is required to be of the order of the first moment of the noise, and the policies we obtain are simple and computable.' 
address: 'Automatic Control Laboratory, ETL I28, ETH Zürich, Physikstrasse 3, 8092 Zürich, Switzerland'
author:
- Federico Ramponi
- Debasish Chatterjee
- 'Andreas Milias-Argeitis'
- Peter Hokayem
- John Lygeros
title: Attaining mean square boundedness of a marginally stable stochastic linear system with a bounded control input
---

Introduction
============

Stabilization of stochastic linear systems with bounded control inputs has attracted considerable attention over the years.  This is due to the fact that incorporating bounds on the control is of paramount importance in practical applications; suboptimal control strategies such as receding-horizon control [@ref:ChaHokLyg-09; @ref:HokChaLyg-09], and rollout algorithms [@ref:bertsekasDP1], among others, were designed to incorporate such constraints with relative ease, and have become widespread in applications.  However, the following question remains open: *when is a linear system with possibly unbounded additive stochastic noise globally stabilizable with bounded inputs?*  In this article we shall provide sufficient conditions that give a positive answer to this question with minimal hypotheses. 

Bounded input control has a rich and important history in the control literature [@ref:sussmannBoundedControls92; @ref:sussmannBoundedControls94; @ref:sussmannBoundedControlsFinalDT; @ref:stoorvogel96; @ref:stoorvogelACC07Incorrect].  The deterministic version of the bounded input stabilization problem was solved completely in a series of articles [@ref:sussmannBoundedControls92; @ref:sussmannBoundedControls94] culminating in [@ref:sussmannBoundedControlsFinalDT].  It was demonstrated in [@ref:sussmannBoundedControlsFinalDT] that global asymptotic stabilization of a discrete-time linear system $$\label{e:detsys}
    x_{t+1} = A x_t + B u_t\tag{$(\star)$}$$ with bounded feedback inputs is possible if and only if the transition matrix has spectral radius at most $1$, and the pair $(A, B)$ is stabilizable with arbitrary controls.  Moreover, extensions to the output feedback case have appeared in [@ref:BaoLinSon-00; @ref:ChiLin-03].  In the presence of affine stochastic noise the linear system becomes $x_{t+1} = A x_t + B u_t + w_t$, where $(w_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ is a collection of independent (but not necessarily identically distributed) random vectors in ${\ensuremath{\mathbb{R}}}^d$ with possibly inter-dependent components at each time $t$. With an arbitrary noise it is clearly not possible to ensure mean-square boundedness; for instance, if the noise has a spherically symmetric Cauchy distribution on ${\ensuremath{\mathbb{R}}}^d$, then given any initial condition $x_0\in{\ensuremath{\mathbb{R}}}^d$, the second moment of $x_1$ does not even exist.  Similarly, if the second moment of the noise becomes unbounded with time, it is not possible to control the second moment of the process $(x_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$. It is necessary to assume, at least, that the noise has bounded variance. 

Going beyond this necessary condition, it is not difficult to establish mean-square boundedness of such a system with bounded controls under the assumption that $A$ is Schur stable, i.e., all eigenvalues of $A$ are contained in the interior of the unit disk (the proof of this fact relies on standard Foster-Lyapunov techniques [@ref:meyn93]).  However, to the best of our knowledge, there is no proof that the same can be ensured for a marginally stable linear system.  Results in this direction were reported in [@ref:stoorvogelACC07Incorrect], but to the best of our understanding *conclusive* proofs of the facts reported in the present article are still missing in the literature. 

In this article, we develop easily computable bounded control policies for the case when $A$ is marginally stable and $(A,B)$ is stabilizable.  Our policy is not anyway stationary and is in general chosen from the class of finite $k$-history-dependent and/or non-stationary policies.  With respect to the case when $A$ is orthogonal, it turns out that if the system is reachable in one step (i.e., $\operatorname{rank}B = $ the dimension of the state space), we do get stationary feedback policies.  In the more general case when the system is reachable in $k$ steps (with arbitrary controls), we propose a feedback policy for a sub-sampled system derived from the original one, which, for the actual system, turns out to be a $k$-history-dependent policy.  In fact, in this case we realize our policy as successive concatenations of a fixed $k$-length policy.  In the most general situation we propose a $k$-history-dependent policy, where $k$ is now the reachability index of the particular subsystem of $(A,B)$ for which the dynamics matrix is orthogonal.  In all the mentioned cases, the length of the policy is at most equal to the dimension of the state space; memory requirements for even the most general case are, therefore, modest. 

Note that in our setting we do *not* assume that the noise is white.  For our purposes the requirements on the noise are rather general, namely, the fourth moment of the noise should be uniformly bounded, and the noise vectors should be independent of each other (identical distribution at each time is not assumed).  In particular, we do *not* assume Gaussian structure of the noise.  It turns out that to ensure stabilization we need the controller to be sufficiently strong, in the sense that the control input norm bound should be bigger than a uniform bound on the first moment of the noise. 

Section \[SECTION\_MAIN\] contains a precise statement of our result in the most general hypotheses ($A$ marginally stable and $(A,B)$ stabilizable), and a brief sketch of the proof.  In Section \[SECTION\_PROOF\], after some preliminary material, we prove the attainability of bounded second moment for a random walk, then we generalize the result under weaker and weaker hypotheses, finally culminating in the proof of the main theorem of Section \[SECTION\_MAIN\].  Section \[SECTION\_EXAMPLE\] presents a numerical example illustrating our results, and Section \[SECTION\_CONCLUSION\] concludes the article with a conjecture. 

Main result {#SECTION_MAIN}
===========

Statement of the theorem
------------------------

Consider the discrete-time linear system $$\label{STABLE_SYSTEM}
x_{t+1} = A x_t + B u_t + w_t,\qquad x_0 = x,\quad t\in{\ensuremath{\mathbb{N}_0}},$$ where the following hold: $x\in{\ensuremath{\mathbb{R}}}^d$ is given; the state $x_t$ at time $t$ takes values in ${\ensuremath{\mathbb{R}}}^d$; $A\in{\ensuremath{\mathbb{R}}}^{d\times d}$, all the eigenvalues of $A$ lie in the closed unit circle, and those eigenvalues $\lambda$ such that $|\lambda|=1$ have equal algebraic and geometric multiplicities; $B\in{\ensuremath{\mathbb{R}}}^{d\times m}$, and the control $u_t$ at time $t$ takes values in ${\ensuremath{\mathbb{R}}}^m$; $(w_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ is an ${\ensuremath{\mathbb{R}}}^d$-valued random process with mean zero and ${\ensuremath{\mathsf{E}}}\bigl[w_t w_t{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}\bigr] = Q_t$.

Our objective is to synthesize a $k$-history-dependent control policy[^1] $\pi = (\pi_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$, consisting of successive concatenations of $k$-length sequence $\tilde\pi_{0:k-1} {\coloneqq}\bigl[\tilde\pi_0, \cdots, \tilde\pi_{k-1}\bigr]$ of maps, $\tilde\pi_i:{\ensuremath{\mathbb{R}}}^{d}{\ensuremath{\longrightarrow}}{\ensuremath{\mathbb{R}}}^m$ for $i = 0, \ldots, k-1$, such that $\pi_t:{\ensuremath{\mathbb{R}}}^{d\times k}{\ensuremath{\longrightarrow}}{\ensuremath{\mathbb{R}}}^m$ is measurable, $u_t {\coloneqq}\pi_t\bigl(x_t, x_{t-1},\ldots, x_{t-k+1}\bigr)$, the sequence $(u_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ is bounded, and the state of the closed-loop system $$\label{e:syscl}
x_{t+1} = A x_t + B \pi_t\bigl(x_t, x_{t-1}, \ldots, x_{t-k+1}\bigr) + w_t, \qquad x_0 = x, \quad t\in{\ensuremath{\mathbb{N}_0}},$$ has bounded second-order moment. ( To simplify the notation, we fix $x_{-k+1} = \cdots = x_{-1} = x_0$.) The following is our main result:

\[MAIN\_THEOREM\] Consider the system .  Suppose that the pair $(A, B)$ is [*stabilizable*]{}, and that $\sup_{t\in{\ensuremath{\mathbb{N}_0}}}{\ensuremath{\mathsf{E}}}\bigl[{\ensuremath{\left\lVert w_t \right\rVert}}^4\bigr] < \infty$. Then there exist an $R > 0$ and a deterministic $k$-history-dependent policy $(\pi_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$, with $k\leq d$ and ${\ensuremath{\left\lVert \pi_t(\cdot) \right\rVert}} {\ensuremath{\leqslant}}R$ for every $t$, such that

1.   for every fixed $x\in{\ensuremath{\mathbb{R}}}^d$ the process $(x_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ that solves the recursion satisfies $\sup_{t\in{\ensuremath{\mathbb{N}_0}}}{\ensuremath{\mathsf{E}}}_x\bigl[{\ensuremath{\left\lVert x_t \right\rVert}}^2\bigr] < \infty$, and

2.   in the absence of the random noise the origin is asymptotically stable for the closed-loop system. 

Sketch of the proof
-------------------

Our proof is built in a series of steps, moving from simpler to progressively more complex systems.  The starting point is the $d$-dimensional random walk $x_{t+1} = x_t + u_t + w_t$. In this case we employ the main result of [@ref:pemantle99] to design a policy that guarantees mean-square boundedness of the closed-loop system.  We then consider the system $x_{t+1} = A x_t + Bu_t + w_t$, where $u_t$ is a $d$-dimensional control input, $\operatorname{rank}B = d$, and $A$ is orthogonal.  With the help of a time-varying injective linear transformation this case is reduced to the $d$-dimensional random walk.  The third case that we consider is that of the system $x_{t+1} = A x_t + B u_t + w_t$, where $u_t\in{\ensuremath{\mathbb{R}}}^m$ and $A$ is orthogonal.  This is reduced to the second case above with the aid of an injective linear transformation derived from the reachability matrix of the pair $(A, B)$ (recall that by assumption the reachability matrix has rank $d$).  Finally, the general case when $A$ is just stable and $(A,B)$ stabilizable is reduced to the third case with the observation that, in view of the stability hypothesis, $A$ acts as an orthogonal map on its invariant subspace that corresponds to the eigenvalues that lie on the unit circle. 

Arguments for establishing mean-square boundedness of stochastic dynamical systems typically rely on $L_1$-bounded-ness of a Lyapunov-like functional of the system.  The latter can be established in at least three different ways: The first is via the classical Foster-Lyapunov drift-conditions [@ref:foss04; @ref:meyn93] and its various refinements; the second is via excursion-theoretic analysis [@ref:palExcur] that relies primarily on the existence of certain supermartingales as long as the process is outside some bounded set; the third is via martingale inequalities [@ref:pemantle99], which applies to more general scalar-valued processes than Markov processes, and in the presence of bounded controls, provides the basic machinery for establishing our Theorem \[MAIN\_THEOREM\]. 

Proof of the main result {#SECTION_PROOF}
========================

Preliminaries {#s:prelims}
-------------

Let ${\ensuremath{\mathbb{N}_0}}$ be the set of nonnegative integers $\{0, 1, 2, \ldots\}$. The standard $2$-norm on Euclidean spaces is denoted by ${\ensuremath{\left\lVert \cdot \right\rVert}}$ and the absolute value on ${\ensuremath{\mathbb{R}}}$ by ${\ensuremath{\left\lvert{\cdot}\right\rvert}}$. In a Euclidean space we denote by ${{\mathcal B}}_r$ the closed Euclidean ball of radius $r$ centered at the origin.  If $(y_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ is a random process on a probability space $(\Omega, {\ensuremath{\mathfrak{F}}}, {\ensuremath{\mathsf{P}}})$, taking values in some Euclidean space, we let ${\ensuremath{\mathsf{E}}}_x[\varphi(y_s; s=0, 1,\ldots, t)]$ denote the conditional expectation of a measurable mapping $\varphi$ of the process up to time $t$, given the initial condition $y_0 = x$; in particular we define the $n$-th moment of $y_t$ as ${\ensuremath{\mathsf{E}}}_x[{\ensuremath{\left\lVert y_t \right\rVert}}^n]$. We denote conditional expectation given a sub-$\sigma$-algebra ${\ensuremath{\mathfrak{F}}}'$ of ${\ensuremath{\mathfrak{F}}}$ as ${\ensuremath{\mathsf{E}}}[\cdot\,|\,{\ensuremath{\mathfrak{F}}}']$. For $r > 0$ let $\operatorname{sat}_r:{\ensuremath{\mathbb{R}}}^d {\ensuremath{\longrightarrow}}{{\mathcal B}}_r$ be defined by $\operatorname{sat}_r(y) {\coloneqq}y$ if $y\in{{\mathcal B}}_r$ and $\operatorname{sat}_r(y) {\coloneqq}r y/{\ensuremath{\left\lVert y \right\rVert}}$ otherwise.  Note that $\operatorname{sat}_r(\cdot)$ is *not* the component-wise saturation function.  Given matrices $A\in {\ensuremath{\mathbb{R}}}^{d\times d}$ and $B\in{\ensuremath{\mathbb{R}}}^{d\times m}$ we define the $k$-step reachability matrix ${{\mathcal R}}_k {\coloneqq}\bmat{cccc} B & AB & \cdots & A^{k-1}B \emat$.

We specialize the general definition of a policy [@ref:hernandez-lerma1996 Chapter 2] to our setting.  A policy $\pi {\coloneqq}(\pi_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ is a sequence of measurable maps $\pi_t:{\ensuremath{\mathbb{R}}}^{d\times k}{\ensuremath{\longrightarrow}}{\ensuremath{\mathbb{R}}}^m$ for some $k\in{\ensuremath{\mathbb{N}}}$, such that the control at time $t$ is $\pi_t\bigl(x_t, x_{t-1}, \ldots, x_{t-k+1}\bigr)$. The policy $\pi = (\pi_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ we have defined is also known as a [*deterministic $k$-history-dependent policy*]{} in the literature.  A special case of these policies is a [*deterministic feedback policy*]{} or simply a [*feedback*]{} if $k = 1$ in the definition of a deterministic history-dependent policy.  Under deterministic feedback policies the closed-loop system is Markovian [@ref:hernandez-lerma1996 Proposition 2.3.5].  A further special case is when $\pi_t = f$, a fixed measurable mapping $f:{\ensuremath{\mathbb{R}}}^d{\ensuremath{\longrightarrow}}{\ensuremath{\mathbb{R}}}^m$ for $t\in{\ensuremath{\mathbb{N}_0}}$; this is known as a [*stationary feedback policy*]{}. 

\[LEMMA\_BASE\] Let $B_1, \cdots, B_k$ be $d\times m$ matrices, $M {\coloneqq}\bmat{ccc} B_1 & \cdots & B_k \emat$, and $\sigma_d$ denote the minimum singular value of $M$. If $\operatorname{rank}M = d$, then for all $r>0$ every vector $v \in {\ensuremath{\mathbb{R}}}^d$ belonging to ${{\mathcal B}}_r$ can be expressed as $v = \sum_{i=1}^k B_i u_i$, with $u_i \in {\ensuremath{\mathbb{R}}}^m$ and ${\left\lVert}u_i {\right\rVert}\leq r\sigma_d^{-1}$. In particular, if $B\in {\ensuremath{\mathbb{R}}}^{d\times d}$ and $\operatorname{rank}B = d$, then every vector $v \in {\ensuremath{\mathbb{R}}}^d$ belonging to ${{\mathcal B}}_r$ can be expressed as $v = B u$, where $u \in {\ensuremath{\mathbb{R}}}^d$, ${\left\lVert}u {\right\rVert}\leq r\sigma_d^{-1}$.

$\operatorname{rank}M = d$ implies that $km \geq d$. Hence, $M = \bmat{ccc} B_1 & \cdots & B_k \emat \in {\ensuremath{\mathbb{R}}}^{d\times km}$ is a “flat” matrix.  Let $
    M = USV{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}= U \bmat{cc} \Sigma & 0 \\ \emat V{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}$ be a singular value decomposition of $M$, where $\Sigma = {\mathop{\rm diag}}(\sigma_1, ..., \sigma_d)$. Since $M$ has full rank, the matrix $\Sigma$ is invertible.  Hence every vector $v \in {\ensuremath{\mathbb{R}}}^d$ can be expressed as $v = Mu$, where $u = M^+v$ and $M^+ = V \bmat{c} \Sigma^{-1} \\0\\ \emat U{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}\in {\ensuremath{\mathbb{R}}}^{km\times d}$ is the Moore-Penrose pseudoinverse of $M$. Since $U, V$ are orthogonal, for any $\rho>0$ we have $
\inf_{{\left\lVert}u {\right\rVert}= \rho}  {\left\lVert}M u {\right\rVert}= \inf_{{\left\lVert}V{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}u {\right\rVert}= \rho}  {\left\lVert}U \bmat{cc} \Sigma & 0\\ \emat V{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}u {\right\rVert}= \inf_{{\left\lVert}\upsilon {\right\rVert}= \rho}  {\left\lVert}\Sigma \upsilon {\right\rVert}= \rho \sigma_d. 
$ Hence, the image of ${{\mathcal B}}_\rho$ under $M$ contains ${{\mathcal B}}_{\rho \sigma_d}$, and if we choose $\rho = r \sigma_d^{-1}$, then the image of ${{\mathcal B}}_\rho$ under $M$ contains ${{\mathcal B}}_r$. Notice that $\sigma_d^{-1}$ is also the greatest singular value of $M^+$, and indeed we have $
\sup_{{\left\lVert}v {\right\rVert}= r}  {\left\lVert}M^+v {\right\rVert}= \sup_{{\left\lVert}U{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}v {\right\rVert}= r}  {\left\lVert}V \bmat{c} \Sigma^{-1} \\ 0\\ \emat U{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}v {\right\rVert}= \sup_{{\left\lVert}\nu {\right\rVert}= r}  \bigl\|\Sigma^{-1} \nu \bigr\|
= r \sigma_d^{-1}. 
$ Summing up, every $v \in {{\mathcal B}}_r$ can be expressed as $v = Mu$, where $u\in {\ensuremath{\mathbb{R}}}^{k m}$ and ${\left\lVert}u {\right\rVert}\leq r\sigma_d^{-1}$. It remains to notice that $u$ can be partitioned according to the partition of $M$, that is $
v = Mu =
\bmat{cccc} B_1 & B_2 & \cdots & B_k \emat
\begin{bmatrix}u_1{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}& \cdots & u_k{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}\end{bmatrix}{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}= \sum_{i=1}^k B_i u_i
$ and the bound ${\left\lVert}u {\right\rVert}\leq r\sigma_d^{-1}$ implies ${\left\lVert}u_i {\right\rVert}\leq r\sigma_d^{-1}$ for all $i = 1\cdots k$.

The $d$-dimensional random walk {#s:drw}
-------------------------------

At the core of our proof is the $d$-dimensional random walk: $$\label{e:sysdrw}
                x_{t+1} = x_t + u_t + w_t,\qquad x_0 = x, \quad t\in{\ensuremath{\mathbb{N}_0}},$$ with the state $x_t\in{\ensuremath{\mathbb{R}}}^d$, the control $u_t\in{\ensuremath{\mathbb{R}}}^d$ with ${\ensuremath{\left\lVert u_t \right\rVert}} {\ensuremath{\leqslant}}r$ for some $r > 0$, the noise process $(w_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ satisfies the following assumption:

\[a:w\]

-   $(w_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ are mutually independent $d$-dimensional random vectors (not necessarily identically distributed),

-   ${\ensuremath{\mathsf{E}}}[w_t] = 0$, ${\ensuremath{\mathsf{E}}}\bigl[w_tw_t{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}\bigr] = Q_t$ for all $t\in{\ensuremath{\mathbb{N}_0}}$,

-   there exist $C_4 > 0$ such that ${\ensuremath{\mathsf{E}}}\bigl[{\ensuremath{\left\lVert w_t \right\rVert}}^4\bigr] {\ensuremath{\leqslant}}C_4$ for all $t\in{\ensuremath{\mathbb{N}_0}}$.

Let $C_1 {\coloneqq}\sup_{t\in{\ensuremath{\mathbb{N}_0}}}{\ensuremath{\mathsf{E}}}\bigl[{\ensuremath{\left\lVert w_t \right\rVert}}\bigr]$; this is well-defined because by Jensen’s inequality we have $C_1 {\ensuremath{\leqslant}}\sqrt[4]{C_4}$. Let $({\ensuremath{\mathfrak{F}}}_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ be the natural filtration of the system .  Our proof of Theorem \[MAIN\_THEOREM\] relies on the following (immediate) adaptation of the fundamental result [@ref:pemantle99 Theorem 1]. 

\[p:pemantle99\] Let $(\xi_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ be a sequence of nonnegative random variables on some probability space $(\Omega, {\ensuremath{\mathfrak{F}}}, {\ensuremath{\mathsf{P}}})$, and let $({\ensuremath{\mathfrak{F}}}_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ be any filtration to which $(\xi_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ is adapted.  Suppose that there exist constants $b > 0$, and $J, M < \infty$, such that $\xi_0{\ensuremath{\leqslant}}J$, and for all $t$: $$\begin{gathered}
                    {\ensuremath{\mathsf{E}}}\bigl[\xi_{t+1} - \xi_t\big|{\ensuremath{\mathfrak{F}}}_t\bigr] {\ensuremath{\leqslant}}-b\quad \text{on the event }\{\xi_t > J\},\quad\text{and}\label{e:FLcond}\\
                    {\ensuremath{\mathsf{E}}}\bigl[{\ensuremath{\left\lvert{\xi_{t+1} - \xi_t}\right\rvert}}^4\big|\xi_0,\ldots, \xi_t\bigr] {\ensuremath{\leqslant}}M.\label{e:pcond}
                \end{gathered}$$ Then there exists a constant $c = c(b, J, M) > 0$ such that $\displaystyle{\sup_{t\in{\ensuremath{\mathbb{N}_0}}}{\ensuremath{\mathsf{E}}}\bigl[\xi_t^{2}\bigr] {\ensuremath{\leqslant}}c}$.

\[l:FLcond\] Consider the system , and define $\xi_t {\coloneqq}{\ensuremath{\left\lVert x_t \right\rVert}}$, $t\in{\ensuremath{\mathbb{N}_0}}$. There exists a constant $b > 0$, such that for any $r > C_1$ condition  holds in closed-loop with the control $u_t = -\operatorname{sat}_r(x_t)$.

Fix $t\in{\ensuremath{\mathbb{N}_0}}$ and $r > C_1$. We have $$\begin{aligned}
                    {\ensuremath{\mathsf{E}}}\bigl[\xi_{t+1} - \xi_t\big|{\ensuremath{\mathfrak{F}}}_t\bigr] & = {\ensuremath{\mathsf{E}}}\bigl[{\ensuremath{\left\lVert x_{t+1} \right\rVert}} - {\ensuremath{\left\lVert x_t \right\rVert}}\big|{\ensuremath{\mathfrak{F}}}_t\bigr] = {\ensuremath{\mathsf{E}}}\bigl[\bigl\|x_t + u_t + w_t\bigr\| - {\ensuremath{\left\lVert x_t \right\rVert}}\big|{\ensuremath{\mathfrak{F}}}_t\bigr]\\
                    & = {\ensuremath{\mathsf{E}}}\bigl[{\ensuremath{\left\lVert x_t - \operatorname{sat}_r(x_t) + w_t \right\rVert}} - {\ensuremath{\left\lVert x_t \right\rVert}}\big|{\ensuremath{\mathfrak{F}}}_t\bigr]\\
                    & {\ensuremath{\leqslant}}{\ensuremath{\mathsf{E}}}\bigl[{\ensuremath{\left\lVert x_t - \operatorname{sat}_r(x_t) \right\rVert}} + {\ensuremath{\left\lVert w_t \right\rVert}} - {\ensuremath{\left\lVert x_t \right\rVert}}\big|{\ensuremath{\mathfrak{F}}}_t\bigr]. 
                \end{aligned}$$ Let $J = r$ and $b{\coloneqq}r - C_1$. On the set $\{{\ensuremath{\left\lVert x_t \right\rVert}} > J\}$ we have ${\ensuremath{\left\lVert x_t - \operatorname{sat}_r(x_t) \right\rVert}} - {\ensuremath{\left\lVert x_t \right\rVert}} = -r$. From the above we get, on the set $\{{\ensuremath{\left\lVert x_t \right\rVert}} > J\}$, $$\begin{aligned}
                    {\ensuremath{\mathsf{E}}}\bigl[\xi_{t+1} - \xi_t\big|{\ensuremath{\mathfrak{F}}}_t\bigr] & {\ensuremath{\leqslant}}{\ensuremath{\mathsf{E}}}\bigl[{\ensuremath{\left\lVert x_t - \operatorname{sat}_r(x_t) \right\rVert}} + {\ensuremath{\left\lVert w_t \right\rVert}} - {\ensuremath{\left\lVert x_t \right\rVert}}\big|{\ensuremath{\mathfrak{F}}}_t\bigr]\\
                    & = -r + {\ensuremath{\mathsf{E}}}\bigl[{\ensuremath{\left\lVert w_t \right\rVert}}\bigr]\\
                    & {\ensuremath{\leqslant}}-b,
                \end{aligned}$$ where $b$ is positive by our hypothesis.  The assertion follows. 

\[l:pcond\] Consider the system  and define $\xi_t {\coloneqq}{\ensuremath{\left\lVert x_t \right\rVert}}$, $t\in{\ensuremath{\mathbb{N}_0}}$. Then for the closed-loop system with $u_t = -\operatorname{sat}_r(x_t)$ there exists a constant $M = M(C_4) > 0$ such that  holds. 

Fix $r > C_1$. Applying the triangle inequality successively, we have $${\ensuremath{\left\lvert{\xi_{t+1} - \xi_t}\right\rvert}}^4 = {\ensuremath{\left\lvert{{\ensuremath{\left\lVert x_{t+1} \right\rVert}} - {\ensuremath{\left\lVert x_t \right\rVert}}}\right\rvert}}^4 {\ensuremath{\leqslant}}{\ensuremath{\left\lVert x_{t+1} - x_t \right\rVert}}^4 = {\ensuremath{\left\lVert u_t + w_t \right\rVert}}^4 {\ensuremath{\leqslant}}\bigl(r + {\ensuremath{\left\lVert w_t \right\rVert}}\bigr)^4,$$ which leads to $${\ensuremath{\mathsf{E}}}\bigl[{\ensuremath{\left\lvert{\xi_{t+1} - \xi_t}\right\rvert}}^4\,\big|\,\xi_0, \ldots, \xi_t\bigr] {\ensuremath{\leqslant}}{\ensuremath{\mathsf{E}}}\bigl[\bigl(r + {\ensuremath{\left\lVert w_t \right\rVert}}\bigr)^4\big|\xi_0, \ldots, \xi_t\bigr] = {\ensuremath{\mathsf{E}}}\bigl[\bigl(r + {\ensuremath{\left\lVert w_t \right\rVert}}\bigr)^4\bigr].$$ Since the fourth moment of $w_t$ is uniformly bounded, expanding the right-hand side above and applying Jensen’s inequality shows that there exists some $M = M(C_4) > 0$ such that ${\ensuremath{\mathsf{E}}}\bigl[\bigl(r + {\ensuremath{\left\lVert w_t \right\rVert}}\bigr)^4\bigr] {\ensuremath{\leqslant}}M$. The assertion follows. 

\[PROPOSITION\_RW\] For $r > 0$ consider the system  under the deterministic stationary feedback policy $u_t = -\operatorname{sat}_r(x_t)$: $$\label{e:randomwalkclosedloop}
                    x_{t+1} = x_t -\operatorname{sat}_r(x_t) + w_t,\qquad x_0 = x, \quad t\in{\ensuremath{\mathbb{N}_0}}.$$ Then for every $r > C_1$ the system  satisfies $\sup_{t\in{\ensuremath{\mathbb{N}_0}}}{\ensuremath{\mathsf{E}}}_x\bigl[{\ensuremath{\left\lVert x_t \right\rVert}}^2\bigr] {\ensuremath{\leqslant}}c$ for some $c = c(x, C_1) < \infty$.

Let $r = C_1 + b$ for some $b > 0$ and $J {\coloneqq}\max\bigl\{r, {\ensuremath{\left\lVert x \right\rVert}}\bigr\}$. Lemma \[l:FLcond\] guarantees that  holds, and Lemma \[l:pcond\] shows that there exists an $M > 0$ such that  holds.  The assertion now is an immediate consequence of Proposition \[p:pemantle99\]. 

The case of $A$ orthogonal {#SUBSEC_ORTHOGONAL}
--------------------------

Next we establish part (P1) of the main theorem in the particular case of $A$ being [*orthogonal*]{}. 

\[LEMMA\_RANDOMWALK\_POLICY\] Consider the system $
y_{t+1} = Ay_t + u_t + w_t
$, where $y_t$ and $u_t$ take values in ${\ensuremath{\mathbb{R}}}^d$, $A$ is orthogonal, and $(w_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ satisfies Assumption [\[a:w\]]{}.  There exist a constant $r > 0$ and a deterministic stationary policy $\pi = (f, f, \cdots)$ such that ${\ensuremath{\left\lVert f(y) \right\rVert}} {\ensuremath{\leqslant}}r$ for all $y\in{\ensuremath{\mathbb{R}}}^d$ and $t\in{\ensuremath{\mathbb{N}_0}}$, and the closed-loop system $$\label{EQ_CLOSEDLOOP_Y}
y_{t+1} = Ay_t + f(y_t) + w_t$$ under this policy satisfies $\sup_{t\in{\ensuremath{\mathbb{N}_0}}}{\ensuremath{\mathsf{E}}}_x\bigl[ {\left\lVert}y_t {\right\rVert}^2\bigr] < \infty$.

Consider the process $(z_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ defined by $z_t {\coloneqq}(A{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}})^t \ y_t$. The second moment of $z_t$ is the same as that of $y_t$ due to orthogonality of $A$: $${\ensuremath{\mathsf{E}}}_x \bigl[ {\left\lVert}z_t {\right\rVert}^2 \bigr]
= {\ensuremath{\mathsf{E}}}_x \bigl[ {\left\lVert}(A{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}})^t \ y_t {\right\rVert}^2 \bigr]
= {\ensuremath{\mathsf{E}}}_x\bigl[ y_t{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}A^t (A{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}})^t y_t \bigr]
= {\ensuremath{\mathsf{E}}}_x\bigl[ y_t{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}}y_t\bigr]
= {\ensuremath{\mathsf{E}}}_x \bigl[ {\left\lVert}y_t {\right\rVert}^2 \bigr].$$ Now we have $$\label{EQ_TRICK}
\begin{split}
z_{t+1}
&= (A{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}})^{t+1} \ y_{t+1}= (A{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}})^{t} \ y_t + (A{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}})^{t+1} \ u_t + (A{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}})^{t+1} \ w_t = z_t + \bar{u}_t + \bar{w}_t,
\end{split}$$ where the mapping $u_t {\ensuremath{\longmapsto}}\bar{u}_t {\coloneqq}(A{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}})^{t+1} \ u_t$ is isometric and invertible, and $(\bar w_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ defined by $\bar{w}_t {\coloneqq}(A{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}})^{t+1} \ w_t$, is a sequence of zero-mean, independent (although in general [*not*]{} identically distributed) random vectors, with fourth moment given by $
{\ensuremath{\mathsf{E}}}\bigl[ {\left\lVert}\bar{w}_t {\right\rVert}^4 \bigr]
= {\ensuremath{\mathsf{E}}}\bigl[ \bigl\|(A{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}})^{t+1} \ w_t \bigr\|^4 \bigr]
= {\ensuremath{\mathsf{E}}}\bigl[ {\left\lVert}w_t {\right\rVert}^4 \bigr]
{\ensuremath{\leqslant}}C_4. 
$ Due to Proposition \[PROPOSITION\_RW\], there exists a constant $r$ such that the closed-loop system under the policy $\bar u_t = -\operatorname{sat}_r(z_t) {\eqqcolon}\bar f(z_t)$ has bounded second moment.  Consequently, the original system has bounded second moment under the policy $$u_t = A^{t+1} \bar u_t = A^{t+1} \bar f(z_t) = -A^{t+1} \operatorname{sat}_r\left( (A{\ensuremath{^{\scriptscriptstyle{\mathrm T}}}})^t \ y_t \right) {\eqqcolon}f_t(y_t).$$Noting that for any orthogonal matrix $A$ we have $\operatorname{sat}_r(Ay) = A \operatorname{sat}_r(y)$, we arrive at $
u_t = f_t(y_t) = - A \operatorname{sat}_r (y_t) {\eqqcolon}f(y_t),
$ which is indeed a stationary feedback.  Moreover, since ${\ensuremath{\left\lVert A\operatorname{sat}_r (y_t) \right\rVert}}\leq r$, we have ${\left\lVert}f(y_t) {\right\rVert}\leq r$.

In the following we will consider a nonstationary policy obtained by successive concatenations of a $k$-length policy $(f_0, f_1, \cdots f_{k-1})$ acting on the “sub-sampled” process $(x_{nk})_{n\in{\ensuremath{\mathbb{N}_0}}}$. More precisely, our policy has the form $
u_t = Bf_{t\ {\bf mod}\ k}(x_{(t \div k)k})
$ where the “$\div$” symbol denotes integer division and “${\bf mod}$” its remainder.  In words, we break the time line into segments of length $k$, and within each segment we let the controls be given by $f_0, f_1, \cdots f_{k-1}$, applied in this order always to the first state observed in the segment.  For example, $x_1 = x_0 + Bf_0(x_0) + w_0$,$x_2 = x_1 + Bf_1(x_0) + w_1$,...,$x_k = x_{k-1} + Bf_{k-1}(x_0) + w_{k-1}$,$x_{k+1} = x_{k} + Bf_0(x_{k}) + w_{k}$,$x_{k+2} = x_{k+1} + Bf_1(x_{k}) + w_{k+1}$,and so on. 

\[LEMMA\_FINAL\] Consider the system $$\label{EQ_SYSTEM_WITH_B}
x_{t+1} = Ax_t + Bu_t + w_t,$$ where $x_t$ takes values in ${\ensuremath{\mathbb{R}}}^d$, $u_t$ takes values in ${\ensuremath{\mathbb{R}}}^m$, $A$ is orthogonal, the pair $(A, B)$ is reachable in $k$ steps (i.e., $\operatorname{rank}{{\mathcal R}}_k = d$, where ${{\mathcal R}}_k = \bmat{cccc}B & AB & \cdots & A^{k-1} B\emat$), and $(w_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ satisfies Assumption [\[a:w\]]{}.  Then there exist a constant $\rho>0$ and a policy $\pi = (f_0, f_1, \cdots f_{k-1}, f_0, f_1, \cdots)$ such that ${\left\lVert}f_i(x) {\right\rVert}\leq \rho$ for all $x\in{\ensuremath{\mathbb{R}}}^d$, and the closed-loop system $$\label{EQ_FINAL_POLICY}
x_{t+1} = Ax_t + Bf_{t\ {\bf mod}\ k}(x_{(t \div k)k}) + w_t$$ under this policy satisfies $\sup_{t\in{\ensuremath{\mathbb{N}_0}}}{\ensuremath{\mathsf{E}}}_x\bigl[ {\left\lVert}x_t {\right\rVert}^2\bigr] < \infty$.

Let $\tau\in{\ensuremath{\mathbb{N}_0}}$ and consider the evolution of from time $\tau k$ to time $(\tau+1)k$: $$\label{EQ_SUBSAMPLED}
\begin{split}
x_{(\tau+1)k}
&= A^{k}\ x_{\tau k} +
{{\mathcal R}}_k
\begin{bmatrix}
u_{(\tau+1)k-1}\\ \vdots \\ u_{\tau k}
\end{bmatrix}
+ \sum_{i=0}^{k-1} A^{k-1-i} w_{\tau k + i}  = \bar{A} x_{\tau k} + \bar{u}_{\tau} + \tilde w_{\tau},
\end{split}$$ where $\tilde w_\tau {\coloneqq}\sum_{i=0}^{k-1} A^{k-1-i} w_{\tau k + i}$ is a random vector with mean zero and bounded fourth moment.  Since ${{\mathcal R}}_k$ has full rank, Lemma \[LEMMA\_BASE\] implies that for arbitrary $r>0$, any $\bar{u}_\tau$ in ${{\mathcal B}}_r$ can be expressed as $\bar{u}_\tau = \sum_{i=0}^{k-1} A^{k-1-i} B u_{\tau k + i}$, where ${\left\lVert}u_{\tau k + i} {\right\rVert}\leq r\sigma_d^{-1}$ and $\sigma_d$ is the smallest singular value of ${{\mathcal R}}_k$. But from Lemma \[LEMMA\_RANDOMWALK\_POLICY\] we know that there exists a particular $r > 0$ such that, under the stationary policy $\bar u_\tau = f(x_{\tau k}) = -\bar{A} \operatorname{sat}_r(x_{\tau k})$, the “sub-sampled” system has bounded second moment, and ${\left\lVert}\bar{u}_\tau {\right\rVert}\leq r$. Therefore, if we choose $\rho = r\sigma_d^{-1}$, there exists a constant $c = c(x, C_1, C_4) > 0$ such that $\sup_{\tau\in{\ensuremath{\mathbb{N}_0}}}{\ensuremath{\mathsf{E}}}_x\bigl[{\ensuremath{\left\lVert x_{\tau k} \right\rVert}}^2\bigr] {\ensuremath{\leqslant}}c$. It follows from the system dynamics that for $n = 0, \ldots, k-1$, $$\begin{aligned}
    {\ensuremath{\mathsf{E}}}_x\bigl[{\ensuremath{\left\lVert x_{\tau k + n} \right\rVert}}^2\bigr] & {\ensuremath{\leqslant}}2\bigl(c +  n^2 r^2 \sigma_1(B)^2\bigl) + k \max_{n=0, \ldots, k-1}\operatorname{tr}Q_{\tau k + n}\\
    & {\ensuremath{\leqslant}}2\bigl(c +  n^2 r^2 \sigma_1(B)^2\bigl) + k \sqrt{C_4},\end{aligned}$$ where the last step follows from Jensen’s inequality.  Since the right-hand side above constitutes a uniform bound, this proves the assertion. 

\[REMARK\_POLICY\] The actual policy for is $
\begin{bmatrix}
u_{(\tau +1)k-1}\\ \vdots\\ u_{\tau k}
\end{bmatrix}
= - {{\mathcal R}}_k^+ \bar{A} \operatorname{sat}_r(x_{\tau k}). 
$ The proof above shows that all the inputs $u_{(\tau+1)k-1}, \cdots, u_{\tau k}$ can be computed at time $\tau k$ in order to counteract the future effect of the current state, i.e. $\bar{A} x_{\tau k}$, and ignoring the effect of the noise for the following $k$ steps.  In the particular case when $B \in {\ensuremath{\mathbb{R}}}^{d\times d}$ has full rank, $m=d$, and obviously $k = 1$, the above policy is [*stationary*]{}, and in particular it has the form: $
u_t = f(x_t) = - B^{-1} A \operatorname{sat}_r(x_t). 
$ Once again we have ${\left\lVert}u_t {\right\rVert}\leq r\sigma_d^{-1}$, where this time $\sigma_d$ is the smallest singular value of $B$.

Proof of Theorem \[MAIN\_THEOREM\]
----------------------------------

Consider the system , with $(A,B)$ stabilizable and $(w_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ with bounded fourth moment.  If $A$ is Schur stable (that is, all the eigenvalues of $A$ belong to the interior of the unit disk), the system with zero input has bounded second moment and is asymptotically stable, and there is nothing to prove.  Otherwise, there exists a change of base in the state-space that brings the original pair $(A, B)$ to a new pair $\bigl(\tilde A, \tilde B\bigr)$, where $\tilde A$ is in real Jordan form [@ref:hornjohnson p. 150].  In particular, choosing a suitable ordering of the Jordan blocks, we can ensure that the pair $\bigl(\tilde A, \tilde B\bigr)$ has the form $\left(\bigl[\begin{smallmatrix}A_{11} & 0\\ 0 & A_{22}\end{smallmatrix}\bigr], \bigl[\begin{smallmatrix}B_1\\ B_2\end{smallmatrix}\bigr]\right)$, where $A_{11}$ is Schur stable, and $A_{22}$ has its eigenvalues on the unit circle.  Due to the stability hypothesis (the algebraic and geometric multiplicities of the eigenvalues of $A_{22}$ are equal), $A_{22}$ is therefore block-diagonal with elements on the diagonal being either $\pm 1$ or $2\times 2$ rotation matrices.  As a consequence, $A_{22}$ is orthogonal.  Moreover, since $(A,B)$ is stabilizable, the pair $(A_{22}, B_2)$ must be reachable in a number of steps $k \leq d$ which depends on the dimension of $A_{22}$ and the structure of $(A_{22}, B_2)$, since it contains precisely the modes of $A$ which are not asymptotically stable.  Summing up, we can reduce the original system $x_{t+1} = A x_t + B u_t + w_t$ to the form $
    \Bigl[\begin{smallmatrix} x^{(1)}_{t+1}\\ x^{(2)}_{t+1}\end{smallmatrix}\Bigr]
    = \Bigl[\begin{smallmatrix} A_{11} x^{(1)}_t\\ A_{22} x^{(2)}_t\end{smallmatrix}\Bigr]
    + \Bigl[\begin{smallmatrix} B_1\\ B_2\end{smallmatrix}\Bigr] u_t
    + \Bigl[\begin{smallmatrix} w^{(1)}_t\\ w^{(2)}_t\end{smallmatrix}\Bigr],
$ where $A_{11}$ is Schur stable, $A_{22}$ is orthogonal, $(A_{22}, B_2)$ is reachable, and $\Bigl(\Bigl[\begin{smallmatrix} w^{(1)}_t\\ w^{(2)}_t\end{smallmatrix}\Bigr]\Bigr)_{t\in{\ensuremath{\mathbb{N}_0}}}$ is derived from $(w_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ by means of linear transformations.  We know that since $A_{11}$ is Schur stable, the noise $\bigl(w^{(1)}_t\bigr)_{t\in{\ensuremath{\mathbb{N}_0}}}$ has bounded second moment, and the control inputs $(u_t)_{t\in{\ensuremath{\mathbb{N}_0}}}$ are bounded, then the $x^{(1)}$ sub-system is mean-square bounded under any Markovian control [@ref:ChaHokLyg-09 §4].  Therefore, if under some bounded policy the $x^{(2)}$ sub-system is mean-square bounded, the original system will also be mean-square bounded under the same policy.  Thus, at least for the proof of (P1), it suffices to restrict our attention to the subsystem described by the pair $\bigl(A_{22}, B_2\bigr)$. Suppose that this subsystem is reachable in a certain number $k\leq d$ of steps. 

The proof of (P1) coincides with the proof of Lemma \[LEMMA\_FINAL\], where we obtain $\rho = r\sigma_d^{-1}$ for $r > C_1$ and $\sigma_d$ is the smallest singular value of ${{\mathcal R}}_k$. (Here, ${{\mathcal R}}_k = \bmat{cccc}B_2 & A_{22} B_2 & \cdots & A_{22}^{k-1} B_2\emat$.) As the control authority required in the claim of the theorem, we choose precisely $R=\rho$.

To prove (P2), notice that for the closed-loop “sub-sampled” system without noise under the policy $u_t = - {{\mathcal R}}_k^{+}\bar{A}\operatorname{sat}_r\bigl(x_t^{(2)}\bigr)$, where $\bar{A} = A_{22}^k$, it holds: $$\label{EQ_SUBSAMPLED2}
x_{(\tau+1)k}^{(2)} = \bar{A} x_{\tau k}^{(2)} - \bar{A} \operatorname{sat}_r\bigl(x_{\tau k}^{(2)}\bigr).$$ As long as $x_{\tau k}^{(2)}$ is outside ${{\mathcal B}}_r$, ${\left\lVert}x_{(\tau+1)k}^{(2)} {\right\rVert}= {\left\lVert}x_{\tau k}^{(2)} {\right\rVert}- r$. Hence, in a finite number of steps it must hold ${\left\lVert}x_{\tau k}^{(2)} {\right\rVert}<r$. When for some $\bar{\tau}$ we have ${\left\lVert}x_{(\bar{\tau}-1) k}^{(2)} {\right\rVert}< r$, by the definition of $\operatorname{sat}_r(\cdot)$ we have $x_{\bar{\tau} k}^{(2)} = 0$, and consequently $x_{\tau k}^{(2)} = 0$ for all $\tau \geq \bar{\tau}$. Hence, the state of the closed-loop “sub-sampled” system converges to zero in [*finite time*]{} for any initial condition.  Then, according to the chosen policy, for all $\tau \geq \bar{\tau}$ we have $
\begin{bmatrix}
u_{(\tau +1)k-1}\\ \vdots\\ u_{\tau k}
\end{bmatrix}
= - {{\mathcal R}}_k^+ \bar{A} x_{\tau k}^{(2)} = 0
$ and $
\bar{u}_\tau =
{{\mathcal R}}_k
\begin{bmatrix}
u_{(\tau+1)k-1}\\ \vdots\\ u_{\tau k}
\end{bmatrix} = 0,
$ and consequently, for $\tau \geq \bar{\tau}$ and $\tau k \leq t < (\tau +1)k$ we also have $x_t^{(2)} = 0$, that is, $x_t^{(2)} = 0 \;\; {\ensuremath{\forall\,}}t \geq \bar{\tau}k$, which proves (P2) for the subsystem $\bigl(A_{22}, B_2\bigr)$ of our system . 

Finally, to extend the result (P2) to the general case (where $A={\mathop{\rm diag}}(A_{11}, A_{22})$), it suffices to note that, since for $t\geq \bar{\tau} k$ it also holds $u_t=0$, from the time $\bar{\tau} k$ onwards the subsystem $(A_{11}, B_1)$ is in open loop.  Since we imposed $A_{11}$ to be Schur stable, the state $x^{(1)}_t$ of the latter converges to zero as $t\rightarrow\infty$. This proves the theorem. 

Numerical Example {#SECTION_EXAMPLE}
=================

An example follows, which shows that our nonlinear policy is readily computable, and effective in bounding the state of a stable linear system in the mean square.  We executed $1000$ runs of simulation of the system $x_{t+1} = A x_t + B u_t + w_t$, where $ 
A = \left[\begin{smallmatrix}
\cos{\varphi_1} & -\sin{\varphi_1} & 0 & 0 \\ 
\sin{\varphi_1} &  \cos{\varphi_1} & 0 & 0 \\ 
0               & 0                & 0.5 & 0\\ 
0               & 0                & 0 & 0.9
\end{smallmatrix}
\right]$, $
B = \left[\begin{smallmatrix}
1 \\
0 \\
0 \\
0 
\end{smallmatrix}\right]$, with $\varphi_1 = 0.8$, $x_0 = \bmat{cccc}10&20&30&40\emat^\top$, and where $w_t$ is a Gaussian white noise with variance $I_4$. This system is marginally stable and, as is easily seen, the $2$-dimensional subsystem with eigenvalues on the unit circle is reachable in $2$ steps, whereas the $2$-dimensional Schur-stable subsystem is not reachable at all.  The control authority $R$ was chosen approximately equal to $3.6$ according to a rough estimate of $C_1 = {\ensuremath{\mathsf{E}}}_x\bigl[{\ensuremath{\left\lVert w_t \right\rVert}}\bigr]$. It should be noticed that smaller values of $R$ are also sufficient to stabilize the system. 

![ Empirical average of $||x_t||^2$ over $1000$ runs.](example.pdf){width="12cm"}

\[fig:EXAMPLE\]

Figure \[fig:EXAMPLE\] shows the empirical average of $||x_t||^2$ over the $1000$ runs, respectively with disabled control, with the chosen control authority, and with one tenth of the chosen control authority. 

A Conjecture {#SECTION_CONCLUSION}
============

We conjecture that if the noise has bounded variance, then *given any arbitrary positive uniform upper-bound* on the norm of the control, there exists a *stationary feedback policy* such that the closed-loop system is mean-square bounded.  It appears to us that a proof of this conjecture will require substantially new and nontrivial techniques. 

[xx]{}

Bao, X., Lin, Z.  Sontag, E. D. 2000, ‘Finite gain stabilization of discrete-time linear systems subject to actuator saturation’, [*Automatica*]{} [ **36**]{}(2), 269–277. 

Bertsekas, D. P. 2000, [*Dynamic [P]{}rogramming and [O]{}ptimal [C]{}ontrol*]{}, Vol.  1, 2 edn, Athena Scientific. 

Chatterjee, D., Hokayem, P.  Lygeros, J. 2009, ‘Stochastic receding horizon control with bounded control inputs—a vector-space approach’, [*IEEE Transactions on Automatic Control*]{} .  Under review.  <http://arxiv.org/abs/0903.5444>. 

Chatterjee, D.  Pal, S. 2008, ‘An excursion-theoretic approach to stability of stochastic hybrid systems’, <http://arxiv.org/abs/0901.2269>. 

Chitour, Y.  Lin, Z. 2003, ‘Finite gain $l_p$ stabilization of discrete-time systems subject to actuator saturation: The case of $p=1$’, [*IEEE Transactions on Automatic Control*]{} [**48**]{}(12), 2196–2198. 

Foss, S.  Konstantopoulos, T. 2004, ‘An overview of some stochastic stability methods’, [*Journal of Operations Research Society of Japan*]{} [**47**]{}(4), 275–303. 

Hern[á]{}ndez-Lerma, O.  Lasserre, J. B. 1996, [*Discrete-[T]{}ime [M]{}arkov [C]{}ontrol [P]{}rocesses: [B]{}asic [O]{}ptimality [C]{}riteria*]{}, Vol.  30 of [*Applications of Mathematics*]{}, Springer-Verlag, New York. 

Hokayem, P., Chatterjee, D.  Lygeros, J. 2009, On stochastic receding horizon control with bounded control inputs, [*in*]{} ‘IEEE Conference on Decision and Control and Chinese Control Conference’, Shanghai, China.  <http://arxiv.org/abs/0902.3944>. 

Horn, R. A.  Johnson, C. R. 1990 , [*Matrix [A]{}nalysis*]{}, Cambridge University Press, Cambridge. 

Lin, Z., Saberi, A.  Stoorvogel, A. A. 1996, ‘Semi-global stabilization of linear discrete-time systems subject to input saturation via linear feedback—an [ARE]{}-based approach’, [*IEEE Transactions on Automatic Control*]{} [ **41**]{}(8), 1203–1207. 

Meyn, S. P.  Tweedie, R. L. 1993 , [*Markov [C]{}hains and [S]{}tochastic [S]{}tability*]{}, Springer-Verlag, London. 

Pemantle, R.  Rosenthal, J. S. 1999, ‘Moment conditions for a sequence with negative drift to be uniformly bounded in [$L\sp r$]{}’, [*Stochastic Processes and their Applications*]{} [**82**]{}(1), 143–155. 

Stoorvogel, A. A., Saberi, A.  Weiland, S. 2007, On external semi-global stochastic stabilization of linear systems with input saturation, [*in*]{} ‘American Control Conference’, pp.  5845–5850. 

Sussmann, H. J., Sontag, E. D.  Yang, Y. 1994, ‘A general result on the stabilization of linear systems using bounded controls’, [*IEEE Transactions on Automatic Control*]{} [**39**]{}(12), 2411–2425. 

Yang, Y., Sontag, E. D.  Sussmann, H. J. 1997, ‘Global stabilization of linear discrete-time systems with bounded feedback’, [*Systems & Control Letters*]{} [ **30**]{}(5), 273–281. 

Yang, Y., Sussmann, H. J.  Sontag, E. D. 1992, Stabilization of linear systems with bounded controls, [*in*]{} M. Fliess, ed., ‘ Proceedings of the Nonlinear Control Systems Design Symposium’, IFAC Publications, pp.  15–20. 

[^1]: See [§\[s:prelims\]]{} for definitions of policies.