# A Covariance Feedback Approach to Covariance Control of Nonlinear Stochastic Systems

Authors: Salman Baroumand; Amir Reza Zaman; Mohammad Reza Mahmoudi
Journal: Complexity
 (2020)

---

## Abstract

In this paper, the covariance control algorithm for nonlinear stochastic systems using covariance feedback is studied. Covariance control of nonlinear systems scenario involves the theory of covariance control based on the idea of the covariance feedback. Therefore, the proposed covariance control algorithm is derived for our case, firstly by applying the covariance control method and linear approximation of nonlinear systems, and then it is achieved by adopting this method for a class of nonlinear stochastic systems by using feedback linearization idea and a covariance feedback controller. The effectiveness of the proposed covariance feedback algorithm is studied using numerous simulations concerning different nonlinear case studies.


---

## Body

## 1. Introduction

Most of the introduced approaches that analyze the dynamics of stochastic systems use a covariance control algorithm to design a control rule to provide limited desired covariance aims. The variance of a random variable is very important in most of the control researches such as predictive systems analysis and state estimation problems. Many researchers have done huge researches to reach and control the constrained variance objectives in related control goals to have a convergence in identification and estimation systems [1, 2] and data filtering fault detection and diagnosis [3] or have a better convergence rate of some intelligent algorithms such as genetic algorithm [4] and neural network [5]. Choosing an incorrect variance can cause instability in the whole system and can reduce the performance of the system.Most papers on covariance assignment focused on the state covariance assignment (SCA) theory for linear systems. In these studies, primary control purposes such as constraints of the variances of the states were assumed. Besides, secondary purposes could be assumed for the input variance values. By expanding the Skelton linear methods [6, 7], some theories about the covariance control of perturbed, bilinear, or nonlinear stochastic systems have been presented and reviewed in [8–13].By improving the SCA idea of a linear system, some theories about the determining the desired range of state covariance variations of nonlinear systems, multivariable, multisensor systems, or placing and setting the system covariance matrix when system parameters change have been presented [9–14]. For example, by expanding the linear methods, Kalandros [10] and Kalandros and Pao [14] deal with the problem of covariance control in multisensor systems and its application for reducing bias effects in interacting target scenarios is presented. Furthermore, in [8], [11], and [12], bilinear random systems have been controlled by the covariance control method.Recently, finite-horizon covariance control problem for stochastic discrete-time linear systems with complete state information subject to input constraints has been considered in Bakolas [15]. Also, the optimal covariance control problem for stochastic discrete-time linear systems subject to probabilistic chance constraints has been studied by Okamoto et al. [16]. The practical covariance control theory incorporating the concept of variance-constrained control for the discrete TakagiSugeno (T-S) fuzzy stochastic systems has been applied in Chang et al. [17].Chang et al. [18] presented a novel multiobjective sliding mode fuzzy control technique for a class of discrete-time nonlinear stochastic systems, such that the closed-loop system achieves passivity constraint and individual state variance constraints, simultaneously. Also, the problem of steering a linear dynamical system with complete state observation from an initial Gaussian distribution in state space to a final one with minimum energy control was addressed in Chen et al. [19].The studies on linear stochastic systems are more than nonlinear random systems. Besides, in conventional covariance control methods, according to the Riccati equation limiting condition, it is not always possible to obtain covariance placement feedback for each desired covariance. Since the reports for nonlinear random systems are few, the researchers are interested in the application of linear systems theories for nonlinear cases as far as it is possible without wasting too much precision and without loss of performance.In this paper, the covariance control algorithm for nonlinear systems using covariance feedback is investigated. The novel contributions of this paper are as follows: firstly, it is shown that, under the condition that the linear approximation of a nonlinear system is a good representation of a nonlinear system, covariance control of nonlinear systems scenario involves the theory of covariance control based on the idea of the covariance feedback. Since, in practice with real conditions, a linear approximation could not be used for any prespecified covariance matrix and numerous class of nonlinear systems; for nonlinear systems, we introduce the theory of covariance control using the feedback linearization approach developed in Khalil Hassan [20]. Also, according to the feedback linearization and covariance feedback idea stated in [12, 21], it is proved that any desired covariance matrix can be placed to its corresponding nonlinear system. For this purpose, we try to linearize the nonlinear stochastic system via feedback linearization. Then, the existing covariance control theory is extended to address the nonlinear stochastic control problems by the application of the linear system covariance control method. By changing the variables method, the nonlinear system is transformed into an equivalent linear system, and then, by using the state feedback controller approach, the closed-loop system will be stable and linear. According to the obtained covariance equations for the new transformed linear closed-loop system, the standard linear state-space model is presented for the covariance system. States of the new modelled system will be the covariance of the main nonlinear system states and also the input of the presented covariance system will be the covariance of the nonlinear system input. Then, based on the proposed model, mathematical approaches are going to be derived for nonlinear stochastic systems by using a state feedback controller and a covariance feedback controller. Finally, the simulation study is used to investigate the effectiveness of the introduced approach in a nonlinear case study.


## 2. Preliminaries and Problem Formulation

In this section, to obtain the linearized approximation of the nonlinear systems, two linearization ideas are brought that include a linear approximation of nonlinear stochastic systems and exact feedback linearization of nonlinear stochastic systems.


### 2.1. Linear Approximation of Nonlinear Stochastic Systems

Consider a general zero-mean nonlinear stochastic system as follows:(1)x˙t=fxt+But+wt,yt=Cxt,where wt is a zero-mean white noise andwt has covariance Q and fxt is an odd function relative to xt, thereby all of the mean values are zero. This system could be also considered as a mapping function such as Ω:w⟶x. Linear approximation of the general system, presented by Leithead [22, 23] in equation (1), can be considered by the following form:(2)z˙t=Azt+vt,and the corresponding mapping function is defined by ψ:υ⟶z. The relationship between Ω and ψ is defined as follows [22, 23]:(3)x=Ωw=Ψw+Ψohx,where h is a nonlinear coupling operator defined by the following function [23]:(4)Hxt=fxt−Axt.Leithead [22] proved that the linear system Ψ is an acceptable representation of a nonlinear system Ω when the error covariance matrix E^ is small, where(5)E^=EΩ−ΨwΩ−ΨwT,where E is the expectation operator. To define “small”, the evaluation of the accuracy of equation (2) from equation (1) is required. This assessment (a measure of the accuracy) can have provided by the following covariance matrix of the error [22, 23]:(6)E˜=TrZCTrZp,For some suitable positive symmetric matrix Z since p is defined as system steady-state covariance matrix and C would be defined as follows:(7)C=EΨohΨwΨohΨwT,where variable E˜ can be used to determine the accuracy of the low-order expansion when it is authentic. Therefore, the linear system Ψ is an appropriate representation of the system Ω only if E˜ is a minimum value, which means that the filtering action of Ψoh must be strong. Also, TrC represents the convergence rate of an asymptotic expansion, and if C is small, the expansions will be reliable, thus there will exist an estimation of nonlinear systems.Remark 1.
In [22], it was proved that the error measurement criterion C is a function of the optimal covariance matrix p. Therefore, in the proposed covariance control procedure, the selection of the optimal control covariance matrix p should be done in a way that is compatible with keeping the matrix C with a small value. For practical applications in zero-mean type systems, the appropriate estimation error value of the variable C should be less than 25%, and in nonzero-mean systems, the allowable estimation error range is assumed to be about 10%.
In the following, a suitable optimal choice for the nonlinear system can be achieved when the system nonlinearities are substituted by their stochastic-input characterizing functions as follows:(8)x˙t=Axt+fxt−Axt+But+wt,ut=Gxt,with nonlinear coupling function Hxt such as(9)Hxt=fxt−Axt,where the matrix A could be calculated by the following equation:(10)A=EfxtxtTP−1,where E is the expectation operator and P is the steady-state covariance of the states. To estimate the precision of the low-order development of the system, the measurement of E˜ can be used when C is small.
The main problem of this method is that the obtained estimation of the nonlinear stochastic system with estimation accuracy parametersC,E˜ is dependent on the system states’ covariance matrix. Moreover, the provided estimation’s compatibility conditions, which are minimizing the equations (6) and (7), must be satisfied simultaneously. Therefore, in practice with real conditions, this method could not be used for numerous class of nonlinear systems. Thus, with this method, we will not be able to place any desired covariance matrix with high accuracy and always there would be a considerable error value based on the system’s final results.


### 2.2. Exact Feedback Linearization of Nonlinear Stochastic Systems

Using state feedback control and change of variables method that transforms the nonlinear system into an equivalent linearized system is one of the most commonly used ideas of practical nonlinear control design [20].Consider a more general form for zero-mean nonlinear stochastic system as follows:(11a)x˙t=fxt+gxtut+Dwt,(11b)yt=cxt,where f,c, and g are smooth vector fields. Also, xt, ut, wt, and y(t) are state vector, input vector, white noise, and output vector of system, respectively. wt is a zero-mean noise with covariance Q and fxt is a function relative to xt. Consequently, all of the mean values are zero.In an input-state linearization technique, the problem of feedback linearization is going to be solved within two steps.First, we consider a state transformz=zx and input transform u=ux,v such that the nonlinear dynamics of the system would be transferred into a linear time-invariant dynamic as the following form:(12a)z˙t=Azt+bvt+dwt,where(12b)A=010…000010…0⋮⋮⋮⋮⋮⋮00…00100…000,b=00⋮01,(12c)ut=αxt+βxtvt.Then, linear standard control techniques (such as pole placement) are used to designvt. So, by using the designed linearization feedback and the suitable state transform z=zx, the problem of covariance control and stabilization of the nonlinear system (11) would become a problem of stabilization and covariance control with the new linear system dynamic (12a) with the new input vt.It should be noticed that the suggested covariance feedback idea Khaloozadeh and Baromand [21] is completely independent of the designed stabilizer state feedback in the tracking or stabilizer problem. So, we try to determine the statistical features of the control signal by considering states’ covariance feedback of the system (xt or zt), and then, we try to control the covariance of the main system states as possible. Because of the independence of this method of the input signal, it enables us not to have some limitations for designing the control signal ut.Unlike most of the presented common methods for placing the covariance matrix for linear or nonlinear systems, we can practically consider the issue of the system output tracking at the same time as the covariance control issue is considered as well. Another thing to pay attention to is that the presented idea of feedback linearization is obtained with a state transform and also an input transform, with this issue varying with the presented common linearization idea in Leithead [22] and Chung and Chang [9], that was determined and considered for the limited work areas, but in the presented feedback linearization idea, we will consider no limitations for this issue. Moreover, the idea of using linearization feedback for nonlinear systems covariance control issues could be taken within two steps.In the first step, by using the state transformz=zx, the nonlinear system (11) would become the linear standard form (12a). The condition of the existence of such a state transform will be mentioned in the following.In the second step, by using the idea of covariance feedback, the covariance of the control signal would be designed in such a way that the system states’ covariance will be placed to the desired preset values.Therefore, as a summary, the linearization feedback and state variable changez=zx could be done by the following steps.First, the vectorfieldg,adfg,…,adfn−1g should be determined for the nonlinear system (11) where adfg=f,g=∇gf−∇fg as a Lie Bracket operator. If we prove the existence of two conditions which are controllability and reversibility Khalil Hassan [20], then the first state zi (the output function that leads to input-output linearization with a relative degree of n) can be calculated as the following equation:(13)∇ziadfig=0,i=1,…,n−2,where adfig=f,adfi−1g. Also, the state transform zx and input transform ux can be calculated as follows:(14)Zx=z1,Lfz1,…,Lfn−1z1,ut=αxt+βxtvt,where αx=Lfnz1/LgLfn−1z1, βx=1/LgLfn−1z1, and Lfh=∇hf are determined as a Lie derivatives of the function h to the vector field f.By using the linearized new states and obtained control law in the previous step, the mentioned nonlinear system can be transferred into the standard linear system form (12a). Furthermore, the unstable nonlinear system can be stable by using the pole placement idea. Now, the covariance feedback idea could be implemented by following to place the desired preset covariance matrix to the new linearized system.Given that most of the nonlinear systems can be transformed into the equivalent linear system by using linearizable feedback, the feedback covariance idea can be used for controlling most of the nonlinear systems.


## 2.1. Linear Approximation of Nonlinear Stochastic Systems

Consider a general zero-mean nonlinear stochastic system as follows:(1)x˙t=fxt+But+wt,yt=Cxt,where wt is a zero-mean white noise andwt has covariance Q and fxt is an odd function relative to xt, thereby all of the mean values are zero. This system could be also considered as a mapping function such as Ω:w⟶x. Linear approximation of the general system, presented by Leithead [22, 23] in equation (1), can be considered by the following form:(2)z˙t=Azt+vt,and the corresponding mapping function is defined by ψ:υ⟶z. The relationship between Ω and ψ is defined as follows [22, 23]:(3)x=Ωw=Ψw+Ψohx,where h is a nonlinear coupling operator defined by the following function [23]:(4)Hxt=fxt−Axt.Leithead [22] proved that the linear system Ψ is an acceptable representation of a nonlinear system Ω when the error covariance matrix E^ is small, where(5)E^=EΩ−ΨwΩ−ΨwT,where E is the expectation operator. To define “small”, the evaluation of the accuracy of equation (2) from equation (1) is required. This assessment (a measure of the accuracy) can have provided by the following covariance matrix of the error [22, 23]:(6)E˜=TrZCTrZp,For some suitable positive symmetric matrix Z since p is defined as system steady-state covariance matrix and C would be defined as follows:(7)C=EΨohΨwΨohΨwT,where variable E˜ can be used to determine the accuracy of the low-order expansion when it is authentic. Therefore, the linear system Ψ is an appropriate representation of the system Ω only if E˜ is a minimum value, which means that the filtering action of Ψoh must be strong. Also, TrC represents the convergence rate of an asymptotic expansion, and if C is small, the expansions will be reliable, thus there will exist an estimation of nonlinear systems.Remark 1.
In [22], it was proved that the error measurement criterion C is a function of the optimal covariance matrix p. Therefore, in the proposed covariance control procedure, the selection of the optimal control covariance matrix p should be done in a way that is compatible with keeping the matrix C with a small value. For practical applications in zero-mean type systems, the appropriate estimation error value of the variable C should be less than 25%, and in nonzero-mean systems, the allowable estimation error range is assumed to be about 10%.
In the following, a suitable optimal choice for the nonlinear system can be achieved when the system nonlinearities are substituted by their stochastic-input characterizing functions as follows:(8)x˙t=Axt+fxt−Axt+But+wt,ut=Gxt,with nonlinear coupling function Hxt such as(9)Hxt=fxt−Axt,where the matrix A could be calculated by the following equation:(10)A=EfxtxtTP−1,where E is the expectation operator and P is the steady-state covariance of the states. To estimate the precision of the low-order development of the system, the measurement of E˜ can be used when C is small.
The main problem of this method is that the obtained estimation of the nonlinear stochastic system with estimation accuracy parametersC,E˜ is dependent on the system states’ covariance matrix. Moreover, the provided estimation’s compatibility conditions, which are minimizing the equations (6) and (7), must be satisfied simultaneously. Therefore, in practice with real conditions, this method could not be used for numerous class of nonlinear systems. Thus, with this method, we will not be able to place any desired covariance matrix with high accuracy and always there would be a considerable error value based on the system’s final results.


## 2.2. Exact Feedback Linearization of Nonlinear Stochastic Systems

Using state feedback control and change of variables method that transforms the nonlinear system into an equivalent linearized system is one of the most commonly used ideas of practical nonlinear control design [20].Consider a more general form for zero-mean nonlinear stochastic system as follows:(11a)x˙t=fxt+gxtut+Dwt,(11b)yt=cxt,where f,c, and g are smooth vector fields. Also, xt, ut, wt, and y(t) are state vector, input vector, white noise, and output vector of system, respectively. wt is a zero-mean noise with covariance Q and fxt is a function relative to xt. Consequently, all of the mean values are zero.In an input-state linearization technique, the problem of feedback linearization is going to be solved within two steps.First, we consider a state transformz=zx and input transform u=ux,v such that the nonlinear dynamics of the system would be transferred into a linear time-invariant dynamic as the following form:(12a)z˙t=Azt+bvt+dwt,where(12b)A=010…000010…0⋮⋮⋮⋮⋮⋮00…00100…000,b=00⋮01,(12c)ut=αxt+βxtvt.Then, linear standard control techniques (such as pole placement) are used to designvt. So, by using the designed linearization feedback and the suitable state transform z=zx, the problem of covariance control and stabilization of the nonlinear system (11) would become a problem of stabilization and covariance control with the new linear system dynamic (12a) with the new input vt.It should be noticed that the suggested covariance feedback idea Khaloozadeh and Baromand [21] is completely independent of the designed stabilizer state feedback in the tracking or stabilizer problem. So, we try to determine the statistical features of the control signal by considering states’ covariance feedback of the system (xt or zt), and then, we try to control the covariance of the main system states as possible. Because of the independence of this method of the input signal, it enables us not to have some limitations for designing the control signal ut.Unlike most of the presented common methods for placing the covariance matrix for linear or nonlinear systems, we can practically consider the issue of the system output tracking at the same time as the covariance control issue is considered as well. Another thing to pay attention to is that the presented idea of feedback linearization is obtained with a state transform and also an input transform, with this issue varying with the presented common linearization idea in Leithead [22] and Chung and Chang [9], that was determined and considered for the limited work areas, but in the presented feedback linearization idea, we will consider no limitations for this issue. Moreover, the idea of using linearization feedback for nonlinear systems covariance control issues could be taken within two steps.In the first step, by using the state transformz=zx, the nonlinear system (11) would become the linear standard form (12a). The condition of the existence of such a state transform will be mentioned in the following.In the second step, by using the idea of covariance feedback, the covariance of the control signal would be designed in such a way that the system states’ covariance will be placed to the desired preset values.Therefore, as a summary, the linearization feedback and state variable changez=zx could be done by the following steps.First, the vectorfieldg,adfg,…,adfn−1g should be determined for the nonlinear system (11) where adfg=f,g=∇gf−∇fg as a Lie Bracket operator. If we prove the existence of two conditions which are controllability and reversibility Khalil Hassan [20], then the first state zi (the output function that leads to input-output linearization with a relative degree of n) can be calculated as the following equation:(13)∇ziadfig=0,i=1,…,n−2,where adfig=f,adfi−1g. Also, the state transform zx and input transform ux can be calculated as follows:(14)Zx=z1,Lfz1,…,Lfn−1z1,ut=αxt+βxtvt,where αx=Lfnz1/LgLfn−1z1, βx=1/LgLfn−1z1, and Lfh=∇hf are determined as a Lie derivatives of the function h to the vector field f.By using the linearized new states and obtained control law in the previous step, the mentioned nonlinear system can be transferred into the standard linear system form (12a). Furthermore, the unstable nonlinear system can be stable by using the pole placement idea. Now, the covariance feedback idea could be implemented by following to place the desired preset covariance matrix to the new linearized system.Given that most of the nonlinear systems can be transformed into the equivalent linear system by using linearizable feedback, the feedback covariance idea can be used for controlling most of the nonlinear systems.


## 3. Description of Covariance System and Design of Covariance Controller

Covariance control theory was described in references to find feedback gain sets that led to assign suitable state covariance by these sets. According to these mentioned references, the following variance limitations will be satisfied:(15)limn⟶∞Exit2≤δi2,i=1,2,…,nx,where δi determines the root-mean squared (RMS) constraints of the variances related to the system states. In this paper, to find the solution for zero-mean nonlinear stochastic system (ZMNSS), it can be simplified to find the covariance state feedback gain, that attains a determined value of p to comply with the variance limitations (equation (11)). If the small value of the measured matrix C is selected and the strong filtering action (6) in statistical linearization is assumed or by considering the controllability and involutivity conditions for input-state feedback linearization idea, finally, the linearized system ((2) and (12a)) will be obtained. Besides, in both cases, the nonlinear system can be specified as(16a)z˙t=A˜zt+B˜vt+D˜wt,(16b)ut=αxt+βxtvt,where A˜zt is the describing function of fxt and ut is a function of xt and vt in any suitable forms, and the applied feedback gain G in the state feedback stabilizing signal vt=Gzt is calculated based on the system parameters in (16a) by using the pole-placement method. Also A˜∈Rn×n, D˜∈Rn×m, B∈Rn×m, zt∈Rn, vt∈Rm, and wt∈Rm. On the other hand, the input noise of the system wt in equation (16a) satisfies the following statistical specification:(17a)Ewt=0;Ex0wTt=0,(17b)Ez0wTt=0,(17c)EwtwTs=Qδt−s,where Q∈Rm×m is the process noise covariance matrix.Also, with the assumption ofEut=0, the positive definite covariance matrix of final states, pt and p˜t, can be defined as follows:(18a)pt=Ext−Extxt−ExtT,(18b)p˜t=Ezt−Eztzt−EztT,where(19)Ext≅1t∑k=1txt.The objective is to imposelimt⟶∞Ext=0 and assume a prespecified positive definite covariance matrix for the states, xt. We name this problem as a covariance assignment problem.In [21], to solve the covariance assignment problem, a method based on the covariance feedback technique is presented. In this research, we utilize the covariance assignment for nonlinear systems via the covariance feedback method.In the following discussion, the aim is to reach a stochastic control signalv˜t with mean zero and the covariance matrix, Vt∈Rm×m, such that the steady-state covariance pt converges to a prespecified value pdt. Then, an additional action input v˜t is determined in a way to assign the prespecified covariance to a nonlinear system based on the covariance feedback technique. Figure 1 demonstrates the structure of the overall control system.Figure 1
The block diagram of the overall control system.Thus, with the new additive control signal,v˜t, (16a) can be rewritten by(20)z˙t=A˜zt+B˜vt+D˜wt+v˜t,y˜t=C˜zt,where vt=Gzt is stabilizable control signal that imposes the action input utut=αxt+βxtvt and v˜t is an additive input noise signal which is specified by a method to assign the prespecified covariance matrix via covariance feedback technique.At first, the case with a zero-mean control signalv˜t is considered. Then, the problem is referred to as a regulator problem such that, for vt, the following assumptions is satisfied:(21)Eut=0,Ez0vTt=0,Evt1wTt2=0,∀t1&t2.Also, for additional input signalv˜t, covariance matrix Vt is defined as(22)Vt=Ev˜tv˜tT,where(23a)Ez0v˜Tt=0,(23b)Ev˜t1wTt2=0,∀t1&t2.For azt with mean zero, the covariance of the system states is represented by(24)p˜t=EztzTt.Using ((20)–(24)), the differential equation matrix of p˜t can be derived by(25)dp˜tdt=A˜+B˜Gpˇt+p˜tA˜+B˜GT+Vt+D˜QtD˜T,=A¯p˜t+p˜tA¯T+Vt+dt.


### 3.1. Description of Covariance System

Letzt=z1,z2,…,znT be the vector of states, v˜t=v˜1,v˜2,…,v˜mT be the additional control input, and wt=w1,w2,…,wnT be the vector of process noise, respectively. By the symmetry of p˜t, Q, and Vt, we can partition and rearrange p˜t, Vt, and Qt by [24](26)pcovt=Ez1tz1tEz1tz2t,…,Ez1tzntEz2tz2t,…,Ez2tznt,…,EzntzntT,(27)Vcovt=Ev˜1tv˜1tEv˜1tv˜2t,Ev˜1tv˜mtEv˜2tv˜2t,…,Ev˜2tv˜mt,…,Ev˜mtv˜mtT,(28)Qcovt=Ew1tw1tEw1tw2tEw1twntEw2tw2t,…,Ew2twnt,…,EwntwntT,where pcov, Vcov, and Qcov are nn+1/2×1, mm+1/2×1, and nn+1/2×1 vectors, respectively. Moreover, DQtDT can be rearranged as d¯t∈Rnn+1/2×1.As a result, the dynamic covariance system (25) can be reduced to the standard state-space model by defining A¯cov, Bcov, and Ccov as follows [24]:(29)dpcovtdt=A¯covpcovt+BcovVcovt+d¯t,Ycovt=Ccovpcovt,where Bcov=I and the elements of A¯cov∈Rnn+1/2×nn+1/2 and d¯t are the state and disturbance matrices of the proposed state-space covariance system, respectively, and computed based on the nonlinear mapping of A¯=A˜+B˜G and dt. Also, d¯t is deterministic, and we can assume it as a disturbance term. Also Ccov is arbitrary output matrix of the presented covariance system.It should be noted that, although the model (20) is stochastic, the given covariance state-space model (29) is deterministic. Also, if A¯cov and Bcov are controllable, then there exists a solution to the covariance assignment problem.


### 3.2. Design of Covariance Controller

In the previous discussions, for the state covariance of a nonlinear stochastic system, a deterministic and linear model was introduced based on the combination of stochastic approximation and feedback linearization and rearrangement ideas. But, designing the covariance feedbackVcovt=Kpcovt (that imposes the action input v˜t) to control the covariance assigned in the specified matrix pd is the main problem. By the linearity of the covariance system presented in (29) using the introduced technique in Khaloozadeh and Baromand [21], with the integral state q˙t=r−Ccovpcovt, the new augmented state space is defined by(30)p˙covtq˙t=A¯¯pcovtqt+B¯¯Vcovt+d¯0+0Ir,where r is defined as the desired covariance vector and Ccov=I and(31)A¯¯=A¯cov0−Ccov0,B¯¯=Bcov0.Thus, the covariance control problem results in the stability of the augmented system (30). Controllability of the pair A¯¯,B¯¯ is the necessary condition to exist as a solution to the covariance tracking problem.The state covariance feedbackVcovt in (23a) is then given as follows:(32)Vcovt=K1K2pcovtqt=Kpcovtqt.The controller gainsK and thus Vcovt is then obtained using the pole placement method.The elements of the vectorVcovt determine the covariance matrix of the action input, v˜t (i.e., Vt=Ev˜tv˜tT). Thus, v˜t must be produced such that v has the covariance matrix V. Because of the regulatory of the problem in the original state system, the control signal v˜t is a white noise with mean zero and the covariance matrix V. Therefore, in each iteration, the input signal v˜t will be derived as follows:(33)v˜t=Vt0.5×ent,where eni=1m is denoted as white noise.Thus, with the presented idea, the control strategy is composed of two stages. The first, state feedback signalvt=Gzt (feedback from the mean value of state) stabilizes and linearized the system. The second, covariance feedback signal v˜t imposes the input strategy such that the system’s states adopt the predefined covariance value.Remark 2.
In the presented covariance feedback and linear approximation idea, unlike most of the derived strategies based on the state feedback that they could not be able to place any desired covariance matrices, with this mentioned covariance feedback idea and obtaining a distinct control signal from the stabilizer control signal, we can even place any covariance matrices. Therefore, we can calculate the desired covariance matrix that satisfies the expected conditions of the control problem for representing a better estimation of the nonlinear system according to the minimum permitted value of the nonlinear error signal. Finally, we can achieve the control goal of the problem.
In summary, the presented covariance assignment algorithm is presented in Algorithm1.Algorithm 1: Nonlinear covariance assignment strategy.
If the small value of the measured matrixc is selected and the strong filtering action (6) in statistical linearization is assumed or by input-state feedback linearization idea, finally, in both cases, the nonlinear system can be specified as follows (16a):Step 1: consider a state transformz=zx and feedback control input (16b) such that the nonlinear dynamic (2) or (11) would be transferred into the linear dynamic (16a).Step 2: online state covariance estimation of the main system (20).Step 3: rearrange and modify the covariance vectorVcov and Pcov from (26) and (27).Step 4: calculate the covariance control variableVcovk from ((29)–(32)).Step 5: modify the covariance matrixVk from Vcovk as covariance of control signal v˜k.Step 6: generate the covariance control signalv˜t with covariance matrix Vk from (33).


## 3.1. Description of Covariance System

Letzt=z1,z2,…,znT be the vector of states, v˜t=v˜1,v˜2,…,v˜mT be the additional control input, and wt=w1,w2,…,wnT be the vector of process noise, respectively. By the symmetry of p˜t, Q, and Vt, we can partition and rearrange p˜t, Vt, and Qt by [24](26)pcovt=Ez1tz1tEz1tz2t,…,Ez1tzntEz2tz2t,…,Ez2tznt,…,EzntzntT,(27)Vcovt=Ev˜1tv˜1tEv˜1tv˜2t,Ev˜1tv˜mtEv˜2tv˜2t,…,Ev˜2tv˜mt,…,Ev˜mtv˜mtT,(28)Qcovt=Ew1tw1tEw1tw2tEw1twntEw2tw2t,…,Ew2twnt,…,EwntwntT,where pcov, Vcov, and Qcov are nn+1/2×1, mm+1/2×1, and nn+1/2×1 vectors, respectively. Moreover, DQtDT can be rearranged as d¯t∈Rnn+1/2×1.As a result, the dynamic covariance system (25) can be reduced to the standard state-space model by defining A¯cov, Bcov, and Ccov as follows [24]:(29)dpcovtdt=A¯covpcovt+BcovVcovt+d¯t,Ycovt=Ccovpcovt,where Bcov=I and the elements of A¯cov∈Rnn+1/2×nn+1/2 and d¯t are the state and disturbance matrices of the proposed state-space covariance system, respectively, and computed based on the nonlinear mapping of A¯=A˜+B˜G and dt. Also, d¯t is deterministic, and we can assume it as a disturbance term. Also Ccov is arbitrary output matrix of the presented covariance system.It should be noted that, although the model (20) is stochastic, the given covariance state-space model (29) is deterministic. Also, if A¯cov and Bcov are controllable, then there exists a solution to the covariance assignment problem.


## 3.2. Design of Covariance Controller

In the previous discussions, for the state covariance of a nonlinear stochastic system, a deterministic and linear model was introduced based on the combination of stochastic approximation and feedback linearization and rearrangement ideas. But, designing the covariance feedbackVcovt=Kpcovt (that imposes the action input v˜t) to control the covariance assigned in the specified matrix pd is the main problem. By the linearity of the covariance system presented in (29) using the introduced technique in Khaloozadeh and Baromand [21], with the integral state q˙t=r−Ccovpcovt, the new augmented state space is defined by(30)p˙covtq˙t=A¯¯pcovtqt+B¯¯Vcovt+d¯0+0Ir,where r is defined as the desired covariance vector and Ccov=I and(31)A¯¯=A¯cov0−Ccov0,B¯¯=Bcov0.Thus, the covariance control problem results in the stability of the augmented system (30). Controllability of the pair A¯¯,B¯¯ is the necessary condition to exist as a solution to the covariance tracking problem.The state covariance feedbackVcovt in (23a) is then given as follows:(32)Vcovt=K1K2pcovtqt=Kpcovtqt.The controller gainsK and thus Vcovt is then obtained using the pole placement method.The elements of the vectorVcovt determine the covariance matrix of the action input, v˜t (i.e., Vt=Ev˜tv˜tT). Thus, v˜t must be produced such that v has the covariance matrix V. Because of the regulatory of the problem in the original state system, the control signal v˜t is a white noise with mean zero and the covariance matrix V. Therefore, in each iteration, the input signal v˜t will be derived as follows:(33)v˜t=Vt0.5×ent,where eni=1m is denoted as white noise.Thus, with the presented idea, the control strategy is composed of two stages. The first, state feedback signalvt=Gzt (feedback from the mean value of state) stabilizes and linearized the system. The second, covariance feedback signal v˜t imposes the input strategy such that the system’s states adopt the predefined covariance value.Remark 2.
In the presented covariance feedback and linear approximation idea, unlike most of the derived strategies based on the state feedback that they could not be able to place any desired covariance matrices, with this mentioned covariance feedback idea and obtaining a distinct control signal from the stabilizer control signal, we can even place any covariance matrices. Therefore, we can calculate the desired covariance matrix that satisfies the expected conditions of the control problem for representing a better estimation of the nonlinear system according to the minimum permitted value of the nonlinear error signal. Finally, we can achieve the control goal of the problem.
In summary, the presented covariance assignment algorithm is presented in Algorithm1.Algorithm 1: Nonlinear covariance assignment strategy.
If the small value of the measured matrixc is selected and the strong filtering action (6) in statistical linearization is assumed or by input-state feedback linearization idea, finally, in both cases, the nonlinear system can be specified as follows (16a):Step 1: consider a state transformz=zx and feedback control input (16b) such that the nonlinear dynamic (2) or (11) would be transferred into the linear dynamic (16a).Step 2: online state covariance estimation of the main system (20).Step 3: rearrange and modify the covariance vectorVcov and Pcov from (26) and (27).Step 4: calculate the covariance control variableVcovk from ((29)–(32)).Step 5: modify the covariance matrixVk from Vcovk as covariance of control signal v˜k.Step 6: generate the covariance control signalv˜t with covariance matrix Vk from (33).


## 4. Illustrative Examples and Numerical Results

This section is devoted to studying the ability of the proposed control technique. For this purpose, four nonlinear stochastic systems are considered. The simulations are accomplished after 100 Monte Carlo runs using the MATLAB R2014a software on a PC (processor: Intel (R) Core i5-4570 CPU @ 3.2 GHz, RAM: 4.00 GB, System Type: 64 bit). In the simulation, the presented methods do not require any additional assumption on the model; however, the result becomes ill-conditioned if the sampling time is large or if the poles of the system are very fast. In all simulation results, the sampling time selected is 0.05 sec.This section is divided into two parts. The first part considers two simple systems. The performance of the introduced covariance control technique is investigated, and the ability of the presented control strategy is also compared with other alternative methods presented by Chung and Chang [9]. The second part considers two complex systems. For these systems, the performance of the introduced covariance control technique is investigated.


### 4.1. Performance Compression

These examples involve two simple zero-mean and nonzero-mean nonlinear stochastic systems that may be found in [22, 23].Example 1.
Consider a zero-mean nonlinear stochastic system described by(34)x˙t=−xt3+ut+wt,where d = b = 1 and wt is zero-mean white noise with variance value 2. Assume that the goal of tracking is to satisfy the variance condition pdt=varxt=0.3. According to the linear approximation idea and from [21, 22], the linearized gain statistical matrix can be easily calculated as A˜=−3p=−1.5. According to the A˜, the linearized system is stable and it is not necessary to use state feedback A¯=A˜. Estimation error between the nonlinear system and linearized system is in Figure 2.
By substitutingA˜ and B˜=D˜=1 in (12a) and assuming pcovt=pt=varxt, the covariance system parameters will be described as A¯cov=2×−1.5, Bcov=1, and Ccov=1.
Using pole placement and the state-space model of the system (30), the feedback gain matrix is calculated and the covariance vector of additive input Vcovt (equation (32)) is computed as(35)Vcovt=−6.9−16pcovtqt,q˙t=0.5−pcovt,pcovt=Ext,xt.
The performance compression between our covariance feedback control design and state feedback controller approach [9]) is shown in Figure 3.
The conventional state feedback only guarantees the upper band of the desired state variance and fails to track the desired trajectory. Compared to conventional state feedback, our proposed controller can track the desired trajectory. The mean value of CPU times for 100 runs (with sample time 0.05 sec) is 16.15 sec. Low CPU time also verified that this approach is not too time-consuming.Figure 2
Estimation error between the nonlinear system and linearized system.Figure 3
State variance and control signal of controlled system with proposed controller and state feedback approach [9]: (a) covariance tracking problem; (b) control signal.
(a)(b)Example 2.
Consider the nonzero-mean nonlinear stochastic system described by(36)x˙t=−sinxt+0.4+but+dwt,where d = b = 1 and wt is the zero-mean white noise with variance value 0.5. Assume that the goal of tracking is to satisfy the variance condition pdt=varxt=0.1.
The gain statistical matrix can be easily calculated as follows [22]:(37)A˜=−exp−0.5pd.
According to theA˜, the linearized system is stable, and using pole placement and the state-space model of the system (29), the feedback gain matrix is calculated and the covariance vector of additive input Vcovt (32) is computed. The performance compression between our covariance feedback control design and state feedback controller approach [9] is shown in Figure 4.
Compared to conventional state feedback, our proposed controller can track the desired trajectory. As can be seen, the variance of the state will converge to the desired covariance matrix value with a relatively good convergence rate. The mean value of CPU times for 100 runs (with sample time 0.05 sec) is also 24.96 sec.Figure 4
State variance and control signal of controlled system with proposed controller and state feedback approach [9]: (a) covariance tracking problem; (b) control signal.
(a)(b)


### 4.2. Examples for More Complex Nonlinear Systems

In methods based on linear approximation idea, the feedback gain is dependent on estimation accuracy parameters (C, E˜), so in examples that are more complex, we do not have an acceptable estimation or the obtained feedback gains do not have an acceptable response. In the next examples, the simulation results are presented for a wide class of nonlinear systems with more complexity.Example 3.
To introduce the idea of feedback linearization, consider a nonlinear stochastic continuous system (11) with(38)x1˙t=x2t+wt,x2˙t=−αsinx1t+δ−sinδ−bx2t+ut+2wt,where wt is the zero-mean white noise with variance value 1. In this example, the goal is to force the system states to track the desired covariance matrix:(39)pdt=covx1t,x2t=0.2−0.1−0.10.3.
To remove the nonlinear term, we can choose a change of variables and state feedback control as(40)ut=αsinx1t+δ−sinδ+vt,zt=z1tz2t=x1tx2t,which yields(41)z˙t=A˜zt+B˜vt+D˜wt+v˜t,where A˜=010−b,B˜=01,D˜=12,α=1,δ=π/4,b=1, and v˜t is zero-mean additive covariance controller input. Thus, the stabilization problem for the nonlinear system has been reduced to a stabilization problem for a controllable linear system with a proper linear state feedback control law vt=−k1−k2zt=−3−2zt, where the poles of the closed-loop systems are assigned to the value −1.5+0.86j−1.5−0.86j. It is worth noting that the values of k1 and k2 are obtained by applying the pole-placement method stated in Chang et al. [25].
Assume(42)pcovt=varx1covx1,x2varx2T,Vcovt=varv1covv1,v2varv2T,r=0.2−0.10.3T,q˙t=r−pcovt.
By computingA¯cov, Bcov, and Ccov=I and considering that the poles of the closed-loop covariance systems were assigned to the value −2−2−2−3−3−3, the feedback gain is calculated and the covariance vector of additive input Vcovt will be computed as(43)Vcovt=−520−6000410−6000300−6pcovtqt.
The elements of vectorVcovt determine the covariance matrix Vt, where Vt=Ev˜tv˜tT is the covariance of additive controller input v˜t.
The performance of the proposed controller for this system is illustrated in Figures5–7.
Simulation results proposed that the controller can track the desired trajectory. As can be seen, the covariance and variance of the states are converged to the optimal covariance matrix value with a relatively good convergence rate. The mean value of CPU times for 100 runs (with sample time 0.05 sec) is also 11.56 sec.Figure 5
States variance and desired states variance.Figure 6
State trajectories of the system.Figure 7
Control signals trajectories.Example 4.
Consider the nonlinear stochastic system described by the following equations:(44)x1˙t=−2x1t+ax2t+sinx1t+2wt,x2˙t=−x2tcosx1t+utcos2x1t,where wt is zero-mean white noise with variance value 1. In this example, the goal is to force the system states to track the desired covariance matrix:(45)pdt=varx1t=0.3.
To remove the nonlinear term, we can choose the change of variables and state feedback control as(46)ut=1acosz1vt−cosz1sinz1+2z1cosz1,zt=z1tz2t=x1tax2t+sinx1,which yields(47)z˙t=A˜zt+B˜vt+D˜wt+v˜t,where A˜=−2100,B˜=01,D˜=20,a=1, and v˜t is zero-mean additivecovariance controller input.
The proper linear state feedback control law is assumed to be asvt=0−5zt, where the poles were assigned to the value −2−5 using the developed pole-placement approach in [25].
By assumingpcovt and Vcovt similar to Example 3 and q˙t=0.3−pcovt, computing A¯cov, Bcov, and Ccov=100, and supposing that the poles of the closed-loop covariance systems assign to the −10−1−2−9, the feedback gain is calculated and the covariance vector of additive input Vcovt is computed as(48)Vcovt=−820−2007100010pcovtqt.
Additionally, the elements of the vectorVcovt determine the covariance matrix Vt, where Vt=Ev˜tv˜tT is the covariance of additive controller input v˜t.
The performance of the proposed controller applied to this system is illustrated in Figures8 and 9.
Simulation results proposed that the controller can track the desired trajectory with a relatively good convergence rate. The mean value of CPU times for 100 runs (with sample time 0.05 sec) is 58.29 sec.Figure 8
State variance and state trajectories of the system.Figure 9
Control signal trajectories.


## 4.1. Performance Compression

These examples involve two simple zero-mean and nonzero-mean nonlinear stochastic systems that may be found in [22, 23].Example 1.
Consider a zero-mean nonlinear stochastic system described by(34)x˙t=−xt3+ut+wt,where d = b = 1 and wt is zero-mean white noise with variance value 2. Assume that the goal of tracking is to satisfy the variance condition pdt=varxt=0.3. According to the linear approximation idea and from [21, 22], the linearized gain statistical matrix can be easily calculated as A˜=−3p=−1.5. According to the A˜, the linearized system is stable and it is not necessary to use state feedback A¯=A˜. Estimation error between the nonlinear system and linearized system is in Figure 2.
By substitutingA˜ and B˜=D˜=1 in (12a) and assuming pcovt=pt=varxt, the covariance system parameters will be described as A¯cov=2×−1.5, Bcov=1, and Ccov=1.
Using pole placement and the state-space model of the system (30), the feedback gain matrix is calculated and the covariance vector of additive input Vcovt (equation (32)) is computed as(35)Vcovt=−6.9−16pcovtqt,q˙t=0.5−pcovt,pcovt=Ext,xt.
The performance compression between our covariance feedback control design and state feedback controller approach [9]) is shown in Figure 3.
The conventional state feedback only guarantees the upper band of the desired state variance and fails to track the desired trajectory. Compared to conventional state feedback, our proposed controller can track the desired trajectory. The mean value of CPU times for 100 runs (with sample time 0.05 sec) is 16.15 sec. Low CPU time also verified that this approach is not too time-consuming.Figure 2
Estimation error between the nonlinear system and linearized system.Figure 3
State variance and control signal of controlled system with proposed controller and state feedback approach [9]: (a) covariance tracking problem; (b) control signal.
(a)(b)Example 2.
Consider the nonzero-mean nonlinear stochastic system described by(36)x˙t=−sinxt+0.4+but+dwt,where d = b = 1 and wt is the zero-mean white noise with variance value 0.5. Assume that the goal of tracking is to satisfy the variance condition pdt=varxt=0.1.
The gain statistical matrix can be easily calculated as follows [22]:(37)A˜=−exp−0.5pd.
According to theA˜, the linearized system is stable, and using pole placement and the state-space model of the system (29), the feedback gain matrix is calculated and the covariance vector of additive input Vcovt (32) is computed. The performance compression between our covariance feedback control design and state feedback controller approach [9] is shown in Figure 4.
Compared to conventional state feedback, our proposed controller can track the desired trajectory. As can be seen, the variance of the state will converge to the desired covariance matrix value with a relatively good convergence rate. The mean value of CPU times for 100 runs (with sample time 0.05 sec) is also 24.96 sec.Figure 4
State variance and control signal of controlled system with proposed controller and state feedback approach [9]: (a) covariance tracking problem; (b) control signal.
(a)(b)


## 4.2. Examples for More Complex Nonlinear Systems

In methods based on linear approximation idea, the feedback gain is dependent on estimation accuracy parameters (C, E˜), so in examples that are more complex, we do not have an acceptable estimation or the obtained feedback gains do not have an acceptable response. In the next examples, the simulation results are presented for a wide class of nonlinear systems with more complexity.Example 3.
To introduce the idea of feedback linearization, consider a nonlinear stochastic continuous system (11) with(38)x1˙t=x2t+wt,x2˙t=−αsinx1t+δ−sinδ−bx2t+ut+2wt,where wt is the zero-mean white noise with variance value 1. In this example, the goal is to force the system states to track the desired covariance matrix:(39)pdt=covx1t,x2t=0.2−0.1−0.10.3.
To remove the nonlinear term, we can choose a change of variables and state feedback control as(40)ut=αsinx1t+δ−sinδ+vt,zt=z1tz2t=x1tx2t,which yields(41)z˙t=A˜zt+B˜vt+D˜wt+v˜t,where A˜=010−b,B˜=01,D˜=12,α=1,δ=π/4,b=1, and v˜t is zero-mean additive covariance controller input. Thus, the stabilization problem for the nonlinear system has been reduced to a stabilization problem for a controllable linear system with a proper linear state feedback control law vt=−k1−k2zt=−3−2zt, where the poles of the closed-loop systems are assigned to the value −1.5+0.86j−1.5−0.86j. It is worth noting that the values of k1 and k2 are obtained by applying the pole-placement method stated in Chang et al. [25].
Assume(42)pcovt=varx1covx1,x2varx2T,Vcovt=varv1covv1,v2varv2T,r=0.2−0.10.3T,q˙t=r−pcovt.
By computingA¯cov, Bcov, and Ccov=I and considering that the poles of the closed-loop covariance systems were assigned to the value −2−2−2−3−3−3, the feedback gain is calculated and the covariance vector of additive input Vcovt will be computed as(43)Vcovt=−520−6000410−6000300−6pcovtqt.
The elements of vectorVcovt determine the covariance matrix Vt, where Vt=Ev˜tv˜tT is the covariance of additive controller input v˜t.
The performance of the proposed controller for this system is illustrated in Figures5–7.
Simulation results proposed that the controller can track the desired trajectory. As can be seen, the covariance and variance of the states are converged to the optimal covariance matrix value with a relatively good convergence rate. The mean value of CPU times for 100 runs (with sample time 0.05 sec) is also 11.56 sec.Figure 5
States variance and desired states variance.Figure 6
State trajectories of the system.Figure 7
Control signals trajectories.Example 4.
Consider the nonlinear stochastic system described by the following equations:(44)x1˙t=−2x1t+ax2t+sinx1t+2wt,x2˙t=−x2tcosx1t+utcos2x1t,where wt is zero-mean white noise with variance value 1. In this example, the goal is to force the system states to track the desired covariance matrix:(45)pdt=varx1t=0.3.
To remove the nonlinear term, we can choose the change of variables and state feedback control as(46)ut=1acosz1vt−cosz1sinz1+2z1cosz1,zt=z1tz2t=x1tax2t+sinx1,which yields(47)z˙t=A˜zt+B˜vt+D˜wt+v˜t,where A˜=−2100,B˜=01,D˜=20,a=1, and v˜t is zero-mean additivecovariance controller input.
The proper linear state feedback control law is assumed to be asvt=0−5zt, where the poles were assigned to the value −2−5 using the developed pole-placement approach in [25].
By assumingpcovt and Vcovt similar to Example 3 and q˙t=0.3−pcovt, computing A¯cov, Bcov, and Ccov=100, and supposing that the poles of the closed-loop covariance systems assign to the −10−1−2−9, the feedback gain is calculated and the covariance vector of additive input Vcovt is computed as(48)Vcovt=−820−2007100010pcovtqt.
Additionally, the elements of the vectorVcovt determine the covariance matrix Vt, where Vt=Ev˜tv˜tT is the covariance of additive controller input v˜t.
The performance of the proposed controller applied to this system is illustrated in Figures8 and 9.
Simulation results proposed that the controller can track the desired trajectory with a relatively good convergence rate. The mean value of CPU times for 100 runs (with sample time 0.05 sec) is 58.29 sec.Figure 8
State variance and state trajectories of the system.Figure 9
Control signal trajectories.


## 5. Conclusion

In this research, the covariance control algorithm for nonlinear systems using covariance feedback was investigated. At first, we showed that, under the condition that the linear approximation of a nonlinear system is a good representation of a nonlinear system, covariance control of nonlinear systems scenario involves the theory of covariance control based on the idea of the covariance feedback.Since in practice with the real condition, a linear approximation could not be used for any prespecified covariance matrix and numerous classes of nonlinear systems or we have poor linear approximation condition, we introduced the theory of covariance control of nonlinear systems based on the idea of feedback linearization. We showed that, with the feedback linearization idea and covariance feedback approach, we can be able to place the desired covariance matrix to a nonlinear system with almost high accuracy in steady state.


## 6. Future Recommendation

Because in some of practical applications, the actual structure of the system is unknown, nonlinear, or time varying, for a grope of time variant stochastic systems, we can introduce the adaptive nonlinear control theory for covariance control. Also, we can introduce presented covariance control theory for more complex systems such as underactuated systems with external disturbances [26] or chaotic systems with nonlinear functions under uncertainties and time-varying delays [27, 28]. Besides, the use of the idea of the prespecified covariance matrix placement is not limited to industrial or applied systems, and for example, the proposed system can produce various types of cyber attacks.



---

*Source: 9580243-2020-11-24.xml*