# Networked Optimal Control with Random Medium Access Protocol and Packet Dropouts

Authors: Chaoqun Zhu; Ge Guo; Bin Yang; Zhiwen Wang
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

This paper is concerned with an optimal control framework for networked control systems in which the channel access of actuators is governed by a group random access protocol. The system is modeled as a switching Markov jump system with multiple modes according to channel-access status of the actuators, and an independent identical distribution Bernoulli process is used to describe the random packet dropouts of the channel. Then, an optimal control design methodology is addressed to satisfy the quadratic cost function by applying the well-developed theory for jump linear systems and stochastic optimal control while guaranteeing the mean-square exponential stability of networked control systems. And, finally, a numerical example is exploited to demonstrate the effectiveness of the proposed method.


---

## Body

## 1. Introduction

Control systems whose components (e.g., sensors, actuators, and controllers) are connected via a real-time communication network are called networked control systems (NCSs). Using a control network has many advantages, such as reduced system wiring and cost, increased flexibility, reliability and safety, easier deployment, diagnosis, and maintenance. However, as any practical communication networks (e.g., wireless networks and underwater acoustic networks) have limited bandwidth, NCSs are subject to several issues that can lead to performance degradation and stability loss. Among these issues, network-induced delays [1–4], packet dropouts [5–8], data rate limitation, and quantization effects [9–11] have attracted the most attention in past decade. This paper is concerned with another interesting problem frequently encountered in NCSs, which is termedmedium access constraint. This issue arises when an NCS has a large number of nodes (sensors/actuators or subsystems) while the network has insufficient channels to accommodate them at a time. In such a situation, for a system to meet certain performance requirements, the procedure of controller design should be accompanied by a channel access assignment or scheduling policy for the nodes.Up to date, there have been generally three types of scheduling policies that are frequently used in addressing the problem of medium access constraint, that is, static scheduling, dynamic scheduling, and hybrid static-dynamic scheduling. Static scheduling policies are also calledstatic access protocols, which are designed offline. One of the earliest efforts on static access scheduling of NCSs can be seen in [12], where the concept of communication sequence was introduced to assign channel access to the nodes. Germinated by this work, a series of network access scheduling methods were presented from different viewpoints; see [13, 14], to name a few. Recently, research attention has been paid todynamic access protocols or dynamic scheduling policies; see, for example, [15, 16]. A network access protocol isdynamic (or online) when it is designed based on real-time feedback information and the system performance requirement. Generally speaking, static access protocols are quite simple and easily realizable, but they are not adaptable to situations with disturbances or uncertainties (e.g., parameter perturbation). As a result, the existing results on static scheduling, like those in [13, 14], can only handle NCSs whose dynamics are uncoupled. On the contrary, the dynamic access protocols in [15, 16], to mention some, can be used to handle systems with coupled dynamics robustly. However, such online medium access control protocols are either very conservative or computationally complicated. In addition, they require frequent monitoring of the channel for feedback information and complicated processing. One can take advantages of both static and dynamic strategies and derive a combined access control policy, which is called hybrid static-dynamic access protocol. A hybrid policy has been shown to be more promising for managing the constrained network medium access in [17, 18].The scheduling strategy presented in [18] is of special interest because it gives an optimal control solution to address the medium access constraint problem, aside from providing a special type of hybrid static-dynamic access protocol. The problem of integrated optimal control and scheduling was first investigated in [19], where a static scheduling policy was obtained by applying dynamic programming with pruning of the search tree to avoid combinatory explosion. Similar studies along this line can be seen in [20, 21]. For example, in [20], an optimal static scheduling strategy was derived by solving a complex combinatorial optimization problem with the neighbourhood search method and more efficient search heuristics. In [21], a novel optimal dynamic scheduling framework for NCSs was established on the basis of mixed integer programming and model predictive control techniques. Since network access scheduling and system control are strongly coupled, finding a jointly optimal design approach is indeed very complicated or even unsolvable [22], unless strong assumptions are made on either the plant to be controlled or the network to be scheduled. For this reason, it would be more practical for a control engineer to seek for optimal control solutions for specific medium access protocols that are given. In fact, even for a given medium access policy, the problem of designing an optimal controller is very hard, especially for random access protocols [23].Though stochastic access protocols are convenient in managing access to shared communication channels without feedback information, they are hardly seen in the literature for control purpose. Here we are interested in investigating optimal control of NCSs in which the network is governed by the random channel-access policy investigated by Wang and Guo [24]. We will consider systems whose actuators share a limited number of channels and the channels are governed by a group random access protocol. At each time, the so-calledgroup random access protocol allows a subset of active actuators to have access to the network and receive the control signal from the controller. At the next time, another subset of actuators is assigned access to the channels. The switching between two successive subsets of active actuators is determined by the random access protocol, which is defined by a vector-valued Markov process [25]. This paper is concerned with an optimal control framework for NCSs in which the channel access of actuators is controlled by a Markov group random access protocol. What makes the problem more complicated is that we consider packet-dropping issue at the same time. The random packet-loss of each network channel is described by an independent identical distribution (iid) Bernoulli process. The channel-access status of the actuators is determined by a group random access protocol, which is described by a Markov process with known transition probability matrix. By transforming the closed-loop system to a Markovian jump system with random packet loss, an optimal control design methodology is presented applying the well-developed theory for jump linear systems and stochastic optimal control.The organization of the paper is as follows. Section2 describes the NCS framework and models of access constraints and packet-loss process. The problem of designing an optimal state feedback law is solved in Section 3. Numerical example is presented in Section 4 to show the effectiveness of our results. Finally, Section 5 gives some concluding remarks and discusses future research directions.Remark 1.
It should be pointed out that the issue of control subject to packet dropout and medium access constraint was previously investigated in [26]. An assumption made in this paper is that multiple sensors or actuators in the system share a single unreliable channel that is manipulated by a static access protocol. Compared with [26], this paper considers a more complicated and challenging situation in which the actuators share multiple but insufficient channels according to a group random access protocol. In addition, we have further considered that packet dropouts could randomly occur during packet transmission in the channel.Notations. The following notations will be used throughout the paper. Rn and Rm×n denote the n-dimensional Euclidean space and the set of m×n real matrices, respectively. Cmp is the combinatorial number where p elements are selected from a total of m elements. P>0 (≥0) means that P is real symmetric and positively definite (semidefinite). XT and X-1 represent the transpose and the inverse of matrix X, and tr⁡X denotes the trace of the matrix X. λmax⁡X and λmin⁡X are maximal and minimal eigenvalue of the matrix X, respectively. Xii=1N denotes the set of the matrices X1,X2,…,XN. Ev stands for the mathematical expectation of the stochastic variable v, and Pr⁡· means the occurrence probability of the event “·”. Z denotes the set of nonnegative integers. Define the sets T=0,1,2,…,T-1 and N=1,2,…,N, where T and N are positive integer.


## 2. Problem Formulation

The NCSs considered in the paper are diagrammatically depicted in Figure1, where the plant is a linear time-invariant system described by
(1)xk+1=Axk+Buk+Gωk,
where x=[x1⋯xn]T∈Rn is the state vector, u=[u1,…,um]T∈Rm is the control vector actually executed by the actuators, ωk∈Rr denotes the white noise with zero means, which satisfies EωkωkT=Zk, and EωkωlT=0 for all l≠k. A,B, and G are constant matrices with appropriate dimensions, respectively.Figure 1
A basic NCS configuration.There arem actuators and n sensors in the NCS. Without loss of generality, we assume that the sensors are time-driven, and the controller and actuators are event-driven. To simplify analysis, we only consider the communication constraints at the actuators side but ignored the same for the sensors. It is assumed that the state of the plant is directly measured by the sensors and sent to the controller with full precision, while the control signals are transmitted in separate packets to the actuators via multiple communication channels. Due to bandwidth limitations, assume that there are m actuators in the system but only p (<m) channels so that the controller can only communicate with p actuators at any time k. Therefore, there are N=Cmp (N is a natural number) possible channel-access modes for the actuators. Furthermore, control packet dropouts can randomly occur during transmission due to node failures, network congestion, or communication noise.Fori=1,…,m, let the binary-valued function ρik denote channel-access status of the actuator i at time instant k. ρik=1 means that the actuator i is accessing the channel, and the corresponding control signal is transmitted; that is, uik=u-ik, where u-k denotes the control vector generated by the controller; otherwise, ρik=0 means that the control input packet is not transmitted and the actuator i will usezero as the control signal; that is, uik=0. Then, at time instant k, the medium access status of the m actuators is represented by them-to-p communication sequence ρk=ρ1k,ρ2k,…,ρmkT. Let Mρ(k)=diag⁡ρ(k) denote the matrix form of the communication sequence that assigns channel access to the m actuators at time instant k. We thus have
(2)u^k=Mρku-k.
It is assumed that Mρ(k) could be modeled by a Markov process taking values in a finite set Mρii=1N with the following conditional probability:
(3)Pr⁡Mρk+1=Mρj∣Mρ(k)=Mρi=πij,Pr⁡Mρ(k)=Mρi=πik,
where π=[πij] is the transition probability matrix and πij>0, i,j∈N, denotes the transition probability from mode i to mode j, satisfying ∑j=1Nπij=1. π0i is the initial probability of mode i, while πik is the probability of mode i at time instant k, and π(k)=π1kπ2k⋯πNkT obeys the recursion π(k+1)=πTπ(k). For simplicity, let θ(k) be the indicator of the mode of the Markov process, θ(k)=i, if Mρ(k)=Mρi.For the packet dropout issue, we use a binary-valued random variableσik, i=1,…,m, to denote the packet-loss phenomenon of the channels at time k. When the packet is successfully transmitted to the actuator, we have σik=1. If the packet is dropped during transmission, we have σik=0. Assume that any packet is lost according to a Bernoulli process with the following probability distribution:
(4)Pr⁡σi(k)=1=Eσi(k)=γ-i,Pr⁡σi(k)=0=1-Eσik=1-γ-i,
where γ-i∈0,1 is a known constant denoting the ith channel’s arrival probability of the control input packet. It is assumed that σi(k) and σi(l) are iid for k≠l and that different channels are also independent of each other as their expected values could be different; that is, γ-i≠γ-s, for i≠s. In consideration of the effect of packet loss, control signals actually received by the actuators can be expressed as follows:
(5)uk=Nσku^k,
where Nσ(k)=diag⁡σ1k,σ2k,…,σmk is the probability matrix that denotes the successful transmission of control signals.We assume that the control network uses aTCP-like protocol with a perfect acknowledgment mechanism. The actuators will send an acknowledgment signal to the controller at time instant k, to denote whether the control input packets were successfully delivered to their respective actuators at the previous time instant. It is noteworthy that the actuator acknowledgment has one-step time delay. Then, we define the corresponding information set as Ik=xt,θt,Nσt-1∣t=0,1,…,k. Apparently, Ik⊂Ik+1⊂IT.Combining the system dynamics (1), the activation status (2) of the actuators, and the effect of packet dropouts (5) and taking into account the fact that Mρθk and Nδk are both diagonal matrices, we can obtain
(6)xk+1=Axk+BθkNσku-k+Gωk,
where Bθk=BMρθk. It is clear that Bθk;k∈Z is also a Markov process.In the forthcoming development, we will need the following definition.Definition 2.
The NCS is said to be exponentially mean-square stable if, for every initial statex0,θ0, there exist constants α>0 and 0<β<1 such that
(7)Exk2∣x0,θ0≤αβkx02,∀k≥0.
The purpose of this paper is to derive an optimal state feedback sequenceu-0,u-1,…,u-T-1, for the presented networked system with the initial conditions x0,θ0, such that the system in (6) is exponentially mean-square stable, and minimize the quadratic cost function
(8)Jx0,θ0,u-=E∑k=0T-1xTkQkxk+u-TkNσkRkNσku-kxTTQTxT+∑k=0T-1xTkQkxk+u-TkNσkRkNσku-k∑k=0T-1xTkQkxk,
where QT≥0, Qk≥0, Rk>0, θ(k)∈N, k∈T. The minimal is denoted by J*x0,θ0.Remark 3.
There are generally two different strategies to deal with the situation that the actuator does not receive an expected control input packet at a given time. The first one is zero-input control strategy, which denotes the fact that plant’s scalar input receives zero values from the designated actuator as its control signal, and the second one is the last available control strategy, which means that the latest control values stored in the actuator buffer will be sent to the plant in this case. In this paper, we adopt the former strategy when actuators could not receive effective control information; see, for example, [15, 27, 28] and references therein.Remark 4.
From the above description, we know that the system in (6) is a discrete-time Markovian jump linear system with random packet dropouts described by an iid Bernoulli process. This system representation is convenient as it allows us to explore stability analysis and controller design of NCSs by utilizing the well-developed theory for jump linear system and stochastic optimal control.


## 3. Control Synthesis with Markov Actuator Assignment and Packet Dropouts

In this section, we first solve the optimal control problem in finite-time horizon for the system in (6) and derive the optimal control sequence to minimize the quadratic cost function in (8). Subsequently, we will prove that the optimal control sequence can also render the system in (6) exponentially mean-square stable. Define an admissible control input set as U=u-k;k∈T. By exploiting the independence of the noise input ωk, we can know that xk, ωk, u-k, and ωk are uncorrelated; then we have
(9)EωkxTk=0,Eωku-Tk=0.
From the definition of the mathematical expectation, for any measurable functions g and h, the following equation holds:
(10)Egωkhθk+1∣Ik=Egωk∣Ik∑j=1Nπθkjhj.Theorem 5.
Given the system in (6), with noise free measurement of the state vector xk, that is, ωk=0, the control law that minimizes the quadratic cost function in (8) is given by
(11)u-k=Kθkxk,
with the optimal cost function
(12)J*x0,θ0=∑i=1N∑k=0T-1πiktr⁡GZkGTP-ik+1tr⁡πi0x0Pi0xT0+∑k=0T-1πiktr⁡GZkGTP-ik+1,
where
(13)Kθkk=-∑I⊂ΩmpINIRk+BθkTP-θkk+1BθkNI-1×N-BθkTP-θkk+1A,(14)Pθkk=Qk+ATP-θkk+1A-ATP-θkk+1BθkN-×∑I⊂ΩmpINIRk+BθkTP-θkk+1BθkNI-1×N-BθkTP-θkk+1A,P-θkk+1=∑j=1NπθkjPjk+1>0,PθTT=QT,θk∈N,N-=diag⁡γ-1,…,γ-m,k=T-1,…,1,0.Proof.
Let us define the following optimal value functionVθk,xk,k:
(15)VθT,xT,T=ExTTQTxT∣IT,(16)Vθk,xk,k=min⁡u-kExTkQkxk+u-TkNσkRkNσku-k+Vθk+1,xk+1,k+1∣IkxT.
Obviously, J*x0,θ0=Vθ0,x0,0. We claim that the optimal value function can be written as follows:
(17)Vθk,xk,k=ExTkPθkkxk∣Ik+αk,iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiik=T,…,1,0,
where αT=0, αk=∑t=kT-1∑i=1Nπittr⁡GZkGTP-it+1, and k=T-1,…,1,0. We will prove the theorem by deducing that (15), (16), and (17) are equivalent. The mathematical induction is employed in the derivation process.
It is easy to verify that the claim in (17) is true for k=T with parameters PθTT=QT, θT∈N, and αT=0. We suppose that the claim in (17) is true for k+1:
(18)Vθk+1,xk+1,k+1=ExTk+1Pθk+1k+1xk+1∣Ik+1+αk+1.
The value function at time k is then
(19)Vθk,xk,k=min⁡u-kExTkQkxk+u-TkNσkRkNσku-k+Vθk+1,xk+1,k+1∣IkxTkQkxkExTkQkxk=min⁡u-kExTkQkxk+u-TkNσkRkNσku-k∣Ik+EExTk+1Pθk+1k+1xk+1∣Ik+1+αk+1∣IkxT.
Note that, the conditional expectations have the smoothing property [29] as follows:
(20)EVk+1∣Ik=EEVk+1∣Ik+1∣Ik.
Substituting (20) into (19) leads to
(21)Vθk,xk,k=min⁡u-kExTkQkxk+u-TkNσkRkNσku-k∣Ik+ExTk+1Pθk+1k+1xk+1∣Ik+αk+1xT.
Since medium access constraints and packet dropouts are two independent random processes,θk and Nσk are uncorrelated random variables. According to (6), we can obtain
(22)ExTk+1Pθk+1k+1xk+1∣Ik=EAxk+BθkNσku-k+GωkT×Pθk+1k+1×Axk+BθkNσku-k+Gωk∣IkAxk+BθkNσku-k+GωkT=ExTkATPθk+1k+1Axk+2u-TkNσkBθkTPθk+1k+1Axk+2xTkATPθk+1k+1Gωk+u-TkNσkBθkTPθk+1k+1×BθkNσku-k+2u-TNσkkBθkTkPθk+1k+1Gωk+ωTkGTPθk+1k+1Gωk∣Ik.
Taking the mathematical expectation on (22) and substituting (10) into (22) yield
(23)ExTk+1Pθk+1k+1xk+1∣Ik=xTkATP-θkk+1Axk+2u-TkN-BθkTP-θkk+1Axk+2tr⁡GEωkxTk∣IkATP-θkk+1+2N-tr⁡GEωku-Tk∣IkBθkTP-θkk+1+tr⁡GEωkωTk∣IkGTP-θkk+1+u-Tk∑I⊂ΩmpINIBθkTP-θkk+1BθkNIu-k,
where I is the set of channels for successful transmission data. It is easy to see that I is a subset of the index set Ωm={1,2,…,m} and has 2m possible values. pI=∏i∈Iγ-i∏i∉I1-γ-i denotes the probability of Nσ(k) taking values in NI, where NI is a diagonal matrix and its diagonal elements are
(24)NIii=1,ifi∈I0,ifi∉I.
It follows from (9) that
(25)Etr⁡GEωkxTk∣IkATP-θkk+1=0,Etr⁡GN-Eωku-Tk∣IkBθkTP-θkk+1=0,(26)Etr⁡GEωkωTk∣IkGTP-θkk+1=Etr⁡GEωkωTkGTP-θkk+1=∑i=1Nπiktr⁡GZkGTP-ik+1.
Substituting (25) and (26) into (23), we obtained
(27)ExTk+1Pθk+1k+1xk+1∣Ik=xTkATP-θkk+1Axk+2u-TkN-BθkTP-θkk+1BθkAxk+∑i=1Nπiktr⁡GZkGTP-ik+1+u-Tk∑I⊂ΩmpINIBθkTP-θkk+1BθkNIu-k.
Substituting (27) into (21), then one has(28)Vθk,xk,k=min⁡u-k∑I⊂ΩmpINIRk+BθkTP-θkk+1BθkNIxTkQkxk+xTkATP-θkk+1Axk+2u-TkN-BθkTP-θkk+1Axk+αk+1+∑i=1Nπiktr⁡GZkGTP-ik+1+u-Tk×∑I⊂ΩmpINIRk+BθkTP-θkk+1BθkNI×u-k∑I⊂ΩmpINIRk+BθkTP-θkk+1BθkNI.Since the control input u-k is unconstrained, let
(29)∂Vθk,xk,k∂u-k=0,
which yields
(30)2N-BθkTP-θkk+1Axk+2∑I⊂ΩmpINI+BθkTP-θkk+1BθkRk+BθkTP-θkk+1BθkNI∑I⊂ΩmpINIRku-k=0.
FromRk>0, we have that Rk+BθkTP-θkk+1Bθk>0, so we can obtain the optimal control sequence in (11). Noting that αk=∑i=1Nπiktr⁡GZkGTP-ik+1+αk+1 and substituting (11) into the optimal value function Vθk,xk,k, we obtained(31)Vθk,xk,k=E∑I⊂ΩmpINIRk+BθkTP-θkk+1BθkNI-1xTkQk+ATP-θkk+1A-ATP-θkk+1BθkTN-×∑I⊂ΩmpINIRk+BθkTP-θkk+1BθkNI-1×N-BθkP-θkk+1Axk∣Ik∑I⊂ΩmpINIRk+BθkTP-θkk+1BθkNI-1+αk.Letting
(32)Pθkk=Qk+ATP-θkk+1A-ATP-θkk+1BθkN-×∑I⊂ΩmpINIRk+BθkTP-θkk+1BθkNI-1×N-BθkTP-θkk+1A,
we can obtain
(33)Vθk,xk,k=ExTkPθkkxk∣Ik+αk.
From the last equation we can easily show that (17) is satisfied also at time instant k for all xk if and only if the matrices Pθkk satisfy (14). As a consequence, since J*x0,θ0=Vθ0,x0,0, it follows that the optimal value function is
(34)J*x0,θ0=ExT0Pθ00x0+α0=∑i=1N∑k=0T-1πiktrGZkGTP-ik+1tr⁡πi0x0Pi0xT0+∑k=0T-1πiktr⁡GZkGTP-ik+1.
The proof is thus completed.Remark 6.
Note that the problem under consideration is challenging since the channel access of actuators is controlled by a Markov group random access protocol. Furthermore, it is difficult to apply directly stochastic optimal control theory because of packet-dropping issue considered simultaneously. Even so, we still achieved a useful analysis and controller design technique in Theorem5 for NCSs under stochastic access protocol and packet-loss.
In the following, we will continue to prove that the optimal control sequence in (11) is not only minimizing the quadratic cost function in (8) but also rendering the system in (6) exponentially mean-square stable.Theorem 7.
If we ignore the effect of the white noise in system (6), that is, ωk=0, k∈T, then the optimal control sequence in (11) renders the system in (6) exponentially mean-square stable.Proof.
From (14), we can obtain Pθ(k)k>0, θ(k)∈N. Define the Lyapunov function Vk=xTkPθ(k)xk, which is positively definite when xk is not equivalent to zero. Then
(35)EVk+1∣Ik-Vk=ExTk+1Pθk+1k+1xk+1∣Ik-xTkPθkkxk=EAxk+BθkNσku-kT×Pθk+1k+1Axk+BθkNσku-k∣IkAxk+BθkNσku-kT-xTkPθkkxk=xTkATP-θkk+1Axk+2u-TkN-BθkTP-θkk+1BθkAxk+u-Tk∑I⊂ΩmpINIBθkTP-θkk+1BθkNIu-k-xTkPθkkxk=-xTkKθkT∑I⊂ΩmpINIRkNIKθk+Qk∑I⊂ΩmpINIRkNIxk.
Let
(36)Fθk=KθkT∑I⊂ΩmpINIRkNIKθk+Qk,
and hence
(37)EVk+1∣Ik=Vk-xTkFθkxk≤1-λmin⁡Fθkλmax⁡PθkVk<1-μσVk=βVk,
where 0<μ<λmin⁡Fθk, σ>λmax⁡Pθk. Apparently, μ<σ, 0<β<1. Similar to (37), we obtain
(38)EVk∣Ik-1<βVk-1.
Applying the conditional expectations to (38) and considering the smoothing property in (20), we obtained
(39)EVk∣Ik-2=EEVk∣Ik-1∣Ik-2<βEVk-1∣Ik-2<β2Vk-2.
Continuing this process, we finally obtained
(40)EVk<βkV0.
So
(41)ExTkPθ(k)kxk<βkx0TPθ00x0.
Then
(42)ExTkxk<βkλmin⁡Pθkkx0TPθ0x0≤λmax⁡Pθ00λmin⁡Pθkkβkx0Tx0=αβkx0Tx0,
where α=λmax⁡Pθ00λmin⁡Pθ(k)k-1. Therefore, the system in (6) is exponentially mean-square stable by Definition 2. This completes the proof.Remark 8.
If there is no packet dropout during data transmission from controller to actuators, the system in (6) can be rewritten as follows:
(43)xk+1=Axk+Bθku-k+Gωk.
Apparently, the system in (43) is a canonical Markovian jump linear system. We can derive the following corollaries immediately for NCSs with group random access protocol by using similar lines of proofs of Theorems 5 and 7.Corollary 9.
Given the system in (43), with noise free measurement of the state vector xk, that is, ωk=0, the optimal control sequence that minimizes the quadratic cost function,
(44)Jx0,θ0,u-=E∑k=0T-1xTkQkxk+u-TkRku-kxTTQTxT+∑k=0T-1xTkQkxk+u-TkRku-k,
is given by
(45)u-k=Kθkxk,
with the optimal cost function
(46)J*x0,θ0=∑i=1N∑k=0T-1πiktrGZkGTP-ik+1tr⁡πi0x0Pi0xT0+∑k=0T-1πiktr⁡GZkGTP-ik+1,
where
(47)Kθkk=-Rk+BθkTP-θkk+1Bθk-1×BθkTP-θkk+1A,(48)Pθkk=Qk+ATP-θkk+1A-ATP-θkk+1Bθk×Rk+BθkTP-θkk+1Bθk-1×BθkTP-θkk+1A,Pθk=PθkT>0,(49)P-θkk+1=∑j=1NπθkjPjk+1,PθTT=QT,θk∈N,k=T-1,…,1,0.Corollary 10.
If we ignore the effect of the white noise, that is,ωk=0, k∈T, then the optimal control sequence in (45) renders the system in (43) exponentially mean-square stable.


## 4. Numerical Example

To illustrate the effectiveness of the presented method, we give a numerical example. Consider a discrete-time plant in the form of (1) with the following parameters:
(50)A=0.1600-1.2005-1.1042-2.3890,B=-1.53501.8918-1.2902-1.6869,G=0.20000.00000.00000.2000.The eigenvalues ofA are 0.6030 and -2.8320; thus the above system is open-loop and unstable. Our objective is to derive an optimal state feedback controller to render the system in (6) exponentially mean-square stable and minimize the quadratic cost function in (8). Assume that packet arrival probability of channels 1 and 2 is γ-1=0.76 and γ-2=0.70, respectively. We consider the worst-case scenario, where p=1. Then, actuator assignment sequences matrices are
(51)Mρ1=diag⁡1,0,Mρ2=diag⁡0,1.
In the simulation, we use the following transition matrix for the Markov chains:
(52)π=0.69750.30250.31500.6850.We formulate the optimal control problem with the state weighting matrixQk=I4, the state final-value weighting matrix QT=I4, and the control weighting matrix Rk=I4. We take the initial state as x0=46-32T and set time horizon as T=50 in the simulation. According to Theorem 5 we obtain the optimal control sequence that minimizes the quadratic cost function in (8). The actuator assignment sequence is shown in Figure 2, where “1” and “2” in the y-axis denote the working modes. The data packet dropouts of channels 1 and 2 are shown in Figures 3 and 4, where “1” and “0” in the y-axis denote cases of the packet arrival and packet loss, respectively. The simulation result of the state trajectories is depicted in Figure 5, which has verified that our goal is achieved. Besides, we can clearly observe from Figure 5 that there are prominent variations in the state curves around the equilibrium state in the control stage, which means that the access constraints and packet dropouts have a larger effect on the dynamic performance of the networked systems.Figure 2
Activation mode sequences of actuators.Figure 3
Channel 1 of data packet dropouts.Figure 4
Channel 2 of data packet dropouts.Figure 5
State trajectories.


## 5. Conclusions

This paper has investigated finite-time optimal controller design problem for networked control systems with network access constraints and packet dropouts. Since the presence of limited bandwidth network, there are limitations on the number of active actuators and packet dropouts between the controller and the actuators of NCSs. The channel-access status of the actuators is governed by a group random access protocol modeled as a Markov process, and packet dropouts are described by an iid Bernoulli process whose acknowledgement is always available to the controller. For such a system, an optimal control design methodology is addressed to satisfy the quadratic cost function by applying the well-developed theory for jump linear systems and stochastic optimal control while guaranteeing the mean-square exponential stability of networked control systems. Simulation results have demonstrated the feasibility and effectiveness of the addressed control scheme.We assume that full-state information is available for measurements in this paper. In many cases, this assumption does not hold. Under such circumstances, we can perform a state estimation and then design an optimal state feedback controller. Furthermore, many systems may be subject to access constraints and pocket-loss at both sides of the communication network. Therefore, it is of great interest to seek for an effective controller design framework by taking into simultaneous consideration the constraints at both sides of the network.



---

*Source: 105416-2015-02-23.xml*