# A Time-Delay-Bounded Data Scheduling Algorithm for Delay Reduction in Distributed Networked Control Systems

Authors: Zeming Fan; Xiaojun Yu; Hao Wan; Meilin Kang; Yuan Liu; Yuye He; Gaoxi Xiao
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

As a key feature of networked control systems (NCSs), the time delays induced by communication medium sharing and data exchange over the system components could largely degrade the NCS performances or may even cause system instability, and thus, it is of critical importance to reduce time delays within NCSs. This paper studies the time-delay reduction problem in distributed NCSs and presents a dual-way data scheduling mechanism for time-delay reductions in delay-bounded NCSs with time-varying delays. We assess the time delays and their influences on the NCSs first with various delay factors being considered and then describe a one-way scheduling mechanism for network-delay reductions in NCSs. Based upon such a method, a dual-way scheduling algorithm is finally proposed for distributed NCSs with different types of transmitted data packets. Experiments are conducted on a remote teaching platform to verify the effectiveness of the proposed dual-way scheduling mechanism. Results demonstrate that, with the stability time-delay bound considered within the scheduling process, the proposed mechanism is effective for NCS time-delay reductions while addressing the stability, control accuracy, and settling time issues efficiently. Such a proposed mechanism could also be implemented together with some other existing control algorithms for time-delay reductions in NCSs. Our work could provide both useful theoretical guidance and application references for stable tracking control of delay-bounded NCSs.


---

## Body

## 1. Introduction

With the rapid advancements of networking technologies over the past decades, there is a growing trend in both industrial and commercial communities to integrate computing, communication, and control systems together to formulate a unified platform via network remote control. In such integrated platforms, the control commands from different information sources are transmitted and exchanged over networks, while their feedback control systems, whose control loops are formed via real-time communication channels, are called networked control systems (NCSs) [1]. Owing to its advantageous properties, e.g., low cost, high reliability, easy reconfiguration, satisfactory flexibility, robustness, and adaptation capabilities, NCS has attracted extensive research interest in recent years and been utilized in various fields, such as the power grids, transportation networks, water distribution networks, telephone networks, global financial networks, and genetic expression networks [2].In practical NCSs, there exist numerous information sources and nodes exchanging data over the networks simultaneously, and thus, the time delays induced by network resource sharing would generate inevitably [3–5]. Such time delays could largely degrade the system performances or sometimes may even cause the system to be instable. Furthermore, as the time delays within NCSs could either be constant, or bounded, or random, they also make the NCS system design and analysis complicated. Therefore, it is of critical importance to reduce the NCS time delays in engineering practice [5, 6].Various mechanisms have been proposed for NCS time-delay reductions in the literature [7, 8]. Based on the different time-delay models being adopted, such mechanisms could be divided into two main categories, namely, the NCS network control-based ones and the NCS system data scheduling-based ones [7]. Specifically, the NCS network control-based mechanisms treat the whole NCS system as a controlled network, and the main focus is on control of the networks, i.e., the main object is to propose efficient methods to address the raised networking issues, such as routing, congestion control, networking protocol, and data communications [6–8]. The NCS network control-based mechanisms have attracted extensive research interest over the past years, and readers could refer to [6–8] for comprehensive details.The data scheduling-based mechanisms consider NCSs to be control systems and try to optimize performances of the overall NCSs [9, 10]. Initially, the data scheduling-based mechanisms assume that the NCSs have fixed data sampling period and propose methods to optimize resource allocations for competing demands [10]. Scheduling algorithms such as rate monotony (RM) scheduling, earliest deadline first (EDF) dynamic scheduling, and dead-bands scheduling, are of this category. Although such methods help reduce time delays, the practical system variables, such as varying transmission delays, transmission variables, packet loss, and communication constraints, have been ignored, and thus, these methods may not be adaptive to real-time network changes during the system operation process [11]. To further overcome these problems, scheduling mechanisms with variable sampling periods, such as dynamic feedback scheduling, fuzzy logic control or neural network-based scheduling, active sampling period scheduling, and delay compensation-based scheduling, have also been proposed [11–14]. Those methods are flexible in handling the time-varying traffic, yet some other constraints, such as the unknown network loads prior to scheduling, the lacks of upper and lower bounds for sampling period, and the difficulties in determining the key control parameters, have been largely ignored [14, 15].Event-triggering feedback control mechanism is another kind of scheduling scheme being proposed to address the above network constraints and has attracted extensive research interest in recent years [16–18]. Specifically, by adopting a subsystem to broadcast the local state information to its neighbors, the distributed event-triggering feedback schemes were proposed for linear and nonlinear systems first [16, 17], and later, such algorithms were extended with various practical constraints, e.g., the data drops, transmission delays, probabilistic nonlinearities, sensor/actuator faults, and external attacks/disturbances, being taken into account, and have also achieved satisfactory results [18–21]. Currently, some other algorithms with more practical constraints taken into account are attracting increasing research interest. For more details of recent advances on event-triggering NCS control and the trends and techniques for NCS delay reductions, readers could refer to [6, 22]. It is worth mentioning that most of those existing mechanisms have ignored the influences of the upper and lower time-delay bounds on the performances of the system stability and the data scheduling schemes.In practice, however, the permissible system time-delay bounds play a critical role in determining the system stability performances for NCSs. This is because, on the one hand, there usually exists a huge amount of information being exchanged over the networks, and such information would inevitably induce time delays, while on the other hand, such information is typically of different types, and each type of information may have its own transmitting priorities within the NCS. For such information being transmitted, once the transmission time exceeds the system permissible time-delay bounds, the system may become unstable or even divergent [23]. The case is especially true for distributed NCSs, since in decentralized systems, some of the plants are first-ordered, while the others may be higher-ordered with much stricter time-delay bound limitations. Therefore, to avoid causing system instability, scheduling in NCSs should be performed within the system time-delay bounds.It is also worth noting that most of the existing scheduling mechanisms are one-way designed, making the scheduling operations inefficient for NCS delay reductions. For instance, for distributed NCSs as shown in Figure1, those existing scheduling mechanisms typically operate either on the field sensor scheduler or on the network remote controller scheduler only. In practice, however, if the scheduling operations are conducted on the sensor scheduler and controller schedulers simultaneously, i.e., dual-way scheduling schemes are devised and adopted, the NCS time-delay reductions could be much more efficient. This is because, in such a case, both sensor terminals and remote controller could transmit their control demands simultaneously within the NCS for processing, and meanwhile, the sensor terminals and remote controllers could work cooperatively and more efficiently with the control variables being transmitted within the NCS control loops.Figure 1
Structure of a typical NCS with time delays.This paper studies the time-delay reduction problem within distributed NCSs and proposes a delay-bounded data scheduling mechanism for time-delay reductions in NCSs. Specifically, with the various delay factors being considered, the time delays and their influences on the NCSs are evaluated first, and then, a one-way scheduling algorithm is presented for data scheduling within NCSs. Based on such a one-way scheduling scheme, a novel dual-way dynamic scheduling algorithm, which is performed on controller and sensor schedulers simultaneously, is finally described. Both simulations and experiments are conducted to verify the effectiveness of the proposed data scheduling mechanism. Results obtained from experiments carried out on a practical remote teaching platform show that, with the scheduling operations conducted within the lower and upper NCS delay bounds, the proposed algorithm could help significantly improve the performances of the distributed NCSs while the system stability could also be guaranteed in different cases.


## 2. Time Delays and Their Impacts on Distributed NCSs

### 2.1. Time Delays within NCSs

A typical NCS comprised several control plants, sensors, controllers, and actuators shown in Figure2. In such an NCS, there are numerous information sources sharing the network resources and transmitting data onto the network via communication channels. Once too much information are transmitted simultaneously via the same channel, the common network transmission issues, e.g., congestions, packet collisions, multipath transmissions, and link interruptions, may arise and thus would cause time delays inevitably [3, 4].Figure 2
Topological structure of network withn controllers.The time delays within NCSs could be categorized into control-induced delays and network-induced delays. While the former is the time consumed by the sensors, plants, and actuators to complete their respective noncommunication functions, the latter is induced by data transmissions within the NCS. For NCSs with feedback and forward channels as shown in Figure1, we further divide the time delays in the system kth control cycle into five parts as follows:(1)
Data preprocessing delay, i.e., the time required by sensors to pack data for transmission, and it is denoted byTi1prcK and Ti2prck for the feedback and forward channels, respectively.(2)
Data packet queuing delay, i.e., the time taken by a data packet to wait for its transmission, and it is denoted byTi1waitk and Ti2waitk for the feedback and forward channels, respectively.(3)
Data transmission delay, i.e., the time taken by a data packet to be transmitted within the system, which is determined by the packet length, network bandwidth, and transmission distance. Such a delay is denoted byTi1trak and Ti2trak for the feedback and forward channels, respectively.(4)
Data postprocessing delay, i.e., the time required for a controller to receive and store a data packet, and it is denoted byTi1posk and Ti2posk for the feedback and forward channels, respectively.(5)
Controller calculation delay is denoted byTick.Denote the overall time delay for the feedback and forward channels to beTi1k and Ti2k; then, we can have(1)Ti1k=Ti1prck+Ti1waitk+Ti1tra+Ti1posk,(2)Ti2k=Ti2prck+Ti2waitk+Ti2tra+Ti2posk,while the total time delay within the NCS is(3)Tik=Ti1k+Ti2k+Tick,which is also the control system round trip time (RTT), i.e., the interval between the time a plant receives its kth control command to the time when it receives its k+1th command.In practice,Tick is determined by computer performances. Specifically, since such a delay is much smaller as compared with the others and could be compensated by algorithms [24], the influences of Tick are neglected and only those of Ti1k and Ti2k are evaluated in this paper.


### 2.2. Influences of Time Delays on NCS Stability

Assume that controllers in the NCS as shown in Figure2 are event-driven, while those sensors and actuators are time-driven, then the theoretical discrete state equations for the ith plant are(4)xik+1=Aixik+Biuik,(5)uik=Kixik,where xik∈Rn is the state variable of the ith plant in its kth control cycle; Ai and Bi are constant matrices, while uik and Ki are control input and feedback gain of the ith control plant, respectively.Further assume that the network-induced delays are time-varying and bounded, satisfying [25, 26](6)0<≤Tik≤TiM,where TiM is a constant, denoting the upper bound of the time-varying delay Tik, and it could be characterized by the summation inequality presented in Theorem 5 in [27].With the network-induced delays being taken into account, we haveuik=Kixik−Tik from (5), while the closed-loop NCS by (4) and (5) could be described as follows:(7)xik+1=Aixik+BiKixik−Tik=Aixik+Aidxik−Tik,yik=cixik,where Aid=BiKi is a loose variable introduced by Lyapunov’s function to characterize the NCS. The schematic diagram of such a control system could be illustrated by Figure 3, wherein the two networks could either be the same or different depending on the practical implementation of the NCSs.Figure 3
Schematic diagram of the control system evaluated.


## 2.1. Time Delays within NCSs

A typical NCS comprised several control plants, sensors, controllers, and actuators shown in Figure2. In such an NCS, there are numerous information sources sharing the network resources and transmitting data onto the network via communication channels. Once too much information are transmitted simultaneously via the same channel, the common network transmission issues, e.g., congestions, packet collisions, multipath transmissions, and link interruptions, may arise and thus would cause time delays inevitably [3, 4].Figure 2
Topological structure of network withn controllers.The time delays within NCSs could be categorized into control-induced delays and network-induced delays. While the former is the time consumed by the sensors, plants, and actuators to complete their respective noncommunication functions, the latter is induced by data transmissions within the NCS. For NCSs with feedback and forward channels as shown in Figure1, we further divide the time delays in the system kth control cycle into five parts as follows:(1)
Data preprocessing delay, i.e., the time required by sensors to pack data for transmission, and it is denoted byTi1prcK and Ti2prck for the feedback and forward channels, respectively.(2)
Data packet queuing delay, i.e., the time taken by a data packet to wait for its transmission, and it is denoted byTi1waitk and Ti2waitk for the feedback and forward channels, respectively.(3)
Data transmission delay, i.e., the time taken by a data packet to be transmitted within the system, which is determined by the packet length, network bandwidth, and transmission distance. Such a delay is denoted byTi1trak and Ti2trak for the feedback and forward channels, respectively.(4)
Data postprocessing delay, i.e., the time required for a controller to receive and store a data packet, and it is denoted byTi1posk and Ti2posk for the feedback and forward channels, respectively.(5)
Controller calculation delay is denoted byTick.Denote the overall time delay for the feedback and forward channels to beTi1k and Ti2k; then, we can have(1)Ti1k=Ti1prck+Ti1waitk+Ti1tra+Ti1posk,(2)Ti2k=Ti2prck+Ti2waitk+Ti2tra+Ti2posk,while the total time delay within the NCS is(3)Tik=Ti1k+Ti2k+Tick,which is also the control system round trip time (RTT), i.e., the interval between the time a plant receives its kth control command to the time when it receives its k+1th command.In practice,Tick is determined by computer performances. Specifically, since such a delay is much smaller as compared with the others and could be compensated by algorithms [24], the influences of Tick are neglected and only those of Ti1k and Ti2k are evaluated in this paper.


## 2.2. Influences of Time Delays on NCS Stability

Assume that controllers in the NCS as shown in Figure2 are event-driven, while those sensors and actuators are time-driven, then the theoretical discrete state equations for the ith plant are(4)xik+1=Aixik+Biuik,(5)uik=Kixik,where xik∈Rn is the state variable of the ith plant in its kth control cycle; Ai and Bi are constant matrices, while uik and Ki are control input and feedback gain of the ith control plant, respectively.Further assume that the network-induced delays are time-varying and bounded, satisfying [25, 26](6)0<≤Tik≤TiM,where TiM is a constant, denoting the upper bound of the time-varying delay Tik, and it could be characterized by the summation inequality presented in Theorem 5 in [27].With the network-induced delays being taken into account, we haveuik=Kixik−Tik from (5), while the closed-loop NCS by (4) and (5) could be described as follows:(7)xik+1=Aixik+BiKixik−Tik=Aixik+Aidxik−Tik,yik=cixik,where Aid=BiKi is a loose variable introduced by Lyapunov’s function to characterize the NCS. The schematic diagram of such a control system could be illustrated by Figure 3, wherein the two networks could either be the same or different depending on the practical implementation of the NCSs.Figure 3
Schematic diagram of the control system evaluated.


## 3. Strategies for Network-Delay Reductions

As shown in (1), the time delay Ti1k in NCS is caused mainly by the data processing delay Ti1prck, queuing delay Ti1waitk, transmission delay Ti1trak, and the postprocessing delay Ti1posk. Among all those factors, since Ti1trak is determined by the network conditions, while Ti1prck and Ti1posk are so short that could be negligible [28], Ti1waitk is regarded to be the main delay factor. This is similar for Ti2k in the forward channels. However, since the data transmitted in NCS system include video/audio information, control commands, sensing data, and control data of all NCS components, which are either periodic or nonperiodic and to be transmitted either in real time or non-real time over the channels, the NCS time delays could be very large. In such a case, the transmission of all control data or system outputs within a single data packet is impractical. Moreover, since the queuing delays of both forward and backward channels could be manipulated by scheduling of the sensors and controllers, it is expected that appropriate scheduling algorithms could be devised to minimize the system time delays.In this paper, we propose a two-way scheduling mechanism for an NCS, wherein scheduling operations are performed on both sensor and controller schedulers. Specifically, once there exist any data collisions, the scheduling mechanism is adopted to assign the data packets to each node different priorities, such that they could be transmitted with the shortest average delay.


### 3.1. Scheduling Preprocessing

To describe the scheduling operations for a plant within a control loop more clearly, we categorize the scheduling operations into data preprocessing on sensor schedulers, data postprocessing on controller schedulers, data queuing on controller schedulers, and data queuing on sensor schedulers. Hence, the time delays for each plant are mainly introduced by these four tasks and could be managed by a scheduling operation. Specifically, for any planti, a transmission description function τin could be established as shown in (8), which could be utilized to calculate the desired scheduling time slot for each data packet.(8)τin=fτinm,τina,τinr,τinf,τine,τinb,where n=1,2,3,4 denoting the four data packet scheduling operations. Hence, the time delays as shown in Figure 1 could also be denoted as Ti1wait=τi1,Ti1pos=τi2,Ti2pos=τi3, and Ti2wait=τi4. τinm describes the significance level of each data packet and is defined to be the reciprocal of the packet time-delay upper bound. τina is the actual packet transmission starting time, which is also utilized to order the scheduling operations for packets with the same significance level. τinr is the longest execution time among all data packets, which is determined by the data packet length, network bandwidth, and transmission distances. τinf and τine are the earliest and latest transmission starting time of a data packet, respectively. τinb is the control cycle of the NCS.Once the system delay boundsTi1M, Ti2Mand the packet transmission description function τin are determined, the scheduling operations then could be decided for each packet in the NCS. Specifically, for a packet to be transmitted, its scheduling operation includes two main steps, i.e., packet significance level determination and scheduling operation determination for all packets with the same significance level. In this study, we call the former as scheduling preprocessing and the latter as packet scheduling operation. The determinations of packet transmission property variables are discussed in the following sections.


### 3.2. Scheduling Mechanism for Sensor Scheduler

#### 3.2.1. Scheduling of Coupled Information

In NCSs with coupled information, data packets are transmitted in predefined order, i.e., for a plant within a control loop, its control information should be sent to the actuator before it collects data from a sensor, while for a controller, it has to receive sensor data first, and then sends the control information to the local controller via network after executing the control mechanism. In this way, the real-time data transmission within NCSs could be guaranteed. To perform scheduling for packets with the same significance level, a data transmission setτs has to be established as below according to the packet transmission time, while each packet should be assigned a time slot, within which the sensor scheduler or controller scheduler is fully occupied by the scheduling task specified:(9)τs=τin⟶τjnτin,τjn,∈τ,where τ represents all data packets to be transmitted, and τin⟶τjn defines the relative priority of two data packets within τ, indicating that τjn can be transmitted only when the transmission of τin is completed.However, since the processing of a packet may last for a period of time, the variablesτinr,τinf,τine have to be calculated. For an existing data transmission set with u packets τink⟶τjnk=1,…,u to be transmitted, the earliest transmission starting time τinf′k among those packets can be calculated as follows:(10)τinf′k=τinak+τinr,where τinr=Ti1tra. The earliest transmission starting time τinf among all packets within the set could be calculated:(11)τinf=maxτinf′1,…,τinf′u.Similarly, if there exists a transmission set withv packets following τin⟶τinkk=1,…,v, then the latest transmission starting time τine′k for those data packets can be calculated as follows:(12)τine′k=minτinek−τinr,Ti1M−τinr,and the latest transmission starting time τine for packets within the transmission set could be determined by (13)τine=minτine′k,τinf′k.


#### 3.2.2. Scheduling of Noncoupled Information

In NCSs, there also exists noncoupled information with the packets being transmitted in an arbitrary order. To determine the scheduling operations for such packets, the packet transmission starting timeτina has to be calculated. In practice, however, since the NCS time delay typically consists of the forward channel delay, feedback channel delay, and controller processing delay, while the forward and feedback channels are symmetric, it is reasonable to assume that half of the overall delay comes from the forward channel while the other half is from the feedback channel. Hence, in the scheduling process, half of the delay TiM is assigned to the forward channel scheduler, and the other half is assigned to the feedback channel scheduler. In such a way, it is expected that the time slots reserved for the controller scheduler are enough for its scheduling operations. Hence, the longest allowable queuing delay Ti1m for those data packets could be determined as follows:(14)Ti1m=12TiM−Ti1trak−1−Ti1prck−1−Ti1posk−1,and the latest starting time would be(15)τine=Ti1m.Moreover, since the noncoupled packets could be transmitted in an arbitrary order, the earliest transmission starting time of a packet could be set asτinf=0, while its execution time could be τinr=Ti1tra.


#### 3.2.3. Scheduling for Hybrid Coupled and Noncoupled Information

In NCSs with both coupled and noncoupled information, assume that there are information coupled packets withu following τink⟶τink=1,…,u and v following τin⟶τinhh=1,…,v, which are determined by (11) and (13), respectively, as well as w information noncoupled packets. If there exists a parameter jj∈u−2,u+w+2 for the scheduled packets τinj satisfying(16)τinf≥τinaj+τinrj,τinaj+1≥τinf+τinr,or(17)τinf≤τinaj+τinrj≤τine,τinaj+1≥τinaj+τinrj+τinr,then we have the actual execution time for nth task of the ith plant as follows:(18)τina=maxτinf,τinaj+τinrj.Hence, the time slot utilized for processing the packet within the control loop could be calculated as follows:(19)τina+τinr.The scheduling for any data packet to be transmitted within NCS could be conducted as shown in Algorithm1.Algorithm 1: Scheduling mechanism in feedback channel for distributed NCSs.
Input: an existing transmission set τink⟶τjnk=1,…,u; a new data packet with τin.Output: a new transmission set τink⟶τjnk=1,…,u+1.(1)
Calculate the time-delay upper boundTiM with Theorem ?? for the data packet;(2)
Calculate transmission priorityτinm with TiM, i.e., τinm∝1/TiM;(3)
Compare calculatedτinm to τinmk of τinkk=1,…,u in the set;(4)
if calculated τinm=τinmk for 1≤k≤uthen(5)
Insertτin into the end of set τink⟶τjnk=1,…,u, go to step 10;(6)
else(7)
Calculate information execution timeτia using equations (10)–(18);(8)
Insertτin into set τink⟶τjnk=1,…,u according to τina in an increasing order;(9)
end if(10)
Calculate the actual packet execution timeτina using equation (18);(11)
Scheduling operation according to the order of transmission setτink⟶τjnk=1,…,u+1;


### 3.3. Scheduling Mechanism for Controller Scheduler

To determine the scheduling operations for each packet, the execution time slot should be calculated for each data packet, and thus, the transmission timeτif,τie, and τia should be calculated using (11) and (13) and (16)–(18), respectively. Finally, the transmission set τs could be determined. The same scheduling operations shown in Algorithm 1 are executed within the controller scheduler for each packet of the plant.While for any plant adopting an NCS structure as shown in Figure1, the scheduling and transmission time slot distribution for each of its data packets could be illustrated in Figure 4.Figure 4
The scheduling and execution time distribution for each packet of the plant adopting an NCS structure as shown in Figure1.


## 3.1. Scheduling Preprocessing

To describe the scheduling operations for a plant within a control loop more clearly, we categorize the scheduling operations into data preprocessing on sensor schedulers, data postprocessing on controller schedulers, data queuing on controller schedulers, and data queuing on sensor schedulers. Hence, the time delays for each plant are mainly introduced by these four tasks and could be managed by a scheduling operation. Specifically, for any planti, a transmission description function τin could be established as shown in (8), which could be utilized to calculate the desired scheduling time slot for each data packet.(8)τin=fτinm,τina,τinr,τinf,τine,τinb,where n=1,2,3,4 denoting the four data packet scheduling operations. Hence, the time delays as shown in Figure 1 could also be denoted as Ti1wait=τi1,Ti1pos=τi2,Ti2pos=τi3, and Ti2wait=τi4. τinm describes the significance level of each data packet and is defined to be the reciprocal of the packet time-delay upper bound. τina is the actual packet transmission starting time, which is also utilized to order the scheduling operations for packets with the same significance level. τinr is the longest execution time among all data packets, which is determined by the data packet length, network bandwidth, and transmission distances. τinf and τine are the earliest and latest transmission starting time of a data packet, respectively. τinb is the control cycle of the NCS.Once the system delay boundsTi1M, Ti2Mand the packet transmission description function τin are determined, the scheduling operations then could be decided for each packet in the NCS. Specifically, for a packet to be transmitted, its scheduling operation includes two main steps, i.e., packet significance level determination and scheduling operation determination for all packets with the same significance level. In this study, we call the former as scheduling preprocessing and the latter as packet scheduling operation. The determinations of packet transmission property variables are discussed in the following sections.


## 3.2. Scheduling Mechanism for Sensor Scheduler

### 3.2.1. Scheduling of Coupled Information

In NCSs with coupled information, data packets are transmitted in predefined order, i.e., for a plant within a control loop, its control information should be sent to the actuator before it collects data from a sensor, while for a controller, it has to receive sensor data first, and then sends the control information to the local controller via network after executing the control mechanism. In this way, the real-time data transmission within NCSs could be guaranteed. To perform scheduling for packets with the same significance level, a data transmission setτs has to be established as below according to the packet transmission time, while each packet should be assigned a time slot, within which the sensor scheduler or controller scheduler is fully occupied by the scheduling task specified:(9)τs=τin⟶τjnτin,τjn,∈τ,where τ represents all data packets to be transmitted, and τin⟶τjn defines the relative priority of two data packets within τ, indicating that τjn can be transmitted only when the transmission of τin is completed.However, since the processing of a packet may last for a period of time, the variablesτinr,τinf,τine have to be calculated. For an existing data transmission set with u packets τink⟶τjnk=1,…,u to be transmitted, the earliest transmission starting time τinf′k among those packets can be calculated as follows:(10)τinf′k=τinak+τinr,where τinr=Ti1tra. The earliest transmission starting time τinf among all packets within the set could be calculated:(11)τinf=maxτinf′1,…,τinf′u.Similarly, if there exists a transmission set withv packets following τin⟶τinkk=1,…,v, then the latest transmission starting time τine′k for those data packets can be calculated as follows:(12)τine′k=minτinek−τinr,Ti1M−τinr,and the latest transmission starting time τine for packets within the transmission set could be determined by (13)τine=minτine′k,τinf′k.


### 3.2.2. Scheduling of Noncoupled Information

In NCSs, there also exists noncoupled information with the packets being transmitted in an arbitrary order. To determine the scheduling operations for such packets, the packet transmission starting timeτina has to be calculated. In practice, however, since the NCS time delay typically consists of the forward channel delay, feedback channel delay, and controller processing delay, while the forward and feedback channels are symmetric, it is reasonable to assume that half of the overall delay comes from the forward channel while the other half is from the feedback channel. Hence, in the scheduling process, half of the delay TiM is assigned to the forward channel scheduler, and the other half is assigned to the feedback channel scheduler. In such a way, it is expected that the time slots reserved for the controller scheduler are enough for its scheduling operations. Hence, the longest allowable queuing delay Ti1m for those data packets could be determined as follows:(14)Ti1m=12TiM−Ti1trak−1−Ti1prck−1−Ti1posk−1,and the latest starting time would be(15)τine=Ti1m.Moreover, since the noncoupled packets could be transmitted in an arbitrary order, the earliest transmission starting time of a packet could be set asτinf=0, while its execution time could be τinr=Ti1tra.


### 3.2.3. Scheduling for Hybrid Coupled and Noncoupled Information

In NCSs with both coupled and noncoupled information, assume that there are information coupled packets withu following τink⟶τink=1,…,u and v following τin⟶τinhh=1,…,v, which are determined by (11) and (13), respectively, as well as w information noncoupled packets. If there exists a parameter jj∈u−2,u+w+2 for the scheduled packets τinj satisfying(16)τinf≥τinaj+τinrj,τinaj+1≥τinf+τinr,or(17)τinf≤τinaj+τinrj≤τine,τinaj+1≥τinaj+τinrj+τinr,then we have the actual execution time for nth task of the ith plant as follows:(18)τina=maxτinf,τinaj+τinrj.Hence, the time slot utilized for processing the packet within the control loop could be calculated as follows:(19)τina+τinr.The scheduling for any data packet to be transmitted within NCS could be conducted as shown in Algorithm1.Algorithm 1: Scheduling mechanism in feedback channel for distributed NCSs.
Input: an existing transmission set τink⟶τjnk=1,…,u; a new data packet with τin.Output: a new transmission set τink⟶τjnk=1,…,u+1.(1)
Calculate the time-delay upper boundTiM with Theorem ?? for the data packet;(2)
Calculate transmission priorityτinm with TiM, i.e., τinm∝1/TiM;(3)
Compare calculatedτinm to τinmk of τinkk=1,…,u in the set;(4)
if calculated τinm=τinmk for 1≤k≤uthen(5)
Insertτin into the end of set τink⟶τjnk=1,…,u, go to step 10;(6)
else(7)
Calculate information execution timeτia using equations (10)–(18);(8)
Insertτin into set τink⟶τjnk=1,…,u according to τina in an increasing order;(9)
end if(10)
Calculate the actual packet execution timeτina using equation (18);(11)
Scheduling operation according to the order of transmission setτink⟶τjnk=1,…,u+1;


## 3.2.1. Scheduling of Coupled Information

In NCSs with coupled information, data packets are transmitted in predefined order, i.e., for a plant within a control loop, its control information should be sent to the actuator before it collects data from a sensor, while for a controller, it has to receive sensor data first, and then sends the control information to the local controller via network after executing the control mechanism. In this way, the real-time data transmission within NCSs could be guaranteed. To perform scheduling for packets with the same significance level, a data transmission setτs has to be established as below according to the packet transmission time, while each packet should be assigned a time slot, within which the sensor scheduler or controller scheduler is fully occupied by the scheduling task specified:(9)τs=τin⟶τjnτin,τjn,∈τ,where τ represents all data packets to be transmitted, and τin⟶τjn defines the relative priority of two data packets within τ, indicating that τjn can be transmitted only when the transmission of τin is completed.However, since the processing of a packet may last for a period of time, the variablesτinr,τinf,τine have to be calculated. For an existing data transmission set with u packets τink⟶τjnk=1,…,u to be transmitted, the earliest transmission starting time τinf′k among those packets can be calculated as follows:(10)τinf′k=τinak+τinr,where τinr=Ti1tra. The earliest transmission starting time τinf among all packets within the set could be calculated:(11)τinf=maxτinf′1,…,τinf′u.Similarly, if there exists a transmission set withv packets following τin⟶τinkk=1,…,v, then the latest transmission starting time τine′k for those data packets can be calculated as follows:(12)τine′k=minτinek−τinr,Ti1M−τinr,and the latest transmission starting time τine for packets within the transmission set could be determined by (13)τine=minτine′k,τinf′k.


## 3.2.2. Scheduling of Noncoupled Information

In NCSs, there also exists noncoupled information with the packets being transmitted in an arbitrary order. To determine the scheduling operations for such packets, the packet transmission starting timeτina has to be calculated. In practice, however, since the NCS time delay typically consists of the forward channel delay, feedback channel delay, and controller processing delay, while the forward and feedback channels are symmetric, it is reasonable to assume that half of the overall delay comes from the forward channel while the other half is from the feedback channel. Hence, in the scheduling process, half of the delay TiM is assigned to the forward channel scheduler, and the other half is assigned to the feedback channel scheduler. In such a way, it is expected that the time slots reserved for the controller scheduler are enough for its scheduling operations. Hence, the longest allowable queuing delay Ti1m for those data packets could be determined as follows:(14)Ti1m=12TiM−Ti1trak−1−Ti1prck−1−Ti1posk−1,and the latest starting time would be(15)τine=Ti1m.Moreover, since the noncoupled packets could be transmitted in an arbitrary order, the earliest transmission starting time of a packet could be set asτinf=0, while its execution time could be τinr=Ti1tra.


## 3.2.3. Scheduling for Hybrid Coupled and Noncoupled Information

In NCSs with both coupled and noncoupled information, assume that there are information coupled packets withu following τink⟶τink=1,…,u and v following τin⟶τinhh=1,…,v, which are determined by (11) and (13), respectively, as well as w information noncoupled packets. If there exists a parameter jj∈u−2,u+w+2 for the scheduled packets τinj satisfying(16)τinf≥τinaj+τinrj,τinaj+1≥τinf+τinr,or(17)τinf≤τinaj+τinrj≤τine,τinaj+1≥τinaj+τinrj+τinr,then we have the actual execution time for nth task of the ith plant as follows:(18)τina=maxτinf,τinaj+τinrj.Hence, the time slot utilized for processing the packet within the control loop could be calculated as follows:(19)τina+τinr.The scheduling for any data packet to be transmitted within NCS could be conducted as shown in Algorithm1.Algorithm 1: Scheduling mechanism in feedback channel for distributed NCSs.
Input: an existing transmission set τink⟶τjnk=1,…,u; a new data packet with τin.Output: a new transmission set τink⟶τjnk=1,…,u+1.(1)
Calculate the time-delay upper boundTiM with Theorem ?? for the data packet;(2)
Calculate transmission priorityτinm with TiM, i.e., τinm∝1/TiM;(3)
Compare calculatedτinm to τinmk of τinkk=1,…,u in the set;(4)
if calculated τinm=τinmk for 1≤k≤uthen(5)
Insertτin into the end of set τink⟶τjnk=1,…,u, go to step 10;(6)
else(7)
Calculate information execution timeτia using equations (10)–(18);(8)
Insertτin into set τink⟶τjnk=1,…,u according to τina in an increasing order;(9)
end if(10)
Calculate the actual packet execution timeτina using equation (18);(11)
Scheduling operation according to the order of transmission setτink⟶τjnk=1,…,u+1;


## 3.3. Scheduling Mechanism for Controller Scheduler

To determine the scheduling operations for each packet, the execution time slot should be calculated for each data packet, and thus, the transmission timeτif,τie, and τia should be calculated using (11) and (13) and (16)–(18), respectively. Finally, the transmission set τs could be determined. The same scheduling operations shown in Algorithm 1 are executed within the controller scheduler for each packet of the plant.While for any plant adopting an NCS structure as shown in Figure1, the scheduling and transmission time slot distribution for each of its data packets could be illustrated in Figure 4.Figure 4
The scheduling and execution time distribution for each packet of the plant adopting an NCS structure as shown in Figure1.


## 4. Experimental Verification

To verify the effectiveness of the proposed time-delay bounded scheduling algorithm, experiments are conducted on a lab-customized teaching system. Figure5 depicts the schematic of the system. As seen, it consists of four layers, with the first layer being the motion platforms connected to the local sensors and actuators and the second layer being the training platform server group connected with the campus network. The third layer is the school information center and the fourth layer is student clients, and they are connected to the public and the campus networks, respectively. Specifically, in our experiments, those motion platforms are located in the new campus acting as NCS control plants, and they are controlled by a local controller, while those student clients are located within the old campus acting as the remote controllers. As those two campuses are 34 kilometers away from each other, the teaching system within the two campuses is interconnected by public transmission networks. In the experiments, scheduling operations are conducted on the local controller and school information center server, respectively, to facilitate the packet transmissions.Figure 5
Experimental setup of the NCS being utilized in our experiments.The motion platform as shown in Figure6 is a typical teaching instrument that has been commonly utilized in the laboratory for both load simulation and the position and attitude control of aircrafts. It has four degrees of freedom (4-DOFs), consisting of a linear motional freedom and three rational freedoms that are orthogonal to each other. Specifically, those 4-DOFs are along four separate axes, which are for inner ring motion, central ring motion, outer ring motion, and the line move motion, respectively, and each axis is equipped with two independent motors, with one for motion control and the other for force control. The force control system could be utilized to simulate the motion control loads or motion control interferences, while the motion control system could be used for force control interferences.Figure 6
The experimental motion platform with four degrees of freedom.In our experiments, the 4-DOFs together with their respective two control systems are utilized to simulate eight independent control plants. Specifically, the system local controllers act as sensor schedulers of the feedback channels, while those student client controllers act as schedulers for the forward channels to control all experimental components, and the main purpose of our experiment is to realize simultaneous closed-loop control of the 8 objects. To realize stable control of the control plants, we measured their dead zones and then set the measured values to be their respective lower and upper thresholds of the adopted relay-based dead-time compensators [29, 30] in the motion platform. In such a way, influences of the frictions within those objects could be eliminated. Each of those objects could be described with the mathematical model presented by (7), wherein discrete-time model matrices Ai and Aid denote the system state matrix and input matrix, respectively. Both Ai and Aid are shown in (20), and the system stability time-delay bound limits could be determined with the stability criterion presented in [27]. The model parameters, together with the loads of the four motion and the other four torque motors, i.e., inner ring, central ring, outer ring, and line move, are shown in Tables 1 and 2, respectively.(20)A=A11A12A21A22,Ad=Ad11Ad12Ad21Ad22.Table 1
Model parameters of the four motion systems simulated in Figure6 and the calculated upper limit of system time delay.
Motion systemA11A12A21A22Ad11Ad12Ad21Ad22TMmsInner ring motion0.8000.050.90−0.100−0.20−0.1038Central ring motion0.8200.060.91−0.110−0.20−0.0937Outer ring motion0.7900.050.93−0.120−0.21−0.0838Line move motion0.7800.060.92−0.130−0.19−0.1238Table 2
Model parameters of the four force systems simulated in Figure6 and the upper limit of system delay calculated using (7) with the same controller.
Force systemA11A12A21A22Ad11Ad12Ad21Ad22TMmsInner ring force1.010.110.110.830.2100.11−0.1020Central ring force0.980.120.090.790.2300.09−0.1121Outer ring force1.030.090.120.820.1900.12−0.1321Line move force1.050.10.080.810.2200.13−0.1419To evaluate the effectiveness of the proposed time-delay-bounded scheduling mechanism, we compare the time-delay performances of the tested motion platform with and without adopting the proposed scheduling mechanisms. Specifically, for anith system, we define the control time of a control loop to be Tik, which starts from the kth control cycle to the time when the local controller sends its control signals. Table 3 presents the measured Tik for the platform system without the proposed scheduling mechanism, while Table 4 shows the measured Tik when the proposed scheduling mechanism is adopted. Results in Table 3 show that without the proposed scheduling mechanisms, there exist a number of measured delays, denoted with red color texts with Tik>TiM, i.e., such delays are larger than the time-delay bound shown in Tables 1 and 2. Such large time delays could largely degrade the system performances or may even cause system instability. However, once the proposed data scheduling mechanisms are adopted, the measured time delays for all 8 channels are significantly reduced with Tik<TiM. Such results indicate that with the time-delay bounds taken into account, the proposed scheduling mechanisms could help facilitate the data transmission for the plants, which, thus, helps realize stable system control.Table 3
Measured NCS time delay for NCS without scheduling mechanism adopted.
SystemTiKith control cycle12345678910111213141516Inner ringMotion12151621192318132418133842435623Force2124353632121413159121624303212Central ringMotion15162119233237234865894745322324Force1215264325364212141591416161310Outer ringMotion27324587674557365623343818193219Force913141581613151434232534141316Line moveMotion64543454394521232534231824232924Force1013815153423213243121617121819Table 4
Measured NCS time delay for NCS with the proposed scheduling mechanism adopted.
SystemTiKith control cycle12345678910111213141516Inner ringMotion11121534382122282118133825163223Force12914131691214914131610141316Central ringMotion15162119233237232119262731322324Force91214914131610141316129141316Outer ringMotion27322334213132123223211218193219Force111513121091214914131013141613Line moveMotion32342128321821232521231824232424Force1316101413161013141612914131613The proposed scheduling mechanism together with different control schemes is also applied onto the teaching system to control the 4-DOFs motion platform online. Specifically, the control schemes are implemented to drive the motion platform, and the proposed scheduling mechanism is utilized to facilitate the data transmission over the campus and the public communication networks. The dynamic performances of the teaching system with and without the scheduling mechanisms are evaluated in different cases.(1)
Scheduling effect verification with the classical PID control strategy: we compare the performances of the outer ring force system before and after the schedule method being adopted. In the experiments, the influences of three main NCS delay factors, i.e., time delays, packet drops, and packet disordering, have been considered, wherein the influences of both packet drops and packet misordering were equivalently converted to be time delays. A simple PID controller was adopted for outer ring force system control, and the controller parameters were obtained with simple yet sophisticated modeling approach in the experiments [31], and they are also listed in Table 5. Figure 7 presents a state space description of such an adopted PID controller. Once the controller parameters are obtained, they would remain fixed for the whole experiments. The step signals starting from t=0.1s are utilized as the system inputs; the theoretical and practical system responses are obtained.Figures8 and 9 depict the theoretical and practical system responses with and without scheduling mechanisms being adopted, respectively. As seen in Figure 8, without adopting the scheduling mechanisms, the system response becomes divergent, i.e., the amplitude of the dashed curve increases with time and is away from the expected system response. In such a case, the system may finally become unstable. While once the scheduling mechanism is adopted, the system response converges, i.e., as shown in Figure 9, the practical system response converges to its expected state, and the system finally becomes stable. Such results demonstrate that the proposed scheduling algorithm could help facilitate the data transmission within NCS, which thus helps stabilize the system, even though the system dynamic performances are still not satisfactory as the system overshoot is large and the stability time is long as indicated in Figure 9.(2)
Scheduling verification with an NCS control mechanism proposed in [32]: we implemented the NCS process control mechanism presented in [32] and applied it to remote control the motion platform in our teaching system. Specifically, the proposed scheduling algorithm was utilized to facilitate data transmission within the NCS.Table 5
The detailed control parameters for the PID controller being utilized in the experiments.
ControllerParameter being utilizedParametersKpKiKdInner ringMotion3.00.10Force0.151.50Central ringMotion3.50.150Force0.161.60Outer ringMotion10.50.50Force0.11.750.01Line moveMotion250.50Force0.051.50.01Figure 7
A state space description of the simple PID controller being the simplified PID control model adopted for the outer ring force system of the motion platform.Figure 8
System step response before scheduling with PID controller.Figure 9
System step response after scheduling with PID controller.Figures10 and 11 present the step responses of the outer ring force system without and with the proposed scheduling algorithm being adopted, respectively. Results demonstrate that, owing to its predictive ability, the implemented control mechanism helps achieve satisfactory control effects even without adopting the scheduling algorithm. However, once the proposed scheduling algorithm is further adopted, the system control effects could be largely improved. As can be seen in Figure 11, the overshoot amplitude, overshoot time, and the stabilization time are significantly reduced as compared with those in Figure 10. Such performance comparison further verifies the effectiveness of the proposed scheduling algorithm.Figure 10
System step response for the control algorithm in [32] without scheduling mechanism.Figure 11
System step response for the control algorithm in [32] with scheduling mechanism adopted.As discussed, for an NCS system, the main factors affecting the system stability are the time delays, packet drops, and packet misorders of the transmitted data within the system; the main objective of NCS control system design is to achieve stable system control while minimizing the time delays. To address such an issue, the main idea of this study is to present a scheduling algorithm to reduce the time delays caused by data transmissions and postprocessing within NCS, i.e.,Ti1waitk, Ti2waitk, Ti1posk, and Ti2posk. Meanwhile, with the system time-delay bounds of the closed-loop NCSs being taken into account, the data within the NCS could be transmitted more efficiently within the system stability delay bound. In such a way, the system stability could be guaranteed while the time delays could be largely reduced. The above experimental results convincingly demonstrated the effectiveness of the proposed time-delay-bounded scheduling algorithm for NCS. Due to the large amount of data to be scheduled in real-time data transmission process, however, the proposed scheduling algorithm still suffers from the heavy data processing load, especially when the real-time audio/video data is huge. Specifically, once the time delay exceeds the time-delay bound characterized by an existing system stability criterion, the system would still be instable.


## 5. Conclusion

This paper investigates the time-delay reduction issue in distributed NCSs with time-varying delays and presents a dual-way data scheduling algorithm with the system stability time-delay bounds taken into account. Specifically, the paper analyzes the influences of various time-delay variables on the system stability first and then presents a one-way scheduling mechanism for data transmission within NCS, followed by an extended dual-way scheduling algorithm for time-delay reductions in NCSs. To verify the effectiveness of the proposed scheduling algorithm, both simulations and experiments are finally conducted on a teaching system to remote control a motion platform in different cases. Results show that the proposed scheduling algorithm could largely facilitate data transmission over the networks, which thus helps improve the NCS stability and control accuracy in different cases. It is believed that with the system time-delay bounds being considered, such a proposed scheduling algorithm could not only help reduce time-delays in NCSs, but also achieve stable system control for NCSs.



---

*Source: 8290879-2020-07-21.xml*