# Modeling and Analysis of Network Control System Based on Hierarchical Coloured Petri Net and Markov Chain

Authors: Jingdong Li; Zhangang Wang; Liankun Sun; Wanru Wang
Journal: Discrete Dynamics in Nature and Society
 (2021)

---

## Abstract

This paper investigates a modified modeling of networked control systems (NCSs) with programmable logic controller (PLC). First, the controller-to-actuator and sensor-to-controller network-induced delays are investigated by a modeling tactics based on hierarchical coloured petri net (HCPN) in a structure-conserving way. Comparing with the recent result, the signal transmission delay is set in a random interval instead of a fixed mode; moreover, the data packet drop out and disorder are also taken into consideration. Second, delays captured form CPN tools are analyzed with a strategy based on Baum–Welch algorithm and statistics science. Besides, time delays are modeled as a Markov chain and the transition probabilities is calculated using the consequent from the previous operation. Finally, a comparison verification illustrates the equivalence property between proposed models.


---

## Body

## 1. Introduction

As a kind of control system [1–6], NCSs have motivated a lot of research studies in the control field during recent years [7–10] by many advantages it shares; for example, reduced weight, high flexibility, simple installation and maintenance, and low cost. Because of those practical characteristics, more and more efforts have been devoted to these systems [11–13]. Admired for past achievements, NCSs have been found in widespread applications such as feedback control systems [14, 15], stabilization of linear systems [16], control of nonlinear systems [17], and adaptive tracking control of nonlinear multiagent systems [18–20]. Compared with traditional point-to-point hardwiring system, typical NCSs is a young generation of control architectures which has a feedback control structure consisting with the controller, sensor, and actuator through network communication. The insertion of this performance condition, with a finite bandwidth, brings serial challenging and undesirable issues on account of data packet loss, scheduling, and latency in the process of communication signal’s transmission to remote analog input/output unit. Affected by these cases, the communication channel, which takes the place of traditional signal transmission technologies, brings hard-to-solve problems in the stability analysis and controller synthesis because of transmission time delays. As a result, this issue received increasing interests in this field [21–23]. Traditional time-delay analysis usually assumes that the time delay is constant, time-varying or obeys some random distribution, and rarely analyzes it from the perspective of system operation mechanism. Modeling from the perspective of operation mechanism can effectively show various state changes that may occur in the system and the relationship between changes. This prompts us to adopt HCPN to carry out structural reservation modeling and then analyze the time-delay characteristics.In 1992, Petri nets were introduced by Petri [24] firstly as a net-theoretic approach to implement a particular purpose. The relationships between departments could be represented by a net, and it is a good approximation to imitate the appearance and character of asynchronous and concurrent operation in discrete event systems. Petri nets is a kind of mathematical structures which is a bipartite-directed graph consisting of two kinds of compositions; one is drawn as a circle which is called Place, and another drown as a rectangles named Transition. These two nodes combine with each other with arcs which are drawn as arrows. Coloured Petri net (CPN) [25–28], in which groups of objects is thus distinguished by colour, inherits all the advantages of classical Petri net. The wide practical application distributes on the direction of distributed control systems [29] and domains environments [30]. Recently, a novel of theoretical results has been done on the application of Petri nets for NCSs [31, 32]. Regarding the execution of the NCSs practical operational fundamental principle, the CPN has been chosen to build a formalism model [31]. With the rapid development of scientific technologies, the process in large-scale systems is more and more complicated, which makes it hard to solve problems in modeling. Therefore, to solve the problem of state explosion during formal verification, HCPN is invented [33–35] which is a new type of Petri net for creating large-scale and complex systems. Its main purpose is to summarize the system model with simple network model and to expand and fill it with substitution change. This method is not only beneficial to the excessive number of reservoirs, transitions, and arcs in the model but also beneficial to avoid the explosion of system state space and simplify the analysis of the model. In [31], a mapping from the established hypothetical delays in Ethernet of NCSs to entities of a CPN model was defined, and the simulation analysis of network induced delays was tested and verified in a simple way. Following a similar way, in [32], a two-step approach was included in the estimation of delays in the modeling of NCSs. However, the Ethernet model in the proposed model was studied in a hypothesis, in which time delays were not interrelating with network bandwidth-limited bit limit and packet dropouts. Further investigation and analysis is in [36, 37] and the references therein.The primary contributions of this paper are as follows:(i)
First, considering the potential cause for the deteriorating performance or instability on NCSs, this paper sets network band delay in an interval instead of the definite transmission delay in [32] to make the model more realistic. In this method, it is supported by the HCPN model to simulate some challenging issues on account of network information propagation.(ii)
Second, the CPN model in [31, 32] is not considered data packet latency and dropout in the Ethernet CPN model of NCSs. Focusing on those challenges in network transmission, this paper has some alterations on the base of the model in [32]; close to the reality, data packets’ dropout and packet out-of-order are taken into consideration in the proposed model of data transmission on the network.(iii)
Third, the exploration and research of random time delays existing in the sensor-to-controller and controller-to-actuator in discrete time networked control systems are acquiescence with random Markovian delays in recent literature [38–40]. However, it is not clear if this theoretical acquiescence can be put into practice. This paper makes a certification of equivalence between time delays and Markov model [41].This paper is organized as follows. Section2 is the conceptual framework of Petri net. Section 3 is the HCPN model of NCSs-PLC. Section 4 analyzes the time delay extracted from the HCPN model. Section 5 is the equivalence analysis of the time delay of both the Markov model and the HCPN model.


## 2. Basic Conception

Definition 1.
A ordinary non-HCPN can be defined as a nine elements’ tuple [42],(1)Traditional CPN=Σ,P,T,A,N,C,G,E,I,satisfying the requirements below:(i)
Σ is called colour set which is a finite set describing nonempty types.(ii)
P is Place representing a ellipse which interprets a passive component with discrete status.(iii)
T is Transition establishing with a rectangle which explains an active component; tokens can consume, produce, and change the carrying information in Transitions.(iv)
A is arcs which connects Places and Transitions in the model. It can be represented by arrows, and it is a finite set which meets the expectations with ∀Pi∈P;Ti∈T:Pi∩Ti=Pi∩Ai=Ti∩Ai=∅.(v)
N is defined as a map into arcs. It is defined from A into Pi×Ti∪Ti×Pi which has two elements, the first element means arcs’ source and the second element means arcs’ destination.(vi)
C is defined a map in places. It is defined from Pi into Σ which means that every token on every Pi has a corresponding colour set type.(vii)
G is the guard function which is defined a map in transitions. GTi is the type appertain to Σ, and the binding must perform every Boolean expressions. It is can be shown as ∀Ti∈TTypeVarGuardT⊆Σ∧TypeGuardTi=Boolean.(viii)
E is called arc expression which maps every element in A to an expression, and the type of it can be written as CPiMS; it can be shown as(2)∀a=ATypeEi=CP^iMS∧TypeVarEi⊆Σ,where P^i is defined as the place of Ni.(ix)
I illustrates a map between Pi and the type C^i It is the initialization function which can be shown as(3)∀Pi∈P:TypeIi=C^i.Definition 2.
A HCPN can be defined as a nine elements-tuple satisfying the requirements below [42]:(4)HCPN=Se,SeN,SeA,PLPN,POT,POA,PLFS,FT,PP.(i)
Se is a congregations for nonhierarchical pages in the model, and each page does not have collaborative net elements. It can be shown as(5)ΣSi,PSi,TSi,ASi,NSi,CSi,GSi,ESi,ISi,∀S1,S2∈Se:S1≠S2⇒PS1∪TS1∪AS1∩PS2∪TS2∪AS2=∅.(ii)
SeN∈T is substitution nodes.(iii)
SeA is called page assignment, and every page must not be the subpage of itself; it is satisfied by the following conditions:(6)s^0,s^1,…,s^i∈S^i|n∈N+∧s^0=s^n∧∀k∈1,…,n:sk∈SeASNSiK−1=∅.(iv)
PLPN⊆P is defined as a set of place nodes or transition nodes.(v)
POT is a function of port type, and it has four types, in type, out type, in/out type, and general.(vi)
POA is port assignment. It is binary relations such that(7)∀t∈SeN:POAt⊆Xt×PLPN−SeAT,∀t∈SeN:∀p1,p2∈POA:POTp2≠general⇒STp1,t=PTp2,∀t∈SeN:∀p1,p2∈POA:Cp1=Cp2∧Ip1=Ip2.(vii)
PLFS∈Ps is a finite set of fusion sets such that(8)∀fs∈PLFS:∀p1,p2∈fs:Cp1=Cp2∧Ip1=Ip2.(viii)
FT is a function of the fusion type. It is defined from fusion into global,page,instance, such that(9)FT:FS⟶global,page,instance,∀fs∈FS:FTfs≠global⇒∃s∈S:fs⊆Ps.(ix)
PP∈SMS is a multiset of the prime page.


## 3. The CPN Model for NCSs

### 3.1. The Top Model

The integrated model can be divided into two levels, one is top level which simulates the relationship between the controller, the sensor, and the actuator with a cursory method, as shown in Figure1. The other level is the detailed function description of the controller, the sensor, and the actuator. In the proposed top-level model, the sensor and the actuator connect with PLC Remote IO. PLC-CPU sends packets from the controller to PLC Remote IO through Ethernet; then, PLC Remote IO sends acknowledgments from the sensor and the actuator back to PLC-CPU. There are three subpages and one top page in the whole model. After some introductions of time delay in PLC-CPU, those pages are explained one by one in detail.Figure 1
The top level in the CPN model for NCSs.


### 3.2. The Analysis of the Time Delays in PLC-NCSs

Figure2 is a representative of traditional NCSs with the PLC controller. The sensor and actuator are connected to digital/analog I/O section through Ethernet. Before entering or leaving network, data packets will enter into a transmission queue to wait for the scan cycle of I/O.Figure 2
Network control system with PLC being the CPU controller.Figure3 gives detailed instructions of time delays in NCSs with the PLC controller, in which different samples were shown as different length arrows. In the upper part of the whole figure, there are some samples which are received in the sensor with periodic sampling. In the middle part, there are three kinds of axis showing the PLC controller. Before data packets arrive at execute program, it has to wait for the scan cycle so that it can be read into PLC. Both of the reading scan cycle and writing scan cycle in PLC communication are Tc. Another primary scan cycle is Tp which means that only at Tp scan cycles data packets can be captured into PLC-CPU to execute the programme. After a few execution time τc, the communication module sends result data to the actuator node. During the process of the data packets’ transmission, two network-induced delays are produced. One is called backward time delay and another is called forward time delay. The execution time delay in the controller is included in Tsc backward time delay which is the time costing in the transmission from the controller to the sensor, and Tca is forward time delay which is from CPU which received the sample to the actuator which received the same sample.Figure 3
Process of time delay in PLC-CPN.


### 3.3. The Controller Model

The module of controller is divided into two parts. One is the communication module part, which is used to sending and receiving data with scan cycles, such as left part in Figure4. The another part is the PLC-CPU model, which simulates functions of CPU reading(input), CPU executing, and CPU writing(output), such as right part in Figure 4. The Send_R transition and the Rec_A transition complete data packets’ sending and receiving, respectively. PLC_R transition, PLC_E transition, and PLC_W transition complete simulation of reading, executing, and writing in the PLC-CPU. The definition of token elements in the model is shown in Table 1.Figure 4
The controller module in the HCPN model for NCSs-PLC.Table 1
Token definition of the HCPN model for NCSs.
colset UNIT=unit timed;colset BOOL=bool;colset INT=int timed;colset TIME=time;colset STRING=string;colsetN=INT;colsetA=withY|N;colsetI,O=int;colset Ten0=int with0,…,10;colset Ten1=int with1,…,10;fun Oks:Ten0,r:Ten1=r<=s;colset Packet=recordN:N∗I:I∗O:O∗A:Atimed;colset Data=product INT∗INT timed;;vari,i1,i2,o1,o2:INT;varp1,p2:Packet;;


### 3.4. The Ethernet Model

Figure5 is the Ethernet model which is the primary part for whole NCSs’ system. There are two modified points contrasting with the model which has been designed by Ghanaim et al. [32]. When the data packets transmit in Ethernet, transmission will be influenced by a lot of uncertainties. In consideration of these cases, the transmission time is set into a random time with an interval time varying rather than a fix data. In the proposed model, time delays of this portion are discrete uniform distributions between 1000 and 2000 when packet data passed from the sensor to the controller or from the controller to the actuator. In addition, in the actual network transmission, the data packets’ transmission is unstable because of some uncertain factors, such as packets lost and packets out of order. In this model, packets will be lost in ten percent probability, and this function is realized by fun Oks:Ten0,r:Ten1=r<=s. Data packets will be retransmitted based on Transmission Control Protocol/Internet Protocol if those transmission mistakes occurred. The mathematic presentation of this model is a tuple:(10)HCPN=S.SN.SA.PN,PT,PA,FS,FT,PP.Figure 5
The Ethernet model for NCSs.By the definition ofHCPN, those elements in this tuple can be used to describe the Ethernet model, as shown in Figure 6.Figure 6
Mathematical description for HCPN.Σ is the Token colour set in Figure 5. P is finite set of places and T is finite set of transitions. A is arc, and Cto Store1 means that this arc is from place C to transition Store1. N is the node function. G is guard, for example, #A,p1=N,T=Net1,Net2 means that when transition is Net1 or Net2, the guard of transition is #A,p1=N; otherwise, transition guards are true. C is the colour definition of Place. Place A, B, C, and D are mean the Packet colour. IP is the initialization set if the place is SP1 or SP2, and the initial token is integer nine. Ea means the arc expression in the Ethernet model, p1,a∈Ato Net1,Cto Store1⋅Store1toA means when the arc is from Ato Net1,Cto Store1, and Store1toA, and the arc expression on those arc is p1.


## 3.1. The Top Model

The integrated model can be divided into two levels, one is top level which simulates the relationship between the controller, the sensor, and the actuator with a cursory method, as shown in Figure1. The other level is the detailed function description of the controller, the sensor, and the actuator. In the proposed top-level model, the sensor and the actuator connect with PLC Remote IO. PLC-CPU sends packets from the controller to PLC Remote IO through Ethernet; then, PLC Remote IO sends acknowledgments from the sensor and the actuator back to PLC-CPU. There are three subpages and one top page in the whole model. After some introductions of time delay in PLC-CPU, those pages are explained one by one in detail.Figure 1
The top level in the CPN model for NCSs.


## 3.2. The Analysis of the Time Delays in PLC-NCSs

Figure2 is a representative of traditional NCSs with the PLC controller. The sensor and actuator are connected to digital/analog I/O section through Ethernet. Before entering or leaving network, data packets will enter into a transmission queue to wait for the scan cycle of I/O.Figure 2
Network control system with PLC being the CPU controller.Figure3 gives detailed instructions of time delays in NCSs with the PLC controller, in which different samples were shown as different length arrows. In the upper part of the whole figure, there are some samples which are received in the sensor with periodic sampling. In the middle part, there are three kinds of axis showing the PLC controller. Before data packets arrive at execute program, it has to wait for the scan cycle so that it can be read into PLC. Both of the reading scan cycle and writing scan cycle in PLC communication are Tc. Another primary scan cycle is Tp which means that only at Tp scan cycles data packets can be captured into PLC-CPU to execute the programme. After a few execution time τc, the communication module sends result data to the actuator node. During the process of the data packets’ transmission, two network-induced delays are produced. One is called backward time delay and another is called forward time delay. The execution time delay in the controller is included in Tsc backward time delay which is the time costing in the transmission from the controller to the sensor, and Tca is forward time delay which is from CPU which received the sample to the actuator which received the same sample.Figure 3
Process of time delay in PLC-CPN.


## 3.3. The Controller Model

The module of controller is divided into two parts. One is the communication module part, which is used to sending and receiving data with scan cycles, such as left part in Figure4. The another part is the PLC-CPU model, which simulates functions of CPU reading(input), CPU executing, and CPU writing(output), such as right part in Figure 4. The Send_R transition and the Rec_A transition complete data packets’ sending and receiving, respectively. PLC_R transition, PLC_E transition, and PLC_W transition complete simulation of reading, executing, and writing in the PLC-CPU. The definition of token elements in the model is shown in Table 1.Figure 4
The controller module in the HCPN model for NCSs-PLC.Table 1
Token definition of the HCPN model for NCSs.
colset UNIT=unit timed;colset BOOL=bool;colset INT=int timed;colset TIME=time;colset STRING=string;colsetN=INT;colsetA=withY|N;colsetI,O=int;colset Ten0=int with0,…,10;colset Ten1=int with1,…,10;fun Oks:Ten0,r:Ten1=r<=s;colset Packet=recordN:N∗I:I∗O:O∗A:Atimed;colset Data=product INT∗INT timed;;vari,i1,i2,o1,o2:INT;varp1,p2:Packet;;


## 3.4. The Ethernet Model

Figure5 is the Ethernet model which is the primary part for whole NCSs’ system. There are two modified points contrasting with the model which has been designed by Ghanaim et al. [32]. When the data packets transmit in Ethernet, transmission will be influenced by a lot of uncertainties. In consideration of these cases, the transmission time is set into a random time with an interval time varying rather than a fix data. In the proposed model, time delays of this portion are discrete uniform distributions between 1000 and 2000 when packet data passed from the sensor to the controller or from the controller to the actuator. In addition, in the actual network transmission, the data packets’ transmission is unstable because of some uncertain factors, such as packets lost and packets out of order. In this model, packets will be lost in ten percent probability, and this function is realized by fun Oks:Ten0,r:Ten1=r<=s. Data packets will be retransmitted based on Transmission Control Protocol/Internet Protocol if those transmission mistakes occurred. The mathematic presentation of this model is a tuple:(10)HCPN=S.SN.SA.PN,PT,PA,FS,FT,PP.Figure 5
The Ethernet model for NCSs.By the definition ofHCPN, those elements in this tuple can be used to describe the Ethernet model, as shown in Figure 6.Figure 6
Mathematical description for HCPN.Σ is the Token colour set in Figure 5. P is finite set of places and T is finite set of transitions. A is arc, and Cto Store1 means that this arc is from place C to transition Store1. N is the node function. G is guard, for example, #A,p1=N,T=Net1,Net2 means that when transition is Net1 or Net2, the guard of transition is #A,p1=N; otherwise, transition guards are true. C is the colour definition of Place. Place A, B, C, and D are mean the Packet colour. IP is the initialization set if the place is SP1 or SP2, and the initial token is integer nine. Ea means the arc expression in the Ethernet model, p1,a∈Ato Net1,Cto Store1⋅Store1toA means when the arc is from Ato Net1,Cto Store1, and Store1toA, and the arc expression on those arc is p1.


## 4. Time-Delay Analysis

### 4.1. Transmission Delay in Ethernet

Network transmission is an important element in the analysis ofTsc and Tca. Besides, it also can help to analyze the system state and performance. Data packets may be lost at a probability such as ten percent in the model of network transmission. With reference to Transmission Control Protocol/Internet Protocol (TCP/IP), data packets would be retransmitted if it is lost in the network transmission. Figure 7 is network transmission latency which is collected in the proposed HCPN model running 5050 steps. From Figure 7, it can be clearly see that, in time delay Tsc and Tca, network transmission time delays are almost greater than 1000 and less than 2000; however, there are still ten percent points out of scope, Figure 7 shows several peak points, for example A, B, and C, which mean that, in this time, data packets are dropped out in network transmission. It will be retransmitted according to TCP/IP so that the value of it becomes very bigger than others. The network transmission latency was the transmission time and the retransmission time and other time spent on extracting data packets.Figure 7
Network transmission latency in the HCPN model for NCSs.


### 4.2. The Time Delay Tca and Tsc

There are three scan cycles which play important roles in the calculation of backward time delay and forward time delay. The first one is periodic scanTp for CPU program, the second one is periodic IO scan Tc for the communication module, and the last one is the periodic sampling Th for the sensor. There exists another important data to calculate: network transmission delay. Table 2 is the parameters defined in the NCSs model which can be used to simulate and analyze the efficiency of the difference system with the HCPN model.Table 2
Main parameters.
ModuleTimePLC moduleIO scan cycle17000Read2000Execute8000Write2000Ethernet module2000Remote analog input/outputIO-read1000IO-write1000In addition, in the communication module,IO periodic scan cycle Tc is a summation of reading time, executing time, and writing time. The periodic scan Tp for CPU program is set to 17,000 and the periodic sampling Th for the sensor is 1000. When network transmission latency was obtained, such as in Figure 7, Tca and Tsc can be calculated under the rule of scan cycles, as shown in Figure 8. Before the calculation of Tca and Tsc, it is necessary to obtain the network latency, which are produced in network transmission and waiting time for the scan cycle. Contrasting with delays obtained from the HCPN model in [32], it can be clearly see that because of the indeterminacy transmission phenomenon and the random transmission time delay the periodicity of Tca and Tsc is broken.Figure 8
Tsc time delay and Tca time delay.


## 4.1. Transmission Delay in Ethernet

Network transmission is an important element in the analysis ofTsc and Tca. Besides, it also can help to analyze the system state and performance. Data packets may be lost at a probability such as ten percent in the model of network transmission. With reference to Transmission Control Protocol/Internet Protocol (TCP/IP), data packets would be retransmitted if it is lost in the network transmission. Figure 7 is network transmission latency which is collected in the proposed HCPN model running 5050 steps. From Figure 7, it can be clearly see that, in time delay Tsc and Tca, network transmission time delays are almost greater than 1000 and less than 2000; however, there are still ten percent points out of scope, Figure 7 shows several peak points, for example A, B, and C, which mean that, in this time, data packets are dropped out in network transmission. It will be retransmitted according to TCP/IP so that the value of it becomes very bigger than others. The network transmission latency was the transmission time and the retransmission time and other time spent on extracting data packets.Figure 7
Network transmission latency in the HCPN model for NCSs.


## 4.2. The Time Delay Tca and Tsc

There are three scan cycles which play important roles in the calculation of backward time delay and forward time delay. The first one is periodic scanTp for CPU program, the second one is periodic IO scan Tc for the communication module, and the last one is the periodic sampling Th for the sensor. There exists another important data to calculate: network transmission delay. Table 2 is the parameters defined in the NCSs model which can be used to simulate and analyze the efficiency of the difference system with the HCPN model.Table 2
Main parameters.
ModuleTimePLC moduleIO scan cycle17000Read2000Execute8000Write2000Ethernet module2000Remote analog input/outputIO-read1000IO-write1000In addition, in the communication module,IO periodic scan cycle Tc is a summation of reading time, executing time, and writing time. The periodic scan Tp for CPU program is set to 17,000 and the periodic sampling Th for the sensor is 1000. When network transmission latency was obtained, such as in Figure 7, Tca and Tsc can be calculated under the rule of scan cycles, as shown in Figure 8. Before the calculation of Tca and Tsc, it is necessary to obtain the network latency, which are produced in network transmission and waiting time for the scan cycle. Contrasting with delays obtained from the HCPN model in [32], it can be clearly see that because of the indeterminacy transmission phenomenon and the random transmission time delay the periodicity of Tca and Tsc is broken.Figure 8
Tsc time delay and Tca time delay.


## 5. Verification and Analysis

### 5.1. Markov Modeling

A Hidden Markov ModelHMM is a statistical Markov lain with hidden states which cannot be directly visible, but the sequence of observations can give some information of HMM. The sequence of observations alphabet is set to Oseq, and a hidden states’ alphabet sequence Sseq can be cached from Oseq. Oseq and Sseq are shown below:(11)Oseq=o1,o2,…,oi,…,Sseq=s1,s2,…,si,…,where Oi∈O=o1,o2,…,oN is the observation state at time k and Si∈S=s1,s2,…,sN is the corresponding state at observation Ok.The formal definition of a hidden Markov modelHMM is as follows:(12)λ=A,B,π0,where A is the state transition probability matrix form one state to another state, which can be written as(13)A=aij,aij=Psk+1=sj|sk=si,B is the observation probability matrix which can be written as(14)B=bij,bij=Pok+1=oj|sk=si,and π0 is the initial state probability which can be written as(15)π0=πi,πi=Ps1=sj.


### 5.2. Markov Model Results

By the definition ofHMM and the Baum Welth algorithm, the algorithm steps are as follows.Step 1
: for a given observation sequence and HMM model, the probability variables of the hidden stateSi are(16)δti=αtiβiPO|λ,PO|λ=∑i=1Nαtiβi,where αti is called forward probability, where(17)αti=Po1o2o3…ot−1ot,qt=Si|λ,and βti is called backward probability, where(18)βti=Pot+1ot+2ot+3…oT−1oT,qt=Si|λ,Step 2
: define the probability between the hidden stateSi and the hidden state Sj as follows:(19)ϕti,j=Pqt=Si,qt+1=Sj|O,λ,ϕti,j=αtiaijbjOt+1βt+1jPO|λ,PO|λ=∑i=1N∑j=1NαtiaijbjOt+1βt+1j.Step 3
: the formula betweenδti and ϕti,j is(20)δti=∑j=1Nϕti,j.Step 4
: find the corresponding expected values of the above two variables∑t=1T−1δti and ∑t=1T−1ϕti,j.Step 5
: the new HMM parameters are estimated by using the two variables defined above and their expected values:(21)a^ij=∑t=1T−1ϕti,j∑t=1T−1δti,b^jk=∑t=1T−1ϕti,j∑t=1Tδti.The Markov model can be built based on time delayTsc and Tcs obtained from HCPN. In order to make analyzing more clear, data will be managed with the rounding method. From Figure 8, it two time delays can be obtained as follows:(22)Tsc=3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,Tca=10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,29.Tsc can be divided into two categories: one category is the low delay Tsclow which includes the value Tsclow=3,4,5,6,7,8,9. The other category is high delay Tschigh which includes the value Tschigh=10,11,12,13,14,15,16,17.The time delayTsc can be written as(23)Tsc=Tsclow,sk=s1,Tschigh,sk=s2.The same procedure can be easily adapted toTca which can be divided into two categories: one category is the low delay Tcalow which includes the value Tcalow=10,11,12,13,14,15,16,17,18,19 and the other category is high delay Tcahigh, and Tcahigh=20,21,22,23,24,25,26,27,29. The time delay Tca can be written as(24)Tca=Tcalow,sk=s1,Tcahigh,sk=s2.Figures9 and 10 show the probability distribute of time delay Tsc and Tca which are the experimental data in HCPN, respectively.Figure 9
Distribution ofTsc in the HCPN model.Figure 10
Distribution ofTca in the HCPN model.The parameters in theHMM model can be calculated with Baum–Welch algorithm and maximum likelihood estimate, and the system λsc=Asc,Bsc,π0SC with 2-state and 15-observations, and the matrix Asc,Bsc can be calculated by Matlab:(25)Asc=0.280.720.840.15,Bsc=Bsc1Bsc2,Bsc1=0.150.100.150.150.150.130.170000000,Bsc2=000000000.200.200.180.200.120.030.040.03.In the similar way, the systemλca=Aca,Bca,π0ca with 2-state and 19-observations can be calculated with(26)Aca=0.780.220.250.75,Bca=Bca1Bca2,Bca1=0.060.100.080.110.110.110.080.190.100.060000000000,Bca2=0000000000.190.120.100.150.060.150.150.060.02.


### 5.3. Validation Results

From the above section, the important matrixes can be obtained as(27)A=Asc,Aca,B=Bsc,Bca.Using the data from those two matrices, the stationary distribution of the Markov modelλsc and λca can be estimated by those two time delays (Tca and Tsc) with the Matlab statics toolbox. The probability density functionPDF can be calculated by sample arrays which can be obtained from HCPN consecutive time delay sequence. The PDF is the distribution of the time delay which can be compared with the stationary distribution of the Markov model.Figures11 and 12 are the stationary distribution of the Hidden Markov Model building above, and every observation probability value is similar to the probability density function of the time-delay sequence. The figures show that the Markov stationary distribution of both the time delay Tsc and the time delay Tca are approximately equivalent to the probability density function of HCPN. It is clearly seen that building a Markov modeling of NCSs is feasible and time delays conform to Markov lain.Figure 11
The comparison ofTsc probability distribution between the HCPN model and the Markov model.Figure 12
The comparison ofTca probability distribution between the HCPN model and the Markov model.


## 5.1. Markov Modeling

A Hidden Markov ModelHMM is a statistical Markov lain with hidden states which cannot be directly visible, but the sequence of observations can give some information of HMM. The sequence of observations alphabet is set to Oseq, and a hidden states’ alphabet sequence Sseq can be cached from Oseq. Oseq and Sseq are shown below:(11)Oseq=o1,o2,…,oi,…,Sseq=s1,s2,…,si,…,where Oi∈O=o1,o2,…,oN is the observation state at time k and Si∈S=s1,s2,…,sN is the corresponding state at observation Ok.The formal definition of a hidden Markov modelHMM is as follows:(12)λ=A,B,π0,where A is the state transition probability matrix form one state to another state, which can be written as(13)A=aij,aij=Psk+1=sj|sk=si,B is the observation probability matrix which can be written as(14)B=bij,bij=Pok+1=oj|sk=si,and π0 is the initial state probability which can be written as(15)π0=πi,πi=Ps1=sj.


## 5.2. Markov Model Results

By the definition ofHMM and the Baum Welth algorithm, the algorithm steps are as follows.Step 1
: for a given observation sequence and HMM model, the probability variables of the hidden stateSi are(16)δti=αtiβiPO|λ,PO|λ=∑i=1Nαtiβi,where αti is called forward probability, where(17)αti=Po1o2o3…ot−1ot,qt=Si|λ,and βti is called backward probability, where(18)βti=Pot+1ot+2ot+3…oT−1oT,qt=Si|λ,Step 2
: define the probability between the hidden stateSi and the hidden state Sj as follows:(19)ϕti,j=Pqt=Si,qt+1=Sj|O,λ,ϕti,j=αtiaijbjOt+1βt+1jPO|λ,PO|λ=∑i=1N∑j=1NαtiaijbjOt+1βt+1j.Step 3
: the formula betweenδti and ϕti,j is(20)δti=∑j=1Nϕti,j.Step 4
: find the corresponding expected values of the above two variables∑t=1T−1δti and ∑t=1T−1ϕti,j.Step 5
: the new HMM parameters are estimated by using the two variables defined above and their expected values:(21)a^ij=∑t=1T−1ϕti,j∑t=1T−1δti,b^jk=∑t=1T−1ϕti,j∑t=1Tδti.The Markov model can be built based on time delayTsc and Tcs obtained from HCPN. In order to make analyzing more clear, data will be managed with the rounding method. From Figure 8, it two time delays can be obtained as follows:(22)Tsc=3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,Tca=10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,29.Tsc can be divided into two categories: one category is the low delay Tsclow which includes the value Tsclow=3,4,5,6,7,8,9. The other category is high delay Tschigh which includes the value Tschigh=10,11,12,13,14,15,16,17.The time delayTsc can be written as(23)Tsc=Tsclow,sk=s1,Tschigh,sk=s2.The same procedure can be easily adapted toTca which can be divided into two categories: one category is the low delay Tcalow which includes the value Tcalow=10,11,12,13,14,15,16,17,18,19 and the other category is high delay Tcahigh, and Tcahigh=20,21,22,23,24,25,26,27,29. The time delay Tca can be written as(24)Tca=Tcalow,sk=s1,Tcahigh,sk=s2.Figures9 and 10 show the probability distribute of time delay Tsc and Tca which are the experimental data in HCPN, respectively.Figure 9
Distribution ofTsc in the HCPN model.Figure 10
Distribution ofTca in the HCPN model.The parameters in theHMM model can be calculated with Baum–Welch algorithm and maximum likelihood estimate, and the system λsc=Asc,Bsc,π0SC with 2-state and 15-observations, and the matrix Asc,Bsc can be calculated by Matlab:(25)Asc=0.280.720.840.15,Bsc=Bsc1Bsc2,Bsc1=0.150.100.150.150.150.130.170000000,Bsc2=000000000.200.200.180.200.120.030.040.03.In the similar way, the systemλca=Aca,Bca,π0ca with 2-state and 19-observations can be calculated with(26)Aca=0.780.220.250.75,Bca=Bca1Bca2,Bca1=0.060.100.080.110.110.110.080.190.100.060000000000,Bca2=0000000000.190.120.100.150.060.150.150.060.02.


## 5.3. Validation Results

From the above section, the important matrixes can be obtained as(27)A=Asc,Aca,B=Bsc,Bca.Using the data from those two matrices, the stationary distribution of the Markov modelλsc and λca can be estimated by those two time delays (Tca and Tsc) with the Matlab statics toolbox. The probability density functionPDF can be calculated by sample arrays which can be obtained from HCPN consecutive time delay sequence. The PDF is the distribution of the time delay which can be compared with the stationary distribution of the Markov model.Figures11 and 12 are the stationary distribution of the Hidden Markov Model building above, and every observation probability value is similar to the probability density function of the time-delay sequence. The figures show that the Markov stationary distribution of both the time delay Tsc and the time delay Tca are approximately equivalent to the probability density function of HCPN. It is clearly seen that building a Markov modeling of NCSs is feasible and time delays conform to Markov lain.Figure 11
The comparison ofTsc probability distribution between the HCPN model and the Markov model.Figure 12
The comparison ofTca probability distribution between the HCPN model and the Markov model.


## 6. Conclusion

In this paper, we have investigated a novel HCPN model approach for the network control system with PLC. The modified structure-conserving model has been accomplished to calculate sequences of delays under the control of scan cycles for PLC-CPU. Besides, a series of special phenomenon has been taken into consideration based on the traditional mode, such as data packets drop and data packets out of order. Time delays in network transmission have been calculated to observe the transmission data packet state in Ethernet. Finally, Markov analytical models have been built for analyzing time delays which can be obtained from the forward step mathematically.



---

*Source: 9948855-2021-06-09.xml*