# Neural PID Control Strategy for Networked Process Control

Authors: Jianhua Zhang; Junghui Chen
Journal: Mathematical Problems in Engineering
 (2013)

---

## Abstract

A new method with a two-layer hierarchy is presented based on a neural proportional-integral-derivative (PID) iterative learning method over the communication network for the closed-loop automatic tuning of a PID controller. It can enhance the performance of the well-known simple PID feedback control loop in the local field when real networked process control applied to systems with uncertain factors, such as external disturbance or randomly delayed measurements. The proposed PID iterative learning method is implemented by backpropagation neural networks whose weights are updated via minimizing tracking error entropy of closed-loop systems. The convergence in the mean square sense is analysed for closed-loop networked control systems. To demonstrate the potential applications of the proposed strategies, a pressure-tank experiment is provided to show the usefulness and effectiveness of the proposed design method in network process control systems.


---

## Body

## 1. Introduction

Networked control systems (NCSs) make it convenient to control large distributed systems. Process control can integrate the controlled process and the communication network of computational devices, but sensors and actuators cannot be directly used in a conventional way because there are some inherent issues in NCS, such as delay, packet loss, quantization, and synchronization. Recently, some efforts have been made to deal with these issues about NCSs. Zhang et al. (2012) investigated the stability problem of a class of delayed neural networks with stabilizing or destabilizing time-varying impulses [1]. The controlling regions in neural networks were identified in three scales using single-objective evolutionary computation methods [2]. Tian et al. (2008) investigated the observer-based output feedback control algorithm for networked control systems with two quantizers using a set of nonlinear matrix inequalities [3]. In robust design, a robust and reliable H∞ filter was designed for a class of nonlinear networked control systems with random sensor failure [4].In addition, some investigations have been made to incorporate the network into process control systems in the literature. In the work of [5], their model predictive control algorithm was extended for processes with random delays using a communication delay model, along with time-stamping and buffering technique. Two novel networked model predictive control schemes based on neighborhood optimization were presented for online optimization and control of a class of serially connected processes [6]. A two-tier control architecture was presented [7]. A lower-tier control system relying on point-to-point communication and continuous measurements was first designed to stabilize the closed-loop system. Also, an upper-tier networked control system was subsequently designed using the Lyapunov-based model predictive control theory to profit from both the continuous and the asynchronous, delayed measurements as well as from additional networked control actuators to improve the closed-loop system performance. A design methodology for fault-tolerant control systems of chemical plants with distributed interconnected processing units was presented [8]. This approach incorporated Lyapunov-based nonlinear control with the hybrid system theory. It was developed based on a hierarchical architecture that integrated lower-level feedback control of the individual units with upper-level logic-based supervisory control over communication networks. An output feedback controller was proposed. It combined a Lyapunov-based controller with a high-gain observer for nonlinear systems subject to sensor data losses [9].Since NCS operates over a network, data transfer between the controller and the remote system, in addition to the controlled processing delay, will inevitably introduce network delays, the controller-to-actuator delay and the sensor-to-controller delay. Random access networks, such as CAN and Ethernet, have random network delays, which may cause deterioration of the system performance [10]. The packet loss caused by communication networks also affects the control system performance; that is, process control systems under the communication environment are stochastic in nature because of the random time delays or packet loss caused by the communications on the employed networks.The output variable of a network process is usually subjected to the network delay of the uncertain duration and/or stochastic disturbances. It can be treated as a random variable that follows a specific probability density function (PDF). Consequently, the output tracking error is also a random variable. PDF of the output tracking error can be determined if the information about the process model and PDF of both disturbances and network delays is available. The dynamic stochastic distribution control theory has been proposed [11]. It is generally true for only linear systems with Gaussian random inputs or at least symmetric PDF that the minimum variance of the output tracking error indicates optimization of controller tuning. However, for non-Gaussian stochastic systems, the control method that is focused only on mean and variance of the output tracking error is not sufficient to capture the probabilistic behavior of this stochastic system. A more general measure of uncertainty should be used to characterize the uncertainty of the output tracking error in the control design.If the network is connected with the regulatory control level and the advanced control level is located at the remote side to cooperate with other plants, the whole control system will have a two-level hierarchy. The control structure of the regulatory control level in the plant site is unchanged. Thus, the simple and robust proportional-integral-derivative (PID) controller can be applied to the operating plant. The conventional PID controller can get the support from the networked process control systems when the operating plants encounter a new operation condition or uncertain factors. On the other hand, recently neural networks attracted a very large research interest.  They have great capability of solving complex mathematical problems and they have received considerable attention in the field of chemical process control and have been applied to system identification and controller design [12]. They are effectively used in the control region for modeling nonlinear processes, especially in model-based control, such as direct and indirect neural network model based control [13], nonlinear internal model control [14], and recurrent neural network model control [15]. Although the control performances of the above methods are satisfactory, the control design still focuses on point-to-point wired communication links between the designed controllers and the controlled processes. It makes the implementation strategy realistic only for in-site control systems. The optimal fuzzy PID controller for NCSs by minimizing the sum of the integral of time multiplied by absolute errors and the squared controller output was proposed [16]. Neural controllers for NCSs were also applied based on minimum tracking error entropy for the tuning PID controller [17], but it was not easy to implement them on practical applications because they needed a set of tracking errors at the instant time point. Ghostine et al. (2011) used Monte Carlo simulation to demonstrate the effectiveness of the traditional PID controller, but they did not tell how to adjust PID controller parameters [18]. Indeed, for networked process control systems, it is the random time delays and packet loss that lead to stochastic tracking errors for closed-loop systems besides random disturbances. As such, it is natural to represent networked process control systems as a stochastic control framework.The main objective of this work is to propose a two-level hierarchical control system based on the PID-like neural network adaptive scheme for networked process control systems. The tracking error entropy of closed-loop control systems is utilized to construct the performance index so as to update the weights of the neural PID controller. The rest of this paper is organized as follows. Section2 investigates the stochastic characteristics in networked process control systems and formulates the control problem and objective. The reason for selecting the minimum entropy of the closed-loop tracking error as a performance index is then fully described. Section 3 presents the neural PID control law for process control systems with network-induced time delays and packet dropout, and, then, the convergent condition in the mean square sense of the proposed controller is presented and proved. Section 4 shows the application of the proposed controller to a networked chemical process control system, and Section 5 concludes this paper.


## 2. Stochastic Characteristics of NCS in Operation Processes

A networked process control system with a two-level hierarchical control architecture shown in Figure1, in which a process plant is controlled by a PID controller at the plant site and the controller can also be self-tuned by a neural network at the remote site, is proposed to improve the performance of the closed-loop system. In this system, the output signal y(k) of the plant is synchronously measured with ideal sampler S at a sampling rate 1/T. The digital controller uses the information of the plant transmitted through the networks to update the new set of the control parameters. The parameter values transmitted through networks and held by a zero-order holder H are then transmitted to drive the plant. y-(k)∈R1 is the input to the controller. r(k) is the setpoint. NCS operates over a network; data transfers between the controller and the remote system will inevitably introduce network delays; that is, the output quality y(k) from the current time point k is often unavailable before the measurement is sent to the controller at the current time point due to network delays. Likewise, the control input u(k) from the current time point k is also unavailable after the control action is calculated at the current time point. τca denotes the induced delay between the controller and the actuator and τsc stands for the induced delay between the sensor and the controller.Figure 1
Networked process control system.The controlled process is nonlinear no matter whether it is a continuous or batch process. The PID controller control variableu(k) at the plant site is
(1)u=us+(KPe+KI∫edt+KDdedt),
where us is the bias value. e(k)=r(k)-y(k) is the output (y(k)) error deviated from the setpoint. KP, KI, and KD are known as the proportional gain, the integral time constant, and derivative time constant, respectively. In Figure 1, the control structure is similar to an adaptive control structure, but the parameters of the PID controller are adjusted via the backpropagation network control system. The control parameters KP, KI, and KD transmitted through networks and held by a zero-order holder H are
(2)KP(t)=KP(kT)KI(t)=KI(kT)forkT≤t<kT+TKD(t)=KD(kT)
And they are then transmitted to the controller to drive the plant.In networked control systems, a continuous signal is sampled, encoded in digital format, transmitted over the network, and finally decoded at the receiver’s side. The overall delay between sampling and eventual decoding at the receiver can be highly variable because both the network access delays and the transmission delays depend on highly variable network conditions, such as congestion and channel quality [19]. In a networked control system, message transmission delay can be divided into two parts: device delay and network delay. The device delay includes the time delay at the source and the destination nodes. The time delay at the source node includes the preprocessing time, Tpre, and the waiting time, Twait. The time delay at the destination node is only the postprocessing time, Tpost. The network time delay includes the total transmission time of a message, Ttx, and the propagation delay of the network (Tp). The total time delay can be expressed as [20]
(3)Tdelay=Tpre+Twait+Tpost+Ttx+Tp.
The waiting time Twait may be significant due to the amount of data sent by the source node, the transmission protocol, and the traffic on the network. Twait is random in most cases. The postprocessing time Tpost is negligible in a networked control system compared with other time delays. The above delays in networked control systems can be obtained by time-stamping techniques although they are normally not available [5].In the networking area, random network delays have been modeled by using various formulations based on probability and the characteristics of sources and destinations [21]. In most cases, the network-induced time delays are random and are not Gaussian. Packet dropouts result from transmission errors in physical network links, which is far more common in wireless than in wired networks, or from buffer overflows due to congestion. Long transmission delays sometimes result in packet reordering, which essentially amounts to a packet dropout if the receiver discards “outdated” arrivals [22]. The packet loss is another random factor in networked control systems. This means that in general the system output will be very unlikely to be a Gaussian noise, leading to a non-Gaussian closed-loop tracking error when the control input is applied to the system.  Thus, the randomness measure of the tracking error via the use of variance would not be sufficient in characterizing the performance of the closed-loop system. Therefore, the problems considered in this paper can be formulated to design a proper controller such that the shape of the tracking error of NCSs is made as narrow as possible.


## 3. Design of an NCS Controller

The output variable of the controlled processes over the communication system is subjected to the network delay of the uncertain duration and/or stochastic disturbances. It can be treated as a random variable that follows a specific PDF. To adjust the shape of PDF, the entropy is used here. Entropy has a more general meaning to arbitrary random variables than mean or variance. When entropy is minimized, all the moments of the error PDF (not only the second moment) are constrained [22]. Consequently, it can be used to measure and form a design criterion for general dynamic stochastic systems subject to arbitrary random inputs whose PDF can be of any shape. The application of entropy in control and estimation is developed in this section.


### 3.1. Estimation of Error PDF and Its Entropy

In practical applications, PDF of the tracking error in NCSs is often unknown a priori. A data-driven method that can get the error PDFγ^e(z) from the error samples ek is employed, where z represents the current error, which is a random variable. The error data are obtained by subtracting the process output as actually received by the controller from the setpoint in a given time period. Using the Parzen method [22], the error PDF γ^e(z) is approximated as the weighted sum of the K Gaussian kernel function defined as κ(z,σ2)=(1/2πσ)exp(-(z-e(k))2/2σ2) with the mean located at the error data point e(k) as
(4)γ^e(z)=1K∑k=1Kκ(z-e(k),σ2),
where the multidimensional Gaussian function with a radially symmetric variance σ2 is used here for simplicity. For minimizing the range of the error and maximizing the concentration of the error about mean of the error PDF at the same time, the entropy of the error should be minimized. The entropy (H) of the error PDF can be calculated using Shannon entropy
(5)HS(e)=-∫-∞∞γe(z)logγe(z)dz.
When Shannon entropy definition is used along with this PDF estimation, the calculation of Shannon entropy becomes very complex, whereas the calculation of Rényi quadratic entropy leads to a simpler form. Rényi’s entropy with order S of the tracking error is given by
(6)H(e)=11-Slog∫-∞∞γeS(e)de.


### 3.2. General Minimum Entropy Based Tuning Neural PID Network

Unlike the conventional deterministic controller design seeking a control input that minimizes the difference between the process output and the desired output at the next time step, the goal of the proposed controller design is to determine control parameters that minimize the shape of error PDF, for the output over NCS is stochastic. The objective is expressed as(7)minKP,KI,KDH=-minKP,KI,KDlog∫γe2(z)dz=-minKP,KI,KDlogV,
where the order of Rényi’s entropy is set to be S=2 owing to the computational efficiency. The above objective function is calculated for the present instant time point k only. Vk is information potential (IP) of quadratic Rényi’s entropy at the time point k. It can be calculated in closed form from the samples using Gaussian kernels as
(8)Vk=∫γe2(z)dz≅∫(1N∑i=1Nκ(z-ei,σ2))2dz≅1N2∑i=1N∑j=1Nκ(ei-ej,2σ2).Although the PID controller is widely utilized because of the simple structure and the intuitive physical meanings, it is difficult to tune proper PID parameters in NCSs owing to induced random delays. Consequently, in this research, a self-tuning neural PID control scheme is proposed in Figure2, where a neural network is used to tune the parameters of the conventional PID controller similar to the adjustment made by an experienced operator. Like the human operator who has accumulated past experience and knowledge on the controlled system and knows how to do the adjustment, the neural network can store such information and digest it for the final adjustment because neural networks have the function approximation abilities, learning abilities, and versatilities that can adapt to unknown environments. Thus, the operator’s experience and knowledge can be included into a neural network and the network is trained based on the past data history. Finally, the trained neural network can be used as means to tune the PID controller parameters online.Figure 2
Structure of a neural network in NCS.Using a three-layered neural network shown in Figure2, the learning rule that finds the suitable PID parameters is realized. Thus, in the present neural network values, the setpoint r(k), the current error e(k), the induced delay between the controller and the actuator τca, and the induced delay between the sensor and the controller τsc are denoted by O1(1),  O2(1)O3(1), and O4(1). The outputs at the output layer are KP, KI, and KD, all of which are denoted by O1(3),  O2(3), and O3(3). Then the outputs of a single hidden layer neural network can be computed via the following steps.The input layer is(9)O0(1)=1,O1(1)=rk,O2(1)=ek,O3(1)=τca,O4(1)=τsc.The input and the output of the hidden layer are(10)netp(2)(k)=∑q=04wpq(2)Oq(1)(k),p=1,2,…,POp(2)(k)=f(netp(2)(k)),p=1,2,…,PO0(2)(k)=1,
where P is the number of hidden neurons. The input and the output of the output layer are
(11)netl(3)(k)=∑p=0Pwlp(3)Op(2)(k),l=1,2,3Ol(3)(k)=g(netl(3)(k)),l=1,2,3KP=O1(3)(k),KI=O2(3)(k),KD=O3(3)(k),
where wpq and wlp are weights trained in the neural networks. The initial values of the weight are selected from the uniform distribution across a small range [-11]. f(x)=(ex-e-x)/(ex+e-x) and g(x)=ex/(ex+e-x), known as the activation functions, perform the weighted sum of the incoming signals from the input layer and the hidden layer, respectively. With the input layer and the hidden layer, the bias is introduced into the first node.In order to derive the self-tuning algorithm of the PID controller, the cost function defined in (7) should be minimized. Based on the steepest descent approach, at the output layer, we have
(12)Δwlp(k)=wlp(k)-wlp(k-1)=-η1∂H(e(k))∂wlp⟺η∂Vk∂wlp,
Consider the following: where η1 or η is the learning factor. 								(13)∂[V(e(k))]∂wlp=12K2σ2∑k1=1K∑k2=1K(e(k2)-e(k1))κ(e(k1)-e(k2),2σ2)×[∂e(k1)∂wlp-∂e(k2)∂wlp](14)∂e(k)∂wlp=-∂y(k)∂wlp=-∂y(k)∂u(k)∂u(k)∂Ol(3)∂Ol(3)∂netl(3)∂netl(3)∂wlpk=1,2,…,K,
where IP shown above is computed using the measurement errors of the past samples. The Jacobian information ∂y(k)/∂u(k) can be replaced by sgn(∂y(k)/∂u(k)) or it can be calculated by the model prediction algorithm of the plant under the assumption of u(k-i)≈u(k-i+1) when the proper sampling of the control system is applied as
(15)∂y(k)∂u(k)=∂y(k)∂u(k-τcak)∂u(k-τcak)∂u(k)=∂y(k)∂u(k-τcak)∂u(k-1)∂u(k)×∂u(k-2)∂u(k-1)⋯∂u(k-τcak)∂u(k-τcak+1)≈∂y(k)∂u(k-τcak).
Also,
(16)∂u(k)∂Ol(3)={e(k)-e(k-1)l=1e(k)l=2e(k)-2e(k-1)+e(k-2)l=3∂Ol(3)∂netl(3)=g′(netl(3)(k))=Ol(3)(1-Ol(3))∂netl(3)∂wlp=Op(2).At the hidden layer,(17)Δwpq(k)=wpq(k)-wpq(k-1)=η∂Vk∂wpq(18)∂[Vk]∂wpq=12K2σ2∑k1=1K∑k2=1K(e(k2)-e(k1))κ(e(k1)-e(k2),2σ2)×[∂e(k1)∂wpq-∂e(k2)∂wpq].
Using the chain rule, ∂e(k1)/∂wpq and ∂e(k2)/∂wpq can be expressed as
(19)∂e(k)∂wpq(k)=-∂y(k)∂wpq(k)=-∂y(k)∂u(k)∂u(k)∂wpq(k)=-∂y(k)∂u(k)∑l∂u(k)∂Ol(3)(k)∂Ol(3)(k)∂netl(3)(k)∂netl(3)(k)∂Op(2)(k)×∂Op(2)(k)∂netp(2)(k)∂netp(2)(k)∂wpq(k)k=1,2,…,K,
where
(20)∂netl(3)(k)∂Op(2)(k)=wlp(3)∂Op(2)(k)∂netp(2)(k)=f′(netp(2)(k))=Op(2)(1-Op(2))∂netp(2)(k)∂wpq(k)=Oq(1)(k).
Although IP (8) calculation can be done through the Monte Carlo method, it is preferred to find a more practical solution using sliding windowing. Supposing that the estimate Vk(e) for the tracking error has been obtained at the kth run, the recursive estimation algorithm for Vk(e) of the tracking error can be obtained by using the new sample e(k+1) as follows:
(21)Vk+1(e)≅(1-λ)Vk(e)+λL∑i=k-L+1kκ(e(i)-e(k+1),2σ2),
where  0≤λ<1 is the forgetting factor. L is the window length. The size of window L should be selected to contain the dynamic characteristics of the plant and network induced delays. Note that even if the sliding windowing is used to produce the information potential, the objective function is still optimized for the present instant k only.


### 3.3. Online Updating PID Controller Based on SPC Monitoring Chart

The conventional control parameter adjustment is triggered on the noise-free or noise-filtered data. However, it has the disadvantage of postponing the parameter adjustment until after an influent transient has subsided. Noise, on the other hand, can induce unjustified control parameter changes, which would, in turn, induce incorrect control action. Process variations in our problem may indeed be a classical electronic type of instrumentation noise and/or packet loss caused by the communications on the employed networks, but they can also be successive, small but real transient in the process output commonly because of external disturbance changes. Thus, in the past, process variations were classified into the common cause and assignable cause variations in manufacturing and service industries. A common cause variation is inherent in the process and is impossible or hard to be eliminated. It is assumed that the sample comes from a known probability distribution. Based on Shewhart's classification, Deming (1982) argued the special cause of variations as “something special, not part of the system of common causes” [23] should be identified and removed from the root. If an error value is calculated from noisy measurements, the variable error will be a random distribution variable.Even if controlled processes are well designed and carefully maintained, a certain amount of inherent natural variability is unavoidable. The mechanism that assesses the performance of the control loop for the current operation is the control chart. The chart summarizes the results obtained for the process conducted under the minimum variance. To detect if there is deviation from the minimum bounds in the current operations, a statistical hypothesis testing approach can be applied to the control output error at each time point. If the controlled process is at a steady state, one can conventionally calculate the error distribution function (4). Based on the traditional statistical process control (SPC) method, the upper (U) and the lower (L) control limits for the error are
(22)∫-∞Lγ^e(z)dz=α2,∫U+∞γ^e(z)dz=α2,
where α represents a probability of falsely rejecting the hypothesis when, in fact, it should be accepted. The selected area α under the normal operation is quite small. In this study, α is 0.05. L and U limits of the integral indicate that the desired value of the probability density function will be between L and U with the integral value (∫LUγ^e(z)dz) is equal to 1-α (22). Thus, whenever the process is around the setpoint, it is highly unlikely that the discrepancy between the observed and the setpoint values will be out of the lower and the upper limits. One aspect of the SPC philosophy is to accept the normal process variability without reporting any change in value unless the cause is identified [24, 25]. Equation (22) is numerically tractable using the probability model to calculate the thresholds (L and U) and detect any departure of the process from its standard behavior. Applying the above control limits to the measured error variable, one would retain the PID controller parameters in the local plant until there is statistically sufficient evidence that the error value is changed. This means that an SPC control chart is superimposed to monitor when the controlled system departures from the target so that the value of the NN-PID controller parameters can be regulated.Therefore, the proposed two-level hierarchical control architecture can be summarized by the following steps.Step 1.
Calculate the tracking error in NCSs{e(k1),e(k2),…,e(kN)} and IP based on the sliding windowing using (21).Step 2.
Train neural networks offline based on the historical data and the conventional turning method, like ITAE (integral of the time-weighted absolute error). Of course, the historical data should cover the desired operation region. A good initial model can be achieved. Also, use (21) to construct L and U control limits. Then the weights of the neural networks are updated by (12) and (17).Step 3.
Check whether the controlled variable is “in statistical control” or not. If the process that is operating with only chance causes of variation present is “in statistical control,” keep the same controller parameters; otherwise, updateKP, KI, and KD obtained via neural networks.Step 4.
Formulate the control inputu(k) by (1) and apply the control input u(k) to actuate the plant, set k←k+1, and repeat Step 1.Note  1. Neural network adjusts the parameters of the controller in terms of the tracking error. The changes of the tracking error cannot be obtained soon when the operating point changes or the disturbance occurs. Nevertheless, the changes of the tracking error can be obtained eventually after a period of transmitting time due to the closed-loop feedback.Note  2. The kernel size (the window width of the Parzen estimator) can be set experimentally after a preliminary analysis of the dynamic range of the tracking error. Some clustering analysis techniques can be applied to get the proper kernel size.To ensure that the proposed algorithm can track the desired output of the controlled system, the convergence condition for the proposed algorithm should be satisfied and is derived as follows.Theorem 1.
If the learning factor satisfies0<η<1/maxi|λi|, the weights of the neural networks are convergent in mean square sense.The weights of the neural networks can be rearranged in a line and denoted as a vectorW. Equations (12) and (17) can be summarized as follows:
(23)W(k+1)=W(k)+η∇V(W(k)),
where ∇V(W(k)) is the gradient of Vk given in (13) or (18) evaluated at W(k). Perform the Taylor series expansion of the gradient ∇V(W(k)) around the optimal weight vector W* as
(24)∇V(W(k))=∇V(W*)+∂∇V(W*)∂W(W-W*).Define a new weight vector spaceW-=W-W* and yield
(25)W-(k+1)=[I+ηR]W-(k),
where the Hessian matrix R=∂∇V(W*)/2∂W=∂2V(W*)/2∂W2. Let Ω=Q-TW-, and Q- is the orthonormal matrix consisting of the eigenvalues of R. Thus,
(26)Ω(k+1)=[I+ηΛ]Ω(k),
where Λ is the diagonal eigenvalue matrix with entries ordered in correspondence with the ordering in Q-. It yields
(27)Ωi(k+1)=[1+ηλi]Ωi(k),i=1,2,…,P(Q+1)+3(P+1).
Define ΔΩi(k)=Ωi(k+1)-Ωi(k). Since the eigenvalues of the Hessian matrix R are negative, |1+ηλi|<1 can guarantee that the ith weight of the neural networks obtains a stable dynamics in mean square sense. Therefore, the following inequality can ensure that the proposed algorithm is convergent in mean square sense:
(28)0<η<1maxi|λi|.This means(29)limk→∞E{ΔΩi2(k)}=0,
where E(·) is mathematical expectation.


## 3.1. Estimation of Error PDF and Its Entropy

In practical applications, PDF of the tracking error in NCSs is often unknown a priori. A data-driven method that can get the error PDFγ^e(z) from the error samples ek is employed, where z represents the current error, which is a random variable. The error data are obtained by subtracting the process output as actually received by the controller from the setpoint in a given time period. Using the Parzen method [22], the error PDF γ^e(z) is approximated as the weighted sum of the K Gaussian kernel function defined as κ(z,σ2)=(1/2πσ)exp(-(z-e(k))2/2σ2) with the mean located at the error data point e(k) as
(4)γ^e(z)=1K∑k=1Kκ(z-e(k),σ2),
where the multidimensional Gaussian function with a radially symmetric variance σ2 is used here for simplicity. For minimizing the range of the error and maximizing the concentration of the error about mean of the error PDF at the same time, the entropy of the error should be minimized. The entropy (H) of the error PDF can be calculated using Shannon entropy
(5)HS(e)=-∫-∞∞γe(z)logγe(z)dz.
When Shannon entropy definition is used along with this PDF estimation, the calculation of Shannon entropy becomes very complex, whereas the calculation of Rényi quadratic entropy leads to a simpler form. Rényi’s entropy with order S of the tracking error is given by
(6)H(e)=11-Slog∫-∞∞γeS(e)de.


## 3.2. General Minimum Entropy Based Tuning Neural PID Network

Unlike the conventional deterministic controller design seeking a control input that minimizes the difference between the process output and the desired output at the next time step, the goal of the proposed controller design is to determine control parameters that minimize the shape of error PDF, for the output over NCS is stochastic. The objective is expressed as(7)minKP,KI,KDH=-minKP,KI,KDlog∫γe2(z)dz=-minKP,KI,KDlogV,
where the order of Rényi’s entropy is set to be S=2 owing to the computational efficiency. The above objective function is calculated for the present instant time point k only. Vk is information potential (IP) of quadratic Rényi’s entropy at the time point k. It can be calculated in closed form from the samples using Gaussian kernels as
(8)Vk=∫γe2(z)dz≅∫(1N∑i=1Nκ(z-ei,σ2))2dz≅1N2∑i=1N∑j=1Nκ(ei-ej,2σ2).Although the PID controller is widely utilized because of the simple structure and the intuitive physical meanings, it is difficult to tune proper PID parameters in NCSs owing to induced random delays. Consequently, in this research, a self-tuning neural PID control scheme is proposed in Figure2, where a neural network is used to tune the parameters of the conventional PID controller similar to the adjustment made by an experienced operator. Like the human operator who has accumulated past experience and knowledge on the controlled system and knows how to do the adjustment, the neural network can store such information and digest it for the final adjustment because neural networks have the function approximation abilities, learning abilities, and versatilities that can adapt to unknown environments. Thus, the operator’s experience and knowledge can be included into a neural network and the network is trained based on the past data history. Finally, the trained neural network can be used as means to tune the PID controller parameters online.Figure 2
Structure of a neural network in NCS.Using a three-layered neural network shown in Figure2, the learning rule that finds the suitable PID parameters is realized. Thus, in the present neural network values, the setpoint r(k), the current error e(k), the induced delay between the controller and the actuator τca, and the induced delay between the sensor and the controller τsc are denoted by O1(1),  O2(1)O3(1), and O4(1). The outputs at the output layer are KP, KI, and KD, all of which are denoted by O1(3),  O2(3), and O3(3). Then the outputs of a single hidden layer neural network can be computed via the following steps.The input layer is(9)O0(1)=1,O1(1)=rk,O2(1)=ek,O3(1)=τca,O4(1)=τsc.The input and the output of the hidden layer are(10)netp(2)(k)=∑q=04wpq(2)Oq(1)(k),p=1,2,…,POp(2)(k)=f(netp(2)(k)),p=1,2,…,PO0(2)(k)=1,
where P is the number of hidden neurons. The input and the output of the output layer are
(11)netl(3)(k)=∑p=0Pwlp(3)Op(2)(k),l=1,2,3Ol(3)(k)=g(netl(3)(k)),l=1,2,3KP=O1(3)(k),KI=O2(3)(k),KD=O3(3)(k),
where wpq and wlp are weights trained in the neural networks. The initial values of the weight are selected from the uniform distribution across a small range [-11]. f(x)=(ex-e-x)/(ex+e-x) and g(x)=ex/(ex+e-x), known as the activation functions, perform the weighted sum of the incoming signals from the input layer and the hidden layer, respectively. With the input layer and the hidden layer, the bias is introduced into the first node.In order to derive the self-tuning algorithm of the PID controller, the cost function defined in (7) should be minimized. Based on the steepest descent approach, at the output layer, we have
(12)Δwlp(k)=wlp(k)-wlp(k-1)=-η1∂H(e(k))∂wlp⟺η∂Vk∂wlp,
Consider the following: where η1 or η is the learning factor. 								(13)∂[V(e(k))]∂wlp=12K2σ2∑k1=1K∑k2=1K(e(k2)-e(k1))κ(e(k1)-e(k2),2σ2)×[∂e(k1)∂wlp-∂e(k2)∂wlp](14)∂e(k)∂wlp=-∂y(k)∂wlp=-∂y(k)∂u(k)∂u(k)∂Ol(3)∂Ol(3)∂netl(3)∂netl(3)∂wlpk=1,2,…,K,
where IP shown above is computed using the measurement errors of the past samples. The Jacobian information ∂y(k)/∂u(k) can be replaced by sgn(∂y(k)/∂u(k)) or it can be calculated by the model prediction algorithm of the plant under the assumption of u(k-i)≈u(k-i+1) when the proper sampling of the control system is applied as
(15)∂y(k)∂u(k)=∂y(k)∂u(k-τcak)∂u(k-τcak)∂u(k)=∂y(k)∂u(k-τcak)∂u(k-1)∂u(k)×∂u(k-2)∂u(k-1)⋯∂u(k-τcak)∂u(k-τcak+1)≈∂y(k)∂u(k-τcak).
Also,
(16)∂u(k)∂Ol(3)={e(k)-e(k-1)l=1e(k)l=2e(k)-2e(k-1)+e(k-2)l=3∂Ol(3)∂netl(3)=g′(netl(3)(k))=Ol(3)(1-Ol(3))∂netl(3)∂wlp=Op(2).At the hidden layer,(17)Δwpq(k)=wpq(k)-wpq(k-1)=η∂Vk∂wpq(18)∂[Vk]∂wpq=12K2σ2∑k1=1K∑k2=1K(e(k2)-e(k1))κ(e(k1)-e(k2),2σ2)×[∂e(k1)∂wpq-∂e(k2)∂wpq].
Using the chain rule, ∂e(k1)/∂wpq and ∂e(k2)/∂wpq can be expressed as
(19)∂e(k)∂wpq(k)=-∂y(k)∂wpq(k)=-∂y(k)∂u(k)∂u(k)∂wpq(k)=-∂y(k)∂u(k)∑l∂u(k)∂Ol(3)(k)∂Ol(3)(k)∂netl(3)(k)∂netl(3)(k)∂Op(2)(k)×∂Op(2)(k)∂netp(2)(k)∂netp(2)(k)∂wpq(k)k=1,2,…,K,
where
(20)∂netl(3)(k)∂Op(2)(k)=wlp(3)∂Op(2)(k)∂netp(2)(k)=f′(netp(2)(k))=Op(2)(1-Op(2))∂netp(2)(k)∂wpq(k)=Oq(1)(k).
Although IP (8) calculation can be done through the Monte Carlo method, it is preferred to find a more practical solution using sliding windowing. Supposing that the estimate Vk(e) for the tracking error has been obtained at the kth run, the recursive estimation algorithm for Vk(e) of the tracking error can be obtained by using the new sample e(k+1) as follows:
(21)Vk+1(e)≅(1-λ)Vk(e)+λL∑i=k-L+1kκ(e(i)-e(k+1),2σ2),
where  0≤λ<1 is the forgetting factor. L is the window length. The size of window L should be selected to contain the dynamic characteristics of the plant and network induced delays. Note that even if the sliding windowing is used to produce the information potential, the objective function is still optimized for the present instant k only.


## 3.3. Online Updating PID Controller Based on SPC Monitoring Chart

The conventional control parameter adjustment is triggered on the noise-free or noise-filtered data. However, it has the disadvantage of postponing the parameter adjustment until after an influent transient has subsided. Noise, on the other hand, can induce unjustified control parameter changes, which would, in turn, induce incorrect control action. Process variations in our problem may indeed be a classical electronic type of instrumentation noise and/or packet loss caused by the communications on the employed networks, but they can also be successive, small but real transient in the process output commonly because of external disturbance changes. Thus, in the past, process variations were classified into the common cause and assignable cause variations in manufacturing and service industries. A common cause variation is inherent in the process and is impossible or hard to be eliminated. It is assumed that the sample comes from a known probability distribution. Based on Shewhart's classification, Deming (1982) argued the special cause of variations as “something special, not part of the system of common causes” [23] should be identified and removed from the root. If an error value is calculated from noisy measurements, the variable error will be a random distribution variable.Even if controlled processes are well designed and carefully maintained, a certain amount of inherent natural variability is unavoidable. The mechanism that assesses the performance of the control loop for the current operation is the control chart. The chart summarizes the results obtained for the process conducted under the minimum variance. To detect if there is deviation from the minimum bounds in the current operations, a statistical hypothesis testing approach can be applied to the control output error at each time point. If the controlled process is at a steady state, one can conventionally calculate the error distribution function (4). Based on the traditional statistical process control (SPC) method, the upper (U) and the lower (L) control limits for the error are
(22)∫-∞Lγ^e(z)dz=α2,∫U+∞γ^e(z)dz=α2,
where α represents a probability of falsely rejecting the hypothesis when, in fact, it should be accepted. The selected area α under the normal operation is quite small. In this study, α is 0.05. L and U limits of the integral indicate that the desired value of the probability density function will be between L and U with the integral value (∫LUγ^e(z)dz) is equal to 1-α (22). Thus, whenever the process is around the setpoint, it is highly unlikely that the discrepancy between the observed and the setpoint values will be out of the lower and the upper limits. One aspect of the SPC philosophy is to accept the normal process variability without reporting any change in value unless the cause is identified [24, 25]. Equation (22) is numerically tractable using the probability model to calculate the thresholds (L and U) and detect any departure of the process from its standard behavior. Applying the above control limits to the measured error variable, one would retain the PID controller parameters in the local plant until there is statistically sufficient evidence that the error value is changed. This means that an SPC control chart is superimposed to monitor when the controlled system departures from the target so that the value of the NN-PID controller parameters can be regulated.Therefore, the proposed two-level hierarchical control architecture can be summarized by the following steps.Step 1.
Calculate the tracking error in NCSs{e(k1),e(k2),…,e(kN)} and IP based on the sliding windowing using (21).Step 2.
Train neural networks offline based on the historical data and the conventional turning method, like ITAE (integral of the time-weighted absolute error). Of course, the historical data should cover the desired operation region. A good initial model can be achieved. Also, use (21) to construct L and U control limits. Then the weights of the neural networks are updated by (12) and (17).Step 3.
Check whether the controlled variable is “in statistical control” or not. If the process that is operating with only chance causes of variation present is “in statistical control,” keep the same controller parameters; otherwise, updateKP, KI, and KD obtained via neural networks.Step 4.
Formulate the control inputu(k) by (1) and apply the control input u(k) to actuate the plant, set k←k+1, and repeat Step 1.Note  1. Neural network adjusts the parameters of the controller in terms of the tracking error. The changes of the tracking error cannot be obtained soon when the operating point changes or the disturbance occurs. Nevertheless, the changes of the tracking error can be obtained eventually after a period of transmitting time due to the closed-loop feedback.Note  2. The kernel size (the window width of the Parzen estimator) can be set experimentally after a preliminary analysis of the dynamic range of the tracking error. Some clustering analysis techniques can be applied to get the proper kernel size.To ensure that the proposed algorithm can track the desired output of the controlled system, the convergence condition for the proposed algorithm should be satisfied and is derived as follows.Theorem 1.
If the learning factor satisfies0<η<1/maxi|λi|, the weights of the neural networks are convergent in mean square sense.The weights of the neural networks can be rearranged in a line and denoted as a vectorW. Equations (12) and (17) can be summarized as follows:
(23)W(k+1)=W(k)+η∇V(W(k)),
where ∇V(W(k)) is the gradient of Vk given in (13) or (18) evaluated at W(k). Perform the Taylor series expansion of the gradient ∇V(W(k)) around the optimal weight vector W* as
(24)∇V(W(k))=∇V(W*)+∂∇V(W*)∂W(W-W*).Define a new weight vector spaceW-=W-W* and yield
(25)W-(k+1)=[I+ηR]W-(k),
where the Hessian matrix R=∂∇V(W*)/2∂W=∂2V(W*)/2∂W2. Let Ω=Q-TW-, and Q- is the orthonormal matrix consisting of the eigenvalues of R. Thus,
(26)Ω(k+1)=[I+ηΛ]Ω(k),
where Λ is the diagonal eigenvalue matrix with entries ordered in correspondence with the ordering in Q-. It yields
(27)Ωi(k+1)=[1+ηλi]Ωi(k),i=1,2,…,P(Q+1)+3(P+1).
Define ΔΩi(k)=Ωi(k+1)-Ωi(k). Since the eigenvalues of the Hessian matrix R are negative, |1+ηλi|<1 can guarantee that the ith weight of the neural networks obtains a stable dynamics in mean square sense. Therefore, the following inequality can ensure that the proposed algorithm is convergent in mean square sense:
(28)0<η<1maxi|λi|.This means(29)limk→∞E{ΔΩi2(k)}=0,
where E(·) is mathematical expectation.


## 4. Case Studies: Air-Pressure Tank

As for dealing with highly nonlinear and time varying chemical processes, although several schemes of self-tuning PID controllers were proposed in the past, their approaches were limited to the process disturbance with the Gaussian distribution assumption. If the data collected have non-Gaussian distribution, those methods would get weird responses and they would not be able to achieve the expected performance using minimizing variance. In this case study, the comparisons between the proposed network control method and the conventional PID turning method should be made.A laboratory scale air-pressure tank system shown in Figure3 is made accessible for control over the computer network. At the local site, the tank is equipped with differential-pressure-to-current (DP/I) transducers to provide a continuous measurement of the pressure. The computer is connected to a PCI-1710 analog/digital I/O expansion card from Advantech. The expansion board uses a 12 bit converter; therefore, the digital signals are 12 bit. The analog signals from the measured pressure are amplified and conditioned EDM35 (4–20 mA/0–5 volts) modules. The embedded applications, including data acquisition software and the PID controller algorithm, are developed using MATLAB. At the remote site, the Windows operating system and TCP/IP over wireless networks that are implemented on laboratory floors to support mobility laptop are chosen. The Linux operation system and TCP/IP over Ethernet for the data storage server are implemented. The time constant of the air-pressure tank is significantly larger than the time needed during controller computation. Thus, the controller under the Microsoft Windows operating system is pretty close to the real-time environment. Furthermore, with the configuration mentioned above, it is easy to apply this test structure to the laboratory process and demonstrate queuing and buffering delays in the network system although an industrial system should be incorporated with a real-time OS using an appropriate network scheme. Moreover, in the discrete control system, if the controller can be actually acted at each fixed time point and the proper information is collected at the interval of two time points, the operating system can be treated as real-time control. The distributions of the delay between the controller and the actuator and those of the delay between the sensor and the controller are shown in Figures 4(a) and 4(b), respectively. The sensor-to-controller time delay can be calculated by the time difference between the time point when the controlled variable is measured and the time point when the controlled variable is stored on the data server. Likewise, the controller-to-actuator time delay is obtained by computing the time difference between the time point when the output neural network is sent out and the time point when the local controller receives the controller parameters. The objective is to assess the control performance in the presence of communication and measurement delays when the proposed NCS method is applied. The variation of the process output with control inputs over time will be used to evaluate the performance of the proposed method.Figure 3
Schematic diagram of the air-pressure tank control loop over the network.The distributions of (a) delay between the controller and the actuator and (b) delay between the sensor and the controller in network control of the air-pressure tank process.
(a)
(b)In this application, the sampling periodT=1 sec. The setpoints of the pressure values have been changed from 3.5 (Kg/cm2) to 2.5 (Kg/cm2) and then from 2.5 (Kg/cm2) to 3.0 (Kg/cm2). The number of nodes in the hidden layer of the neural controller is 3, the learning factor η=0.01, the kernel size used to estimate the entropy is experimentally set at σ=0.5, and the samples for neural controller are trained with a segment of L=50 at each instant. The procedures of designing the controller are implemented according to the summarized algorithm in Section 3. The proposed controller uses an instant performance index shown as (7), whereas the conventional PID controller turning method, ITAE, uses a cumulative performance index (integral means accumulating a series of data). To avoid improper initial weights of NN in our controller and produce large overshoot, NN is initially trained using the historical operation data. In this case, NN is trained before the time point 580 when the controller parameters are fixed without change. The performance using the proposed neural network PID controller is shown in Figure 5, where the 95% control limit is also included. Due to the fluctuation of the upstream pressure, the controlled pressure is still oscillatory even though the required time completes responses for the new set point change. With the new coming measurement, the PDF is regularly updated. The control limit is recalculated based on (22) when PDF is significantly changed. Thus, the control limits shown in Figure 5 are different at different time periods. In order to detect nonrandom changes in operating conditions, the superimposed rules on the chart are used to decide whether the pressure is “in statistical control” or not. The following decision-making rules are adopted [26] when the control parameters are updated:eight consecutive points increasing in value;eight consecutive points decreasing in value;three consecutive points above the control limit;three consecutive points below the control limit;Figure 5
Network PID controller in setpoint change results in the air-pressure tank process: pressure (dashed line) and setpoint (solid line). The dotted line represents the 95% control limits.The control parameters shown in Figure6 are calculated with the new coming measurement, but the PID controller is not regularly updated until the pressure is significantly changed. Using the proposed rule to automatically adjust the control parameters, the controlled results will not aggressively react to the random changes beforehand if the pressure is not “in statistical control.”Network PID controller parameters in the air-pressure tank process: (a)KP; (b) KI; (c) KD.
(a)
(b)
(c)If the response to the controlled system with the same controller parameters is tested, the performance is shown in Figure7. Note that the controller parameters are computed based on the process reaction curve [27]. First, based on the minimum ITAE tuning formula, the controller parameters are computed when the system is operated at the pressure around 3.5 (Kg/cm2). Figure 7 indicates that after the setpoint changes, the fixed tuning parameters result in a larger overshoot than the proposed neural network PID controller shown in Figure 5. The large oscillations in the process output are caused by the oscillatory control inputs. On the contrary, the neural network PID controller of the process output is less oscillatory and reaches the desired setpoint much more quickly. To clearly illustrate the improvement of the performance, Figure 8 shows the comparison of the error distributions between PID and neural network PID controllers at time point 1,500 where the set point has been changed for a while. The error distribution of the neural network PID controller is sharper and narrower after the controlled process reaches the desired set point. To clearly illustrate the improvement of the PDF distribution based on the proposed method, PDF at time points 1,250, 1,350, and 1,500 is replotted in Figure 9. It is shown that the PDF distribution obtained from the last time points is narrower and sharper than that from the early time points. As would be expected, the adaptive tuning parameters are needed for the nonlinear air-pressure tank process.Figure 7
The PID controller with the fixed controller parameters in setpoint change results in the air-pressure tank process: pressure (dashed line) and setpoint (solid line).  The dotted line represents the 95% control limits.Figure 8
The estimated error distributions of the PID controller (dashed line) and the network PID controller (solid line) in the air-pressure tank process.Figure 9
Plot of PDF for the time point at 1,250, 1,350, and 1,500 in the air-pressure tank process.


## 5. Conclusions

In recent years, there has been a growing interest in the study of the effects of NCSs mainly because of the broad use in NCSs. NCSs can provide Web clients a platform not only for remote monitoring of the current behavior of the operation plants but also for remote control of the plant. These advantages make NCSs applicable to many fields, including spacecraft automotive, remote robots, and manufacturing processes, but not much work has been done in the process control field so far. In a networked environment, the communication delays (also called network-induced time delays) and packet dropouts (probabilistic information missing, missing measurement) are two main problems that would degrade the system performance. In most relevant literature, the network-induced time delays have been commonly assumed to be deterministic, which is fairly unrealistic as, by nature, delays resulting from network transmissions are typically time varying and random.In this paper, the proposed neural network PID controller for the controlled processes with the communication delays in various probabilistic ways is developed. In comparison with the past work, the merits of the proposed method are as following.(i)
Instead of Monte Carlo that generates a set of tracking errors at the instant time point, the proposed method uses the sliding windowing technique to obtain the estimation of PDF, entropy, and information potential of the tracking error. Then the design of the controller is straightforward based on minimizing the entropy of the tracking error or maximizing the information potential of the tracking error. This is a more practical solution.(ii)
Due to the stochastic behavior, the SPC monitoring chart is introduced to solve practical problems. It improves the control performance of the nonlinear process in the network. The implementation is much simpler and less computationally demanding. The control strategy is the balance between the network delays and the conventional nonlinear control designs.(iii)
To verify the proposed practice-oriented algorithm, one practical case is conducted in an air-pressure tank. The results show that the proposed method is effective in following the setpoint and reducing the variance of the output caused by unknown stochastic network delays.This strategy is particularly suitable for the industrial processes that is characterized by high nonlinearity and analytical difficulty in network process control. Because of communication delay, packet dropouts, noises, and disturbance, the tracking error may not obey the Gaussian distribution, the Rényi quadratic entropy of the tracking error is used to design the neural network PID controller for networked operation processes. Hence, the method proposed in this paper can be directly extended to deal with the controller design of the NCS with nonlinear plants, noises, and random delays. All the results presented in this paper yield encouraging outcomes in the single variable process, but sensitivity analysis on different initial PID parameters is not considered. The extension of the neural network PID to the sensitivity problem using theoretic analysis and MIMO systems is needed in the future.



---

*Source: 752489-2013-04-23.xml*