# Direct Data-Driven Control for Cascade Control System

Authors: Hong Jianwang; Ricardo A. Ramirez-Mendoza; Tang Xiaojun
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

This paper combines system identification, direct data-driven control, and optimization algorithm to design two controllers for one cascade control system, that is, the inner controller and the outer controller. More specifically, when these two controllers in the cascade control system are parameterized by two unknown parameter vectors, respectively, the problem of controller design is changed to parameter identification. To avoid the modeling process for the unknown plants in the cascade control system, a direct data-driven control scheme is proposed to identify those two parameter vectors through minimizing two optimization problems, which do not need any knowledge of the unknown plants. Furthermore, the detailed first-order gradient algorithm is applied to solve our constructed optimization problems, and its convergence property is also analyzed. To extend the above idea to design a nonlinear controller in the cascade control system, a direct data-driven scheme is proposed to get one optimal nonlinear controller, by using some spectral knowledge. Finally, one simulation example of flight simulation is used to prove the efficiency of our proposed direct data-driven control for the cascade control system.


---

## Body

## 1. Introduction

The main mission of advanced control theory is to design a detailed controller in an open-loop or closed-loop structure so that this designed controller can drive the output of a plant to track an expected set point or to satisfy a given target. Two categories exist for controller design, that is, the model-based approach and the data-driven approach. Considering the first model-based approach, a mathematical model of the considered plant is required for the next controller design. It tells that no mathematical model means no controller. Constructing the corresponding mathematical model for the unknown plant is very necessary for this first type of model-based approach, and it is also the most difficult step, as it needs some knowledge of other subjects, such as probability theory and linear and nonlinear system theory. This modeling process corresponds to model identification or system identification, which is adopted to obtain the mathematical model exploiting measured data from the experiment on the considered open-loop or closed-loop system. The whole steps for system identification include four main steps, namely, model structure selection, optimal input design, parameter estimation, and model validation. These four steps are implemented iteratively until getting one satisfying model, so system identification is the first step or premise for the next controller design, that is, the idea of identification for control.Usually, trying to apply different system identification strategies to produce a mathematical model, this mathematical model may be high order and high property of nonlinearity; then, it leads to controllers with high order and high nonlinearity. Thus, due to the controllers with high nonlinearity, one extra controller reduction procedure is added in the practical application, because the complex controllers are difficult or costly to design and implement. Generally, the obtained controller, designed by the model-based approach, depends on the identified model for the unknown plant. It means the above four identification processes are repeated again and again while guaranteeing that the identified model can be used to replace the original plant perfectly.To alleviate the dependence on the identified model for the controller, the notion of the data-driven approach is widely studied in recent years. The attracting property of the data-driven approach is that the controller is designed directly based on measured data. As the data-driven approach is still in its infancy, different names are given in the references to describe it, such as data-driven, data-based, model-free. To the best of our knowledge, the principles between the data-driven approach and system identification are similar to each other, as the measured data are applied to get the mathematical model for the unknown plant in the framework of system identification, but they get the approximated controller for the case of the data-driven approach. The idea of direct data-driven control was first proposed in machine learning; then, it attracted many researchers in the advanced control field recently. Now, this data-driven theory is widely applied in the control field, for example, direct data-driven control, data-driven estimation, data-driven detection, data-driven optimization, and so on. The common property among them is that the measured data are used to achieve our main goals; then, it means some useful information is extracted from these measured data. On the other hand, the data-driven approach needs lots of measured data; that is, the number of measured data is sufficiently large. This requirement is feasible in our information period, and the data-driven approach was born to overcome the limitation of the model-based approach, so the data-driven approach is studied very popularly from theory and practice application.Due to the application of the data-driven approach widely in the control field and the similar point between the data-driven approach and system identification, we call their combination as identification for control, that is, system identification for direct data-driven control. More specifically, we describe a concise introduction or contribution on system identification for direct data-driven control, which belongs to the data-driven approach. In case of the unknown but bounded noise, one bounded error identification is proposed to identify the unknown systems with time-varying parameters. Then, one feasible parameter set is constructed to include the unknown parameter with a given probability level. In [1], the feasible parameter set is replaced by one confidence interval, as this confidence interval can accurately describe the actual probability that the future predictor will fall into the constructed confidence interval. The problem about how to construct this confidence interval is solved by a linear approximation/programming approach [2], which can identify the unknown parameter only for the linear regression model. According to the obtained feasible parameter set or confidence interval, the midpoint or center can be deemed as the final parameter estimation; further, a unified framework for solving the center of the confidence interval is modified to satisfy the robustness. This robustness corresponds to other external noises, such as outlier, unmeasured disturbance [3]. The above-mentioned identification strategy, used to construct one set or interval for an unknown parameter, is called the set membership identification, dealing with the unknown but bounded noise. There are two kinds of descriptions of external noise: one is the probabilistic description and the other is the deterministic description, corresponding to the unknown but bounded noise [4]. For the probabilistic description of external noise, the noise is always assumed to be one white noise, and its probabilistic density function (PDF) is known in advance. On the contrary for deterministic description on external noise, the only information about noise is bound, so this deterministic description can relax the strict assumption on the probabilistic description. In reality or practice, bounded noise is more common than white noise. Within the deterministic description of external noise, set membership identification is adjusted to design controllers with two degrees of freedom [5]; it corresponds to direct data-driven control or set membership control. Set membership control is applied to design feedback control in a closed-loop system with a nonlinear system in [6], where the considered system is identified by set membership identification, and the obtained system parameter will be a benefit for the prediction output. After substituting the obtained system parameter into the prediction output to construct one cost function, [7] takes the derivative of the above cost function with respect to control input to achieve one optimal input. Set membership identification can be not only applied in MC, but also in stochastic adaptive control [8], where a learning theory-kernel is introduced to achieve the approximation for nonlinear function or system. Based on the bounded noise, many parameters are also included in known intervals in prior; then, robust optimal control with adjustable uncertainty sets is studied in [9], where robust optimization is introduced to consider uncertain noise and uncertain parameters simultaneously. To solve the expectation operation with dependence on the uncertainty, the sample size of random convex programs is considered to replace the expectation by finite sum [10]. Generally, many practical problems in systems and control, such as controller synthesis and state estimation, are often formulated as optimization problems [11]. In many cases, the cost function incorporates variables that are used to model uncertainty, in addition to optimization variables, and [12] employs the uncertainty described as probabilistic variables. Reference [13] studied the data-driven output feedback controllers for the nonlinear system and applied event-triggered mode to analyze the robust stability. Data-driven estimation is used to achieve hybrid system identification [14], whose nonlinearity is described by one kernel function. During these recent years, the first author studied this direct data-driven control too, for example, the closed relation between system identification and direct data-driven control [15], and data-driven model predictive control [16]. Based on the above descriptions on direct data-driven control and our existing research about system identification, model predictive control, direct data-driven control, convex optimization theory, and so forth, our mission in this paper is to combine our previous results and apply them in practical engineering.Here, in this paper, we continue to do some research on the direct data-driven control from theory and its application. Based on our previous results on direct data-driven control for a one-closed-loop system, we see that there exist multiple-closed-loop system in real engineering, for example, flight simulation, UAV flight control, robot control, and target tracking. Although the existing results on control and identification for one single closed loop can be used in this multiple-closed-loop system, some adaptations must be needed to satisfy this special system, and no detailed direct data-driven control exists for it. As the multiple-closed-loop system is one extended case for the two-closed-loop system, it is necessary to consider the control problem for this two-closed-loop system. This two-closed-loop system has other names, such as dual control system or cascade control system; then, the cascade control system is adopted during this whole system. As two closed loops exist in the cascade control system, they correspond to one inner loop and the other outer loop. Under these circumstances, two controllers are unknown and needed to devise. In order to design these two unknown controllers, that is, the inner controller and outer controller, firstly the data-driven approach is applied to tune controllers with their parameterized structures. Indeed, in the most practical case of parameterized controllers, the problem of controller design is changed into one identification problem, corresponding to the unknown controller parameter. Generally, when the inner controller and outer controller are parameterized by two unknown parameter vectors respectively, for example PID, the data-driven approach is applied to design these two unknown parameter vectors. After substituting these two parameter vectors into their own forms, then those two controllers are obtained. During the detailed derivation process about direct data-driven control for cascade control system, two optimization problems are solved to get those unknown parameter vectors. Furthermore, one single and easy optimization algorithm—first-order gradient algorithm—is introduced to solve our constructed optimization problems, and the property of the first-order gradient algorithm is described to complete the analysis for optimization algorithm. The above descriptions about our mission on direct data-driven control for cascade control system hold in case of the parameterized controller; that is, data-driven approach is used to tune the parameter vector. Secondly, we also give some suggestions on the direct data-driven control for nonlinear controller. To deal with this complex problem without any controller parameter, we also think how to use the measured data to design the nonlinear controller directly. To the best of our knowledge, theory must serve for practical engineering, so from a practical point of view, our mentioned direct data-driven control for cascade control system is applied to control the flight simulation, whose mission is to track one given target, for example, UAV, plane, and ship. To be convenient to understand our flight simulation, its background and controller structure are explained in detail, and the most important aspect is that the cascade control system exists in the control structure for flight simulation.Generally, the main contributions of the paper are formulated as follows:(1)
It combines some different subjects, such as system identification, direct data-driven control, and optimization theory into one real practical engineering.(2)
The detailed processes for controller design are given for parameterized controller and nonlinear controller simultaneously.(3)
It paves a road to identification for control.This paper is organized as follows: in Section2, a cascade control system with two parameterized controllers is considered, and some preliminaries and physical variables are formulated, such as measured data, inner loop, and outer loop. In Section 3, a direct data-driven control scheme is proposed to design the inner controller and outer controller, existing in the cascade control system simultaneously. The control design process is changed to two mathematical optimization problems, whose decision variables correspond to those unknown parameter vectors. Then, the first-order gradient algorithm is used to solve our constructed optimization problems, and the property of this first-order gradient algorithm is analyzed too. In Section 4, direct data-driven control for two parameterized controllers is extended to direct data-driven control for the nonlinear controller, which is applied to replace the formal parameterized controller. To testify our proposed direct data-driven control for the cascade control system, one example of flight simulation is used in Section 5, where the background and the detailed control structure of flight simulation are also described to understand easily. Section 6 ends the paper with a final conclusion and points out the next subject of ongoing research. By the way, here, all the mathematical derivations in this paper are obtained through our own contributions.


## 2. Cascade Control System

Consider one cascade control system in Figure1, where two closed loops are shown without loss of generality and a noiseless environment being given.Figure 1
Cascade control system with two closed loops.In Figure1, Gq is the unknown plant in the inner loop and G′q is the unknown plant in the outer loop. Two families of linear proper controllers Ciθi=Ciq,θi,θi∈Rin and C0θ0=C0q,θ0,θ0∈R0n exist in the inner loop and outer loop, respectively. For convenience, the dimensions for parameter vectors θi and θ0 are all sent to be n. If they are not the same, then some extra elements are chosen to be zero. These two controllers Ciq,θi and C0q,θ0 are parameterized by parameter vectors θi and θ0, respectively, that is,(1)θi=θi1,θi2,…,θinT,θ0=θ01,θ02,…,θ0nT.The set of measured dataDN=ut,yit,y0tt=1N, where ut is the control variable, yit is the output of the inner loop, y0t is the output of the outer loop, and N is the amount of measured data. eit is the input signal for the inner controller, e0t is the input signal for the outer controller, the output for the outer controller is rit, and r0t is the input signal for this whole cascade control system.


### 2.1. Comment

The inner controllerCiq,θi and outer controller C0q,θ0 are related to their parameter vectors θi and θ0, that is, two linear affine forms, such as the following special forms:(2)Ciz,θi=1,q,q2,…,qnθi1θi2⋮θin,C0z,θ0=1,q,q2,…,qnθ01θ02⋮θ0n.Observing Figure1 again, two reference models Miq and M0q are given for the inner loop and outer loop, respectively, and q is the backward shift operator. Two reference signals corresponding to the inner loop and outer loop are rit and r0t. Those two reference models Miq and M0q are all prior known, so they satisfy(3)yit=Miqrit,y0t=M0qr0t.Due to the inverse property for two reference modelsMiq and M0q, two reference signals rit and r0t are(4)rit=Mi−1qyit,r0t=M0−1qy0t.In that cascade control system (Figure1), two reference models Miq and M0q are given, and set of data DN=ut,yit,y0tt=1N with ut being measured by some sensors; the problem is to design those two parameterized controllers Ciq,θi,C0q,θ0, while guaranteeing equation (1) holds in case of two unknown plants Gq and G′q.


## 2.1. Comment

The inner controllerCiq,θi and outer controller C0q,θ0 are related to their parameter vectors θi and θ0, that is, two linear affine forms, such as the following special forms:(2)Ciz,θi=1,q,q2,…,qnθi1θi2⋮θin,C0z,θ0=1,q,q2,…,qnθ01θ02⋮θ0n.Observing Figure1 again, two reference models Miq and M0q are given for the inner loop and outer loop, respectively, and q is the backward shift operator. Two reference signals corresponding to the inner loop and outer loop are rit and r0t. Those two reference models Miq and M0q are all prior known, so they satisfy(3)yit=Miqrit,y0t=M0qr0t.Due to the inverse property for two reference modelsMiq and M0q, two reference signals rit and r0t are(4)rit=Mi−1qyit,r0t=M0−1qy0t.In that cascade control system (Figure1), two reference models Miq and M0q are given, and set of data DN=ut,yit,y0tt=1N with ut being measured by some sensors; the problem is to design those two parameterized controllers Ciq,θi,C0q,θ0, while guaranteeing equation (1) holds in case of two unknown plants Gq and G′q.


## 3. Direct Date-Driven Control Scheme

To design those two parameterized controllersCiq,θi,C0q,θ0, existing in the inner loop and outer loop, respectively, we notice there are two unknown plants Gq and G′q. In the classical model-based control scheme, time is spent in identifying Gq and G′q; then the identified models are used for the next controller design. Furthermore, as two parameterized controllers Ciq,θi,C0q,θ0 are parameterized by vectors θi and θ0, to avoid the complex modeling process for unknown plants Gq and G′q, a direct data-driven control scheme is proposed to achieve our goal.


### 3.1. Inner Controller

Firstly considering the inner loop in Figure1, we have(5)yit=Miqrit,yit=Gqut=GqCiq,θieit=GqCiq,θirit−yit,where the first equation in (5) is the expected relation, and the second equation is the real relation between the input and output of the inner loop.Formulating equation (5) again, it holds that(6)yit=GqCiq,θirit1+GqCiq,θi.Comparing equations (5) and (6) and using the reference model Miq, then transfer function from rit to yit must be equal to its reference model, through tuning the parameter vector θi. It means parameter vector θi is chosen to minimize the following mathematical optimization problem:(7)J0θi=GqCiq,θi1+GqCiq,θi−Miq2,where, in optimization problem (7), 2 is the common L2 norm.


#### 3.1.1. Comment

When minimizing the optimization problem (7) to get parameter vector θi, unknown plant Gq is needed to identify firstly. This optimization problem (7) is not solvable without any knowledge about unknown plant Gq.To avoid the appearance of unknown plantGq in the optimization problem, direct data-driven control is proposed here. Using the set of measured data ut,yitt=1N, parameter vector θi is extracted from the measured data ut,yitt=1N, while avoiding the modeling process for unknown plant Gq. Observing the input signal and output signal for that unknown inner controller Ciq,θi, we have(8)ut=Ciq,θieit=Ciq,θirit−yit.Substituting equations (4) into (8), we get(9)ut=Ciq,θiMi−1qyit−yit=Ciq,θiMi−1q−1yit,where the input and output signals correspond to that measured data ut,yitt=1N and no knowledge of unknown plant Gq exists. Then, we construct the following minimization problem for the inner controller Ciq,θi:(10)θ^i=argminθiJ1θi,J1θi=1N∑t=1Nut−Ciq,θiMi−1q−1yit,where θ^i is the parameter estimation of parameter vector θi, measured data ut,yitt=1N are collected during the whole experiment, and reference model Miq is given, so only parameter vector θi is unknown in that cost function (10).Because all variables are known in cost function (10), expect for unknown parameter vector θi, parameter estimation θ^i can be obtained through minimizing that optimization problem (10). The detailed optimization algorithm, used to minimize optimization problem, will be described in Section 3.3.


### 3.2. Outer Controller

After minimizing that optimization problem (10) and substituting the parameter estimation θ^i into the inner controller, we denote the designed inner controller as Ciq,θ^i. Then, we turn to design the outer controller C0q,θ0 by the direct data-driven control scheme. Considering the outer loop and using the obtained inner controller Ciq,θ^i, it holds that(11)y0t=G′qyit=G′qGqCiq,θ^i1+GqCiq,θ^irit=G′qGqCiq,θ^i1+GqCiq,θ^iC0q,θ0e0t,that is,(12)y0t=G′qGqCiq,θ^i1+GqCiq,θ^iC0q,θ0r0t−y0t,1+G′qGqCiq,θ^iC0q,θ01+GqCiq,θ^iy0t=G′qGqCiq,θ^iC0q,θ01+GqCiq,θ^ir0t,y0t=G′qGqCiq,θ^iC0q,θ01+G′qGqCiq,θ^iC0q,θ0r0t.The minimization of the L2 norm of the modeling error is the same as the classical model reference controller; unknown parameter vectorθ0 is yielded by minimizing the following L2 norm:(13)J2θ0=G′qGqCiq,θ^iC0q,θ01+G′qGqCiq,θ^iC0q,θ0−M0q22.To avoid the modeling process for unknown plantGq,G′, the direct data-driven controller scheme is proposed to identify the unknown parameter vector θ0. It is obvious that(14)rit=C0q,θ0e0t=C0q,θ0r0t−y0t=C0q,θ0M0−1q−1y0t.Construct the minimization problem for the outer controllerC0q,θ0 during the framework of the direct data-driven control scheme, that is,(15)θ^0=argminθ0J3θ0,J3θ0=1N∑t=1Nrit−C0q,θ0M0−1q−1y0t2where θ^0 denotes the parameter estimation for the outer controller C0q,θ^0.But, in optimization problem (15), variable rit is unknown and unmeasured, so we need other relation to replace rit. Observing Figure 1 again, we have(16)eit=rit−yit,ut=Ciq,θ^ieit.Then,(17)eit=Ci−1q,θ^iut,rit=eit+yit=Ci−1q,θ^iut+yit.Substituting equation (17) into that L2 norm (15), it holds that(18)θ^0=argminθ0J3θ0,J3θ0=1N∑t=1NC0q,θ0M0−1q−1y0t−Ci−1q,θ^iut+yit2.Combining two optimization problems for the direct data-driven control scheme, cost functions (10) and (18) depend on our set of measured data DN=ut,yit,y0tt=1N. Furthermore, the other two cost functions (7) and (13) are related to those unknown plants Gq,G′q, so they correspond to the model-based control scheme.


### 3.3. First-Order Gradient Algorithm

Whatever for model-based control scheme or direct data-driven control scheme, two optimization problems are needed to solve. The difference between the above two schemes is that whether the cost function is related to the unknown plant, it is necessary to study one fast and real-time algorithm to solve those two optimization problems. Consider only one optimization problem (10) for estimation of unknown parameter vector θi again; for convenience, equation (10) is rewritten here again:(19)θ^i=argminθiJ1θi,J1θi=1N∑t=1Nut−Ciq,θiMi−1q−1yit.For this unconstrain optimization problem, the parameter estimationθ^i is easily solved by the commonly used Newton algorithm, that is,(20)θ^ik+1=θ^ik−∇2J1θi∇θi2−1∇J1θi∇θi=θ^ik−∇2J1θ^ik−1∇J1θ^ik,where θ^ik means the parameter estimation at kth iterative and ∇2J1θ^ik and ∇J1θ^ik are the Hessian matrix and gradient matrix at iteration θ^ik. More precisely, their detailed forms show(21)∇J1θi=2N∑t=1Nut−Ciq,θiMi−1q−1yit×∂Ciq,θi∂θi,(22)∇2J1θi=−2N∑t=1NMi−1q−12yi2t∂Ciq,θi∂θi+2N∑t=1Nut−Ciq,θiMi−1q−1yit×−Mi−1q−1yit∂2Ciq,θi∂θi2.In case of linear parameterized controller,(23)∂2Ciq,θi∂θi2=0,that is,(24)∇2J1θi=−2N∑t=1NMi−1q−12yi2t∂Ciq,θi∂θi.Substituting equations (22) and (24) into (20), the Newton algorithm is yielded. The merit of the Newton algorithm is formulated as follows.Theorem 1.
Whenθ^ik is closed to the optimal value θi∗, that is, ∇J1θi∗=0, set ∇2J1θi∗ positive definite and the Hessian matrix ∇2J1θi satisfies Lipschitz condition:(25)∇2J1θi1−∇2J1θi2≤θi1−θi2.Then, the obtained sequenceθ^ik will converge to its optimal value θi∗.From the computation process for the Hessian matrix∇2J1θi, square terms are needed. But the computation of this square term is difficult, so we use the first-order gradient algorithm to simplify the iterative value or avoid that Hessian matrix. Applying the first-order gradient algorithm to solve the above optimization problem, the scheme of the first-order gradient algorithm is as follows:Choose initial parameter vectorθi0kth iteration (k≥0)ComputeJ1θ^ik and ∇J1θ^ikFindθ^ik+1=θ^ik−h∇J1θ^ikWhereh>0 is one step size.For all other step size rules, the rate of convergence of the first-order gradient algorithm remains the same as Theorem1.Theorem 2.
Assume the first-order gradient∇J1θ^ik satisfies (26)∇J1θi1−∇J1θi2≤Lθi1−θi2,and 0<h<2/L; L is one positive value. Then, the first-order gradient algorithm generates the sequence θ^ik such that(27)J1θik−J1θi∗≤2J1θi0−J1θi∗θi0−θi∗22θi0−θi∗2+J1θi0−J1θi∗h92−2hk.Proof.
Set(28)rk=θik−θi∗.
Then,(29)rk+12=θik−θi∗−h∇J1θik2=rk2−2h∇J1,θik−θi∗+h2∇J12≤rk2−h2L−h∇J12,where we use(30)∇J1θik,θik−θi∗=∇J1θik−∇J1θi∗,θik−θi∗≥1L∇J12,∇J1θi∗=0.
So,rk≤r0. further, we have(31)J1θik+1≤J1θik+∇J1θik,θik−θi∗+L2θik+1−θik2=J1θik−w∇J1θik2,where w=h1−L/2−h.
DenoteΔk=J1θik−J1θi∗; then(32)Δk≤∇J1θik,θik−θi∗≤r0∇J1θik.
Therefore,(33)Δk+1≤Δk−wr02Δk2.
Thus,(34)1Δk+1≥1Δk+wr02ΔkΔk+1≥1Δk+wr02.
Summarizing these inequalities, we have(35)1Δk+1≥1Δ0+wr02k+1,r0=θi0−θi∗.
This completes the proof.
Comparing the first-order gradient algorithm and Newton algorithm, the merit of the first-order gradient algorithm is that one step size is used to replace the Hessian matrix; then, the computation of the Hessian matrix is avoided while guaranteeing the convergence property.


## 3.1. Inner Controller

Firstly considering the inner loop in Figure1, we have(5)yit=Miqrit,yit=Gqut=GqCiq,θieit=GqCiq,θirit−yit,where the first equation in (5) is the expected relation, and the second equation is the real relation between the input and output of the inner loop.Formulating equation (5) again, it holds that(6)yit=GqCiq,θirit1+GqCiq,θi.Comparing equations (5) and (6) and using the reference model Miq, then transfer function from rit to yit must be equal to its reference model, through tuning the parameter vector θi. It means parameter vector θi is chosen to minimize the following mathematical optimization problem:(7)J0θi=GqCiq,θi1+GqCiq,θi−Miq2,where, in optimization problem (7), 2 is the common L2 norm.


### 3.1.1. Comment

When minimizing the optimization problem (7) to get parameter vector θi, unknown plant Gq is needed to identify firstly. This optimization problem (7) is not solvable without any knowledge about unknown plant Gq.To avoid the appearance of unknown plantGq in the optimization problem, direct data-driven control is proposed here. Using the set of measured data ut,yitt=1N, parameter vector θi is extracted from the measured data ut,yitt=1N, while avoiding the modeling process for unknown plant Gq. Observing the input signal and output signal for that unknown inner controller Ciq,θi, we have(8)ut=Ciq,θieit=Ciq,θirit−yit.Substituting equations (4) into (8), we get(9)ut=Ciq,θiMi−1qyit−yit=Ciq,θiMi−1q−1yit,where the input and output signals correspond to that measured data ut,yitt=1N and no knowledge of unknown plant Gq exists. Then, we construct the following minimization problem for the inner controller Ciq,θi:(10)θ^i=argminθiJ1θi,J1θi=1N∑t=1Nut−Ciq,θiMi−1q−1yit,where θ^i is the parameter estimation of parameter vector θi, measured data ut,yitt=1N are collected during the whole experiment, and reference model Miq is given, so only parameter vector θi is unknown in that cost function (10).Because all variables are known in cost function (10), expect for unknown parameter vector θi, parameter estimation θ^i can be obtained through minimizing that optimization problem (10). The detailed optimization algorithm, used to minimize optimization problem, will be described in Section 3.3.


## 3.1.1. Comment

When minimizing the optimization problem (7) to get parameter vector θi, unknown plant Gq is needed to identify firstly. This optimization problem (7) is not solvable without any knowledge about unknown plant Gq.To avoid the appearance of unknown plantGq in the optimization problem, direct data-driven control is proposed here. Using the set of measured data ut,yitt=1N, parameter vector θi is extracted from the measured data ut,yitt=1N, while avoiding the modeling process for unknown plant Gq. Observing the input signal and output signal for that unknown inner controller Ciq,θi, we have(8)ut=Ciq,θieit=Ciq,θirit−yit.Substituting equations (4) into (8), we get(9)ut=Ciq,θiMi−1qyit−yit=Ciq,θiMi−1q−1yit,where the input and output signals correspond to that measured data ut,yitt=1N and no knowledge of unknown plant Gq exists. Then, we construct the following minimization problem for the inner controller Ciq,θi:(10)θ^i=argminθiJ1θi,J1θi=1N∑t=1Nut−Ciq,θiMi−1q−1yit,where θ^i is the parameter estimation of parameter vector θi, measured data ut,yitt=1N are collected during the whole experiment, and reference model Miq is given, so only parameter vector θi is unknown in that cost function (10).Because all variables are known in cost function (10), expect for unknown parameter vector θi, parameter estimation θ^i can be obtained through minimizing that optimization problem (10). The detailed optimization algorithm, used to minimize optimization problem, will be described in Section 3.3.


## 3.2. Outer Controller

After minimizing that optimization problem (10) and substituting the parameter estimation θ^i into the inner controller, we denote the designed inner controller as Ciq,θ^i. Then, we turn to design the outer controller C0q,θ0 by the direct data-driven control scheme. Considering the outer loop and using the obtained inner controller Ciq,θ^i, it holds that(11)y0t=G′qyit=G′qGqCiq,θ^i1+GqCiq,θ^irit=G′qGqCiq,θ^i1+GqCiq,θ^iC0q,θ0e0t,that is,(12)y0t=G′qGqCiq,θ^i1+GqCiq,θ^iC0q,θ0r0t−y0t,1+G′qGqCiq,θ^iC0q,θ01+GqCiq,θ^iy0t=G′qGqCiq,θ^iC0q,θ01+GqCiq,θ^ir0t,y0t=G′qGqCiq,θ^iC0q,θ01+G′qGqCiq,θ^iC0q,θ0r0t.The minimization of the L2 norm of the modeling error is the same as the classical model reference controller; unknown parameter vectorθ0 is yielded by minimizing the following L2 norm:(13)J2θ0=G′qGqCiq,θ^iC0q,θ01+G′qGqCiq,θ^iC0q,θ0−M0q22.To avoid the modeling process for unknown plantGq,G′, the direct data-driven controller scheme is proposed to identify the unknown parameter vector θ0. It is obvious that(14)rit=C0q,θ0e0t=C0q,θ0r0t−y0t=C0q,θ0M0−1q−1y0t.Construct the minimization problem for the outer controllerC0q,θ0 during the framework of the direct data-driven control scheme, that is,(15)θ^0=argminθ0J3θ0,J3θ0=1N∑t=1Nrit−C0q,θ0M0−1q−1y0t2where θ^0 denotes the parameter estimation for the outer controller C0q,θ^0.But, in optimization problem (15), variable rit is unknown and unmeasured, so we need other relation to replace rit. Observing Figure 1 again, we have(16)eit=rit−yit,ut=Ciq,θ^ieit.Then,(17)eit=Ci−1q,θ^iut,rit=eit+yit=Ci−1q,θ^iut+yit.Substituting equation (17) into that L2 norm (15), it holds that(18)θ^0=argminθ0J3θ0,J3θ0=1N∑t=1NC0q,θ0M0−1q−1y0t−Ci−1q,θ^iut+yit2.Combining two optimization problems for the direct data-driven control scheme, cost functions (10) and (18) depend on our set of measured data DN=ut,yit,y0tt=1N. Furthermore, the other two cost functions (7) and (13) are related to those unknown plants Gq,G′q, so they correspond to the model-based control scheme.


## 3.3. First-Order Gradient Algorithm

Whatever for model-based control scheme or direct data-driven control scheme, two optimization problems are needed to solve. The difference between the above two schemes is that whether the cost function is related to the unknown plant, it is necessary to study one fast and real-time algorithm to solve those two optimization problems. Consider only one optimization problem (10) for estimation of unknown parameter vector θi again; for convenience, equation (10) is rewritten here again:(19)θ^i=argminθiJ1θi,J1θi=1N∑t=1Nut−Ciq,θiMi−1q−1yit.For this unconstrain optimization problem, the parameter estimationθ^i is easily solved by the commonly used Newton algorithm, that is,(20)θ^ik+1=θ^ik−∇2J1θi∇θi2−1∇J1θi∇θi=θ^ik−∇2J1θ^ik−1∇J1θ^ik,where θ^ik means the parameter estimation at kth iterative and ∇2J1θ^ik and ∇J1θ^ik are the Hessian matrix and gradient matrix at iteration θ^ik. More precisely, their detailed forms show(21)∇J1θi=2N∑t=1Nut−Ciq,θiMi−1q−1yit×∂Ciq,θi∂θi,(22)∇2J1θi=−2N∑t=1NMi−1q−12yi2t∂Ciq,θi∂θi+2N∑t=1Nut−Ciq,θiMi−1q−1yit×−Mi−1q−1yit∂2Ciq,θi∂θi2.In case of linear parameterized controller,(23)∂2Ciq,θi∂θi2=0,that is,(24)∇2J1θi=−2N∑t=1NMi−1q−12yi2t∂Ciq,θi∂θi.Substituting equations (22) and (24) into (20), the Newton algorithm is yielded. The merit of the Newton algorithm is formulated as follows.Theorem 1.
Whenθ^ik is closed to the optimal value θi∗, that is, ∇J1θi∗=0, set ∇2J1θi∗ positive definite and the Hessian matrix ∇2J1θi satisfies Lipschitz condition:(25)∇2J1θi1−∇2J1θi2≤θi1−θi2.Then, the obtained sequenceθ^ik will converge to its optimal value θi∗.From the computation process for the Hessian matrix∇2J1θi, square terms are needed. But the computation of this square term is difficult, so we use the first-order gradient algorithm to simplify the iterative value or avoid that Hessian matrix. Applying the first-order gradient algorithm to solve the above optimization problem, the scheme of the first-order gradient algorithm is as follows:Choose initial parameter vectorθi0kth iteration (k≥0)ComputeJ1θ^ik and ∇J1θ^ikFindθ^ik+1=θ^ik−h∇J1θ^ikWhereh>0 is one step size.For all other step size rules, the rate of convergence of the first-order gradient algorithm remains the same as Theorem1.Theorem 2.
Assume the first-order gradient∇J1θ^ik satisfies (26)∇J1θi1−∇J1θi2≤Lθi1−θi2,and 0<h<2/L; L is one positive value. Then, the first-order gradient algorithm generates the sequence θ^ik such that(27)J1θik−J1θi∗≤2J1θi0−J1θi∗θi0−θi∗22θi0−θi∗2+J1θi0−J1θi∗h92−2hk.Proof.
Set(28)rk=θik−θi∗.
Then,(29)rk+12=θik−θi∗−h∇J1θik2=rk2−2h∇J1,θik−θi∗+h2∇J12≤rk2−h2L−h∇J12,where we use(30)∇J1θik,θik−θi∗=∇J1θik−∇J1θi∗,θik−θi∗≥1L∇J12,∇J1θi∗=0.
So,rk≤r0. further, we have(31)J1θik+1≤J1θik+∇J1θik,θik−θi∗+L2θik+1−θik2=J1θik−w∇J1θik2,where w=h1−L/2−h.
DenoteΔk=J1θik−J1θi∗; then(32)Δk≤∇J1θik,θik−θi∗≤r0∇J1θik.
Therefore,(33)Δk+1≤Δk−wr02Δk2.
Thus,(34)1Δk+1≥1Δk+wr02ΔkΔk+1≥1Δk+wr02.
Summarizing these inequalities, we have(35)1Δk+1≥1Δ0+wr02k+1,r0=θi0−θi∗.
This completes the proof.
Comparing the first-order gradient algorithm and Newton algorithm, the merit of the first-order gradient algorithm is that one step size is used to replace the Hessian matrix; then, the computation of the Hessian matrix is avoided while guaranteeing the convergence property.


## 4. Suggestion for Nonlinear Controller Design

In Section3, all results hold on the condition of parameterized controllers Ciq,θi,C0q,θ0. This section gives one suggestion for nonlinear controller design in the framework of the direct data-driven control scheme. Without loss of generality, we only analyze the inner loop, as the analysis for the outer loop is the same as the other.Replot that inner loop system in Figure2.Figure 2
Inner loop system.In Figure2, inner controller Ciq is one nonlinear form, not still a parameterized form. From equation (10), the ideal case for that inner controller Ciq satisfies that(36)ut=CiqMi−1q−1yit=Ciqyi′t,yi′t=Mi−1q−1yit.After multiplying both sides byut, it gives(37)utuTt=Ciqyi′tut.Take the expectation on both sides of equation (37) to get(38)EutuTt=CiqEyi′tut,that is,(39)ϕuw=Ciqϕyi′uw,where ϕuw is the auto power spectral and ϕyi′uw is the cross power spectral between yi′t and ut.To expand both sides of equation (38), from Figure 2, we see(40)yit=Gqut,ut=Ciqeit=Ciqrit−yit=Ciqrit−Gqut.It means(41)ut=Ciqrit−CiqGqut,ut=Ciq1+CiqGqrit,yit=GqCiq1+CiqGqrit.Apply the spectral theory to get(42)ϕuw=Ci2q1+CiqGq2ϕriw,ϕyiuw=GqCi2q1+CiqGq2ϕriw,where ϕriw is the auto power spectral for reference signal rit.Then, using equation (36), we have(43)ϕyi′uw=EMi−1q−1yitut=Mi−1q−1GqCi2q1+CiqGq2ϕriw.Substituting equations (41) and (42) into (38), the nonlinear inner controller Ciq is equal to(44)Ciq=ϕyi′uwϕuw=Mi−1q−1GqCi2qϕriw1+CiqGq2×1+CiqGq2Ci2qϕriw=Mi−1q−1Gq.Equation (43) tells that when the inner controller is one nonlinear form, its optimal form is equal to Mi−1q−1Gq, where Mi−1q is one given reference model. Thus, the designed nonlinear controller is related to that unknown plant Gq, which can be identified by system identification.Generally, from the above two sections, the direct data-driven control scheme can be applied to design two parameterized controllers, namely, the inner controller and router controller. On the contrary, for the case of the nonlinear inner controller and outer controller, direct data-driven estimation or identification is used to get the plantsGq,G′q; then, nonlinear inner controller and outer controller are obtained easily by simple multiply operations.


## 5. Flight Simulation

In this section, we apply our proposed direct data-driven control for cascade control system into one control problem in flight simulation. To be convenient for understanding well, the basic knowledge about the control structure for flight simulation is introduced in Figure3.Figure 3
Flight simulation.The structure of the UAV simulation system includes a flight simulation with three degrees of freedom, a turntable control cabinet, a steering gear angle measuring device, and a simulation master console. Flight simulation realizes the azimuth axis motion simulation, and the pitch, roll, and heading attitudes are adjusted by the motion controller and fed back to the flight simulation computer. Flight simulation is a high-precision servo system. For an analogue flight simulation with a motor as a driving element, its essence is an electric position closed-loop control system. We can use negative feedback to improve the sensitivity of the system, and connect regulators in series in each closed loop, which can effectively reduce the sensitivity of the system to parameter changes. Usually, we design the motor control system as a three-loop structure: from the inside to the outside, the current loop (torque feedback), speed loop, and position loop. Among them, the current loop and the speed loop effectively improve the rigidity of the system and suppress the external disturbance of the system, and the position loop ensures the accuracy of the control system. According to the type of control law, the current loop and speed loop are generally controlled by PID and low-pass filter compensation. The control loop principle of each attitude angle is shown in Figure4.Figure 4
Control loop principle of each attitude angle.The tracking of the UAV pitch angle signal by the three-degree-of-freedom motion simulation platform is realized by the actuator motor. The two screw rods and the middle single shaft support the load platform in a right-angled triangle manner, and when used as a moving cylinder to move forward and backward, the load platform can simulate the corresponding pitch angle. The pitch angle tracking loop realizes the automatic tracking of the UAV pitch angle. The platform pitch angle command comes from the simulation output of the UAV dynamic model, and the pitch angle feedback is obtained through the tilt angle sensor on the load platform. The principle block diagram of the pitch angle tracking loop is shown in Figure5.Figure 5
Pitch angle tracking loop.The angle deviation is corrected by the pitch angle position loop corrector. The corrected signal is used as the actuator to execute the angular velocity of the motor. The speed feedback is the differential signal of the angular displacement of the motor. The three-point velocity method is used to calculate the differential signal. After the speed deviation is corrected, the torque motor is driven to rotate by the power amplification, and the actuator drives the load platform to move up and down under the drive of the torque motor to realize the accurate tracking of the UAV pitch angle signal.Based on each model in Figure5, its corresponding mathematical model for pitch angle tracking loop is shown in Figure 6, where θg is the pitch instruction or order, θf is one feedback signal from sensor, wg is one given angular velocity signal, and ua1 is the armature voltage for servo motor. Tf is interference torque, kpwm is power amplifier, Gθ is one correction link for pitch angle tracking loop, and Gw1 is also one correction link for servo motor speed loop.Figure 6
Model for pitch angle tracking loop.Classical PID controller is used for this pitch angle tracking loop, that is,(45)Ciq,θi=kp+kiqq−1+kdq−1q,θi=kp,ki,kd,where kp,ki,kd correspond to three PID controller parameters.We regard motor and load as one whole part and collect the input signal and output signal with respect to this whole part. That expected reference modelMiq is given as(46)Miq=0.921q2+1.842q+0.921q2+1.837q+0.847.Then, the complete design process is formulated as follows:Step 1. Before designing this PID controller or estimating these three unknown parameterskp,ki,kd, their initial values are chosen as 9.8,3.0,2.0.Step 2. ComputeJ1θ^ik and ∇J1θ^ik.Step 3. Constructθ^ik+1=θ^ik−h∇J1θ^ik where h=0.6.Step 4. Chooseε=0.1; if θ^ik+1−θ^ik≤ε, then terminate the first-order gradient algorithm, or go back to Step 1.Step 5. The final controller parameters are obtained as17,3.8,2.8, whose iterative convergence curves are shown in Figure 7.Figure 7
Convergence curves for three controller parameters. (a) Iterative convergence curve for controller parameterkp. (b) Iterative convergence curve for controller parameter ki. (c) Iterative convergence curve for controller parameter kd.
(a)(b)(c)


## 6. Conclusion

This paper connects system identification, direct data-driven, and optimization theory to study the direct data-driven control scheme for the cascade control system, which is more widely used in practical engineering. Due to the inner controller and outer controller existing in cascade control system simultaneously, the direct data-driven control scheme is proposed to design the inner controller and outer controller without any knowledge of the two unknown plants. We propose to design these two controllers not only for parameterized controller structure but also for nonlinear control in virtue of our considered direct data-driven control scheme. As this paper is one preliminary analysis in a noiseless environment, that is, no noise is considered through this paper, the direct data-driven control scheme for the cascade control system in a stochastic environment is our next ongoing work.



---

*Source: 1465493-2021-08-29.xml*