# Fault Detection Filter Design and Optimization for Switched Systems with All Modes Unstable

Authors: Hanqiao Huang; Haoyu Cheng; Ruijia Song; Gonghao Sun; Yangwang Fang; Guan Huang
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

This problem of intelligent switched fault detection filter design is investigated in this article. Firstly, the mode-dependent average dwell time (MDADT) method is applied to generate the time-dependent switching signal for switched systems with all subsystems unstable. Afterwards, the switched fault detection filter is proposed for the generation of residual signal, which consists of dynamics-based filter and learning-based filter. The MDADT method and multiple Lyapunov function (MLF) method are employed to guarantee the stability and prescribed attenuation performance. The parameters of dynamics-based filter are given by solving a series of linear matrix inequalities. To improve the transient performance, the deep reinforcement learning is introduced to design learning-based filter in the framework of actor-critic. The output of learning-based filter can be viewed as uncertainties of dynamics-based filter. The deep deterministic policy gradient algorithm and nonfragile control are adopted to guarantee the stability of algorithm and compensate the external disturbance. Finally, simulation results are given to illustrate the effectiveness of the method in the paper.


---

## Body

## 1. Introduction

Switched systems [1–3] are an important component of hybrid systems, which are composed of a series of subsystems and build the connection between nonlinear systems and linear systems. As an efficient way to deal with complex nonlinear systems, switched systems have attacked considerable attentions and were applied in the areas of both military and economics [4, 5], such as industrial manufacturing, flight control, robotic control, process control, and so on. The problems of modeling, stability analysis [6], controller design [7], and filter design [8] have been investigated recently and fruitful encouraging results emerged [9]. To mention a few, the problem of switched fault tolerant controller design is investigated in [10], the event-triggered controller design method for discrete-time switched systems is proposed in [11], and the stability and l2-gain analysis are given in [12].The basic problem for switched systems lies in stability analysis [13, 14]. The scholars have developed tools to analyze the stability in past decades, such as common Lyapunov function (CLF) method, MLF method, and persistence Lyapunov function method [15]. The CLF method is mainly applied for the switched systems with arbitrary switching, which means that the switched systems are stable if all the subsystems share a CLF. However, it is difficult to design a CLF for all subsystems. To obtain tighter bounds, the stability analysis can be developed by the aid of MLF, which is mainly applied to constrained switching. Moreover, in practical system, the switching signal always depends on state or time. Because it is difficult to obtain the state measurements, the switching signal is time-dependent in many situations. Thus, the average dwell time (ADT) method and MDADT method provide efficient solutions to deal with the stability analysis for switched systems with constrained switching. In the work of [16], the issue of stability analysis for switched positive linear systems is studied. The ADT method and multiple linear copositive Lyapunov function are combined and sufficient stability criteria for stability analysis are given. However, it is straightforward that common parameters for all subsystems with different characteristic will lead to conservativeness, which motivates the researches on MDADT. The definition of MDADT is firstly proposed in [17]. The problem of stability analysis is studied in the framework of MDADT method and tighter bounds are obtained. In [18], the problem of stability and robust controller design for switched systems with external disturbances is studied. The MDADT switching and MLF method are introduced to ensure the stability. Moreover, in practice, the error of the switched system will lead to unstability. Therefore, the researches of stability analysis for switched systems with unstable subsystems are still one of the most important topics in control areas. In [19], the piecewise Lyapunov functions and MDADT method are combined to deal with the problem of stability analysis for switched systems with unstable modes. The fast switching is applied to unstable modes and slow switching is applied to stable modes. In [20], the problem of stability for switched systems with input time delay is studied. The unstable subsystems and asynchronous switching caused by time delay are taken into consideration. The extended stability criteria are obtained by the aid of Lyapunov-Krasovskii function method and sufficient conditions for stability analysis are presented.The fault case of system will no doubt lead to undesirable response [21, 22]. One issue in the industrial systems or aeronautic engineering is that the maintenance cannot be given immediately to ensure the reliability and safety [23]. Thus, the presence of undesirable fault and the possibility of the occurrence of faults has to be considered in the stability analysis and system design, which motivates the study on fault detection and fault tolerant control. As an efficient way to deal with the undesirable faults, fault detection technique has attracted more and more attention, which can be seen in the recent important results, such as [24–26]. The fault detection system can detect the fault in time and efficiently so that we can reconstruct the control diagram to adjust to the fault environment. Among these fault detection methods, the most valuable and applicable method is the model-based fault detection filter design method. It can be inferred in existing literature that the model-based method consists of a residual generator and residual evaluator, which is applied in the areas of networked control systems, aerospace engineering, and process control. In [27], the problem of H∞/H-fault detection for switched systems with unstable modes is investigated. The robustness to external disturbance and the sensitivity to fault are both taken into consideration. In [28], the problem of cooperative control for multiagent systems is investigated based on adaptive control and fuzzy control. The unknown control parameters and actuator fault are taken into consideration. The backstepping control is utilized to derive the controller. In the work of [29], the problem of fault detection subject to nonlinearities and disturbance is investigated according to event-triggered scheme. Sufficient conditions to guarantee the system is stable with the prescribed performance are obtained based on ADT method and MLF method. Based on the literature mentioned above, it can be inferred that the model-based fault detection filter is popularly applied in most of engineering problems because of the design simplicity and physical execution. However, it is difficult to achieve optimal compromise between robustness and transient performance. The model-based method can guarantee the stability and robustness of the closed-loop system. But the transient performance cannot be guaranteed. It is essential and significant to improve the transient performance of fault detection filters. Moreover, due to the lack of online-learning ability, this type of fault detection filters is relatively ineffective to have optimal performance in the real-world uncertain environment.With the development of machine learning and computer science, the intelligent control has drawn considerable attention [30]. As a powerful nonlinear approximation approach, deep learning and deep reinforcement learning has promoted the considerable performance in realistic applications, such as controller parameters tuning, decision making, and so on. In particular, the DDPG algorithm has been illustrated to perform stably and efficiently on many high-dimensional action control tasks. In the work of [31], a noninteger PID controller based on DDPG algorithm is proposed for the tracking problem. To accomplish the control methodology, a kinematic controller and a dynamic controller are established independently, in which the kinematic controller is proposed based on the model of the vehicle and the dynamic controller is realized for the supplementary kinematic controller to achieve optimal performance. In [32], the assembly task is defined as a Markov decision process and a fuzzy DDPG algorithm is given to realize the task. To improve the performance and learning efficiency, a fuzzy reward system is developed for the assembly task. In addition, in the work of [33–35], the machine learning is applied in the design of controller and guidance law.Inspired by the aforementioned discussion, an effort is conducted in the paper to design an intelligent fault detection filter for switched system with all modes unstable, which is composed of dynamics-based filter and learning-based filter. As well known, the model-based fault detection filter is designed in the existing literature, which can guarantee the stability of closed-loop system and the robustness to external disturbance. However, how to improve the transient performance of fault detection still remains an open problem. On the other hand, in most literature, the fault detection filter is proposed based on the assumption that all the subsystems are stable. But in many practical situations, the subsystems may be unstable, which motivates the study in this paper. The robust control theory is applied to establish the dynamics-based filter. The switched system with all modes unstable is considered and the fault detection filter is presented for generation of the residual signal. The MDADT method and MLF method are combined to ensure the stability and prescribed attenuation performance index of switched system. To achieve optimal performance, the learning-based filter is introduced based on DDPG algorithm in the framework of actor-critic, in which the filter parameters are optimized by online learning. The output of learning-based filter can be viewed as variation of the parameters of dynamics-based filter. Thus, the nonfragile control theory is introduced to guarantee the stability of switched systems. The main contributions of this article are stated as follows: (1) An observer-based filter is proposed to deal with the problem of fault detection for switched system with all modes unstable, in which the stability and attenuation performance index can be guaranteed by MDADT method and MLF method. Compared to the existing results, the characteristic of each subsystem is considered, and tighter bounds can be obtained, which provides more room for the improvement of flexibility. (2) The learning-based fault detection filter is presented based on DDPG algorithm to achieve optimal performance, which overcomes the undesirable response caused by external disturbance and uncertainties. The action is defined by the variation of parameters of the fault detection filter. (3) The nonfragile control theory is applied to ensure the stability of closed-loop system, for the reason that the output of actor network is viewed as the variation of parameters of dynamics-based filter.The rest of the paper is presented as follows: the model of switched system with all modes unstable is established in Section2, in which the fault detection filter is proposed to generate the residual signal. In Section 2, the main results of fault detection are proposed, which is composed of dynamics-based filter and learning-based filter. The numerical example is given in Section 4 to validate the effectiveness of proposed method, which is followed by the conclusion in Section 5.


## 2. Problem Statement

The continuous-time switched system in this paper can be described as follows:(1)x˙t=Aσtxt+Bσtut+Dσtωt+Fσtft,yt=Cσtxt.xt is the state vector; ut denotes the input signal; yt is the output signal; ωt∈L20,∞ is the external disturbances; ft represents the fault signal to be detected; i=σk:0,∞⟶N=1,2,...,n is piecewise switching signal; Aσt, Bσt, Cσt, and Fσt are system matrices with appropriate dimensions.To improve the transient performance of fault detection filter and achieve optimal performance, it is supposed that the fault detection filter in this paper is composed of two parts: the dynamics-based filter and the learning-based filter. The dynamics-based filter is designed according to robust control theory and the learning-based filter is proposed based on deep reinforcement learning, which can be described as follows:(2)Lσt=Lr,σt+Ld,σt.Lr,σt is the parameters of dynamics-based fault detection filter, which is determined by robust control theory; Ld,σt is the compensation for external disturbance, which is generated by the aid of DDPG algorithm. To ensure the stability of DDPG algorithm, the compensation can be viewed as the variation of parameters. Thus, the stability of optimization algorithm is guaranteed by nonfragile control theory. The parameters obtained by DDPG algorithm can be written as(3)Ld,σt=ΔLi=MiEiNi.Mi and Ni are the known matrices with appropriate dimensions; Ei are unknown matrices satisfying EiTEi≤I.Remark 1.
The designing process can be divided into two steps: (1) regarding the design of robust control theory, it is proposed to ensure the stability of closed-loop system and prescribed attenuation performance. (2) The deep reinforcement learning is utilized to improve the performance of fault detection, where the additional compensation of fault detection filter is viewed as the action of the agent. Then the parameters of fault detection filter are optimized based on DDPG algorithm in the framework of actor-critic. Compared with the traditional methods, not only can the stability be guaranteed by robust control theory, but also the transient performance of fault detection can be improved based on DDPG algorithm.Remark 2.
The problem of fault detection for switched systems with all modes unstable is studied in this paper. The unstable modes are taken into consideration and it is more applicable. The MDADT method and MLF method are combined to analyze the stability and tighter bounds on dwell time can be obtained than the traditional ADT method. There is more room for the switched systems to dwell long enough time to decrease the system energy. Therefore, the method proposed in this paper will improve the design flexibility of fault detection system.
The dynamics-based fault detection filter is proposed to generate the residual signal, which can be described as follows:(4)x^˙t=Aσtx^t+Bσtut+Lr,σtyt−y^t,y^t=Cσtx^t,rt=yt−y^t,where x^t is the state of fault detection filter; y^t denotes the estimation of output signal yt; rt is the residual signal and Lσt denote the parameter matrices of fault detection filter to be determined.
Define the error of state measurement aset=xt−x^t; the error of fault estimation r˜t=rt−ft. Thus, we set the augmented state vector x˜t=xTteTtT and the augmented input signal as ω˜t=uTtωTtfTtT, so we can obtain the closed-loop switched system as follows:(5)x˜˙t=A˜ix˜t+B˜iω˜t,r˜t=C˜ix˜t+D˜iω˜t,where A˜i=Ai00Ai−LiCi, B˜i=BiDiFi0DiFi, C˜i=0Ci, D˜i=00−I.
Then we can conclude that the design of robust fault detection filter can be converted by the problem ofH∞ filter design. Therefore, for given prescribed attenuation performance index γw, the closed-loop switched system in equation (5) is globally uniformly exponentially stable (GUES) when ω˜t=0; the following inequality holds for all nonzero ω˜t under zero-initial condition.(6)∫0∞r˜Tsr˜sds≤γ2∫0∞ω˜Tsω˜sds.
The residual signal is generated by fault detection filter; it is necessary to design a residual evaluator, which consists of an evaluation function and a threshold. The evaluation function and threshold can be written as(7)Jt=1η∫s=t−ηtrTsrs,Jth=supω∈L20,∞,f=0Jk.η denotes the length of time window of evaluation function.
Based on residual evaluator and threshold, the decision logic can be expressed as follows:(8)Jth≥Jt,no fault⇒no alarm,Jth<Jt,fault⇒alarm.


## 3. Main Results

### 3.1. Robust Filter Design

The definitions and lemmas are given as follows to make the proof convenient.Definition 1.
[17]. For given switching logic σt and time interval t0,t1, we define Nσ,it0,t1 as the number of switching during the time interval t0,t1. If there exist constants N0i≥0 and τai>0, such that(9)Nσ,it0,t1≥N0i+Tit0,t1τai.
Then,τai is called the MDADT of fast switching and N0i is called the chattering bounds; we set N0i=0 in this paper.Definition 2.
(see [12]). For given switching logic σt, if there exist constant scalars δ>0, ε>0, and equation (10) holds for ω˜t=0, then the switched system in (5) is GUES.(10)et≤δe−εt−t0et0.Lemma 1.
(see [27]). For given matrices S, T and symmetric matrix Y, if there exist constant scalar κ>0, such that(11)Y+κ−1STS+κTTT<0.
then for matrixEi with EiTEi≤I, we have(12)Y+STET+TTETS<0,Theorem 1.
For given constant scalars0<μi<1, λi>0, if there exist positive definite matrices Pi, such that(13)Pi≤μiPj,∀i≠j,(14)A˜iT≤Pi+PiA˜i≤λiPi
then the switched system in equation (5) is GUES if MDADT satisfies the following equation:(15)0≤τai≤τai∗=−lnμσtiλσti.Proof.
We set the Lyapunov function as follows:(16)Vix˜t=x˜TtPix˜t.
Thus, we have(17)V˙it−λiVit=x˜tω˜tTA˜iB˜iTPix˜t+x˜TtPiA˜iB˜ix˜tω˜t−λix˜TtPix˜t=x˜tω˜tTA˜iTPi+PiA˜i−λiPiPiB˜i∗0x˜tω˜t.
Combining equation (14) and ω˜t=0, we can conclude that(18)V˙it−λiVit=x˜TtA˜iTPi+PiA˜i−λiPix˜t≤0.
It is supposed that the switching instants during time interval0,t are set to be t1, t2, …，tk with tk+1 = t; we can derive that(19)Vσtt≤eλσtt−tkVσttk≤μσteλσtt−tkVσtktk−≤μσteλσtt−tkeλσtktk−tk−1Vσtktk−1≤μσtμσtkeλσtt−tkeλσtktk−tk−1Vσtk−1tk−1−⋅⋅⋅≤∏s=1kμσtse∑s=1kλσtsts+1−tsVσ00.
Together with Definition 1, we have(20)Vσtt≤exp∑i=1kNσ,i0,tlnμσti+∑i=1kTi0,tλσtiVσ00≤exp∑i=1kTi0,tτailnμσti+∑i=1kTi0,tλσtiVσ00≤exp∑i=1klnμσtiτai+λσtiTi0,tVσ00.
If the MDADT of switched system satisfies equation (15), we have(21)lnμσtiτai+λσti≤0.
Combining with Definition2, one can conclude that the switched system in equation (5) is GUES when ω˜t=0.Theorem 2.
For given constants0<μi<1, λi>0, γ>0, if there exist positive definite matrices Pi, such that(22)Pi≤μiPj,∀i≠j,A˜iTPi+PiA˜i−λiPiPiB˜iC˜iT∗−γ2ID˜iT∗∗−I≤0.
Then, the switched system with MDADT satisfying equation (15) is GUES with prescribed attenuation performance γd, where γd=γ2μmin−k/2.Proof.
Define the Lyapunov function in equation (16); we can obtain the following equations under zero-initial condition.(23)V˙it−λiVit+r˜Ttr˜t−γ2ω˜Ttω˜t=x˜tω˜tTA˜iTPi+PiA˜i−λiPiPiB˜i∗0x˜tω˜t+x˜tω˜tTC˜iD˜iTC˜iD˜ix˜tω˜t−γ2ω˜Ttω˜t=x˜tω˜tTA˜iTPi+PiA˜i−λiPiPiB˜i∗−γ2I+C˜iD˜iTC˜iD˜ix˜tω˜t.
Based on Schur complement, we have(24)V˙it−λiVit+r˜Ttr˜t−γ2ω˜Ttω˜t=x˜tω˜tTA˜iTPi+PiA˜i−λiPiPiB˜iC˜iT∗−γ2ID˜iT∗∗−Ix˜tω˜t≤0.
According to the statement above, we have(25)Vσtt≤eλσtt−tkVσttk−∫tkteλσtt−sr˜Tsr˜s−γ2ω˜Tsω˜sds≤μσteλσtt−tkVσtktk−−∫tkteλσtt−sr˜Tsr˜s−γ2ω˜Tsω˜sds≤μσteλσtt−tkeλσtktk−tk−1Vσtktk−1−∫tk−1tkeλσttk−sr˜Tsr˜s−γ2ω˜Tsω˜sds−∫tkteλσtt−sr˜Tsr˜s−γ2ω˜Tsω˜sds≤μσtμσtkeλσtt−tkeλσtktk−tk−1Vσtk−1tk−1−−μσteλσtt−tk∫tk−1tkeλσttk−sr˜Tsr˜s−γ2ω˜Tsω˜sds−∫tkteλσtt−sr˜Tsr˜s−γ2ω˜Tsω˜sds⋅⋅⋅≤∏s=1kμσtse∑s=0kλσtsts+1−tsVσ00−∫tkteλσtt−sr˜Tsr˜s−γ2ω˜Tsω˜sds−μσteλσtt−tk∫tk−1tkeλσttk−sr˜Tsr˜s−γ2ω˜Tsω˜sds⋅⋅⋅−∏s=1kμσtse∑s=1kλσtsts+1−ts∫0t1eλσtt1−sr˜Tsr˜s−γ2ω˜Tsω˜sds≤exp∑i=1kNσ,i0,tlnμσti+∑i=1kTi0,tλσtiVσ00−∫0texp∑i=1kNσ,is,tlnμσti+∑i=1kTis,tλσtir˜Tsr˜s−γ2ω˜Tsω˜sds.
Together with the conditionVσtt≥0, one can conclude(26)∫0texp∑i=1kNσ,is,tlnμσti+∑i=1kTis,tλσtir˜Tsr˜s−γ2ω˜Tsω˜sds≤0⇔∫0texp∑i=1kNσ,is,tlnμσti+∑i=1kTis,tλσtir˜Tsr˜sds≤γ2∫0texp∑i=1kNσ,is,tlnμσti+∑i=1kTis,tλσtiω˜Tsω˜sds.
Multiplying both sides of equation (26) byexp−∑i=1kNσ,i0,tlnμσti, we can obtain(27)∫0texp−∑i=1kNσ,i0,slnμσti+∑i=1kTis,tλσtir˜Tsr˜sds≤γ2∫0texp−∑i=1kNσ,i0,slnμσti+∑i=1kTis,tλσtiω˜Tsω˜sds.
Together with equation (21) and the condition in equation (28), we can obtain equation (29).(28)Nσ,i0,slnμσti≤Ti0,sτailnμσti≤−Ti0,sλσti,(29)∫0texp∑i=1kTi0,sλσti+∑i=1kTis,tλσtir˜Tsr˜sds=∫0texp∑i=1kTi0,tλσtir˜Tsr˜sds≤∫0texp−∑i=1kNσ,i0,slnμσti+∑i=1kTis,tλσtir˜Tsr˜sds≤γ2∫0texp−∑i=1kNσ,i0,slnμσti+∑i=1kTis,tλσtiω˜Tsω˜sds≤γ2∫0texp−∑i=1kNσ,i0,tlnμσti+∑i=1kTis,tλσtiω˜Tsω˜sds≤γ2e−∑i=1kNσ,i0,tlnμmin∫0texp∑i=1kTis,tλσtiω˜Tsω˜sds≤γ2μmin−k∫0texp∑i=1kTis,tλσtiω˜Tsω˜sds≤γ2μmin−k∫0texp∑i=1kTi0,tλσtiω˜Tsω˜sds,and it can be inferred that(30)∫0texp∑i=1kTi0,tλσtir˜Tsr˜sds≤γ2μmin−k∫0texp∑i=1kTi0,tλσtiω˜Tsω˜sds∫0tr˜Tsr˜sds≤γ2μmin−k∫0tω˜Tsω˜sds.
Integrating both sides of equation (30) from 0 to ∞, we have(31)∫0∞r˜Tsr˜sds≤γ2μmin−k∫0∞ω˜Tsω˜sds.
We can conclude that the switched system in equation (5) is GUES with prescribed attenuation performance γd=γ2μmin−k/2.Theorem 3.
For given constant scalars0<μi<1, λi>0, and γ>0, if there exist positive definite matrices Pi, such that(32)Pi≤μiPj,∀i≠j,ΦiPiB˜iC˜iTΨ1iΨ2i∗−γ2ID˜iT00∗∗−I00∗∗∗Wi0∗∗∗∗Wi≤0.
Then, the switched system with MDADT satisfying equation (16) is GUES with prescribed attenuation performance γd; the parameters of fault detection filter can be given by(33)Lni=P2i−1X1i,where Pi=P1iP2i∗P3i, Φi=AiTP1i+P1iAi−λP1i,AiTP2i+P2iAi−λP2i−X1iCi,∗,AiTP3i+P3iAi−λP3i−X2iCi−CiTX2iT,, X1i=P2iLni, X2i=P3iLni, Ψ1i=Θ1iTΘ2iT, Ψ2i=Θ3iTΘ4iT, Θ1i=−M1iTP2iT0, Θ2i=0N1iCi, Θ3i=0−M2iTP3iT, Θ4i=0N2iCi, Wi=−κi00−κi−1.Proof.
It can be inferred that equation (14) can be rewritten as(34)Yi+S1iTEiT1i+T1iTEiTS1i+S2iTEiT2i+T2iTEiTS2i≤0,where Yi=A˜1iTPi+PiA˜1i−λiPiPiB˜iC˜iT∗−γ2ID˜iT∗∗−I, A˜1i=Ai00Ai−LniCi, S1i=Θ1i00, T1i=Θ2i00S2i=Θ3i00, T2i=Θ4i00.
Based on Lemma1, we have(35)Yi+κ−1S1iTS1i+κT1iTT1i+κ−1S2iTS2i+κT2iTT2i<0.
Thus, by the aid of Schur complement, we can conclude that the switched system in equation (5) is GUES with prescribed performance γd; the parameters of fault detection filters can be obtained by equation (33).Remark 3.
The learning-based fault detection filter is viewed as the compensation of the robust filter. Therefore, we can obtain the scheduling interval to ensure the stability and prescribed attenuation performance by the aid of nonfragile control theory in Theorem3. The upper bounds on action in DDPG algorithm can be obtained by the predefined scheduling interval.


### 3.2. Intelligent Filter Design

The stability and prescribed attenuation performance are guaranteed by the theorems aforementioned. However, the transient performance needs to be improved. To solve the problem, deep reinforcement learning is applied in the framework of actor-critic. The optimization of parameters of fault detection filter can be viewed as an infinite Markov decision process, which is a series of continuous optimization processes. Thus, the DDPG algorithm is developed in this paper to improve the performance of the filter.The frame of reinforcement learning consists of an agent and the environment. The state atkth time instant is defined as sk, an action ak is chosen by the agent, and then a reward function rk and the state of next step are obtained, where rk is developed to evaluate the performance of state-action pair generated by the agent. The fault detection system is viewed as the environment. We define the additional compensation Ld,σt as the action, which is utilized to maximize the sum of the expected discounted reward function over given future steps. The action and the sum of expected discounted reward function can be described in equations (36) and (37).(36)Rk=re,k+γfre,k+1+γf2re,k+2+⋯+γfKf−kre,Kf=re,k+γfRk+1,(37)ak=ΔLi,where γf∈0,1 is defined as the discount factor and Kf represents the terminal step.The state of the agent is given in the following equation:(38)sk=xkekrkr˜k.The DDPG algorithm is proposed in the framework of deepQ learning and actor-critic. There are two actor networks and two critic networks. The optimal policy is tried for realization based on policy gradient theory in continuous action spaces. The action-value is approximated by employing the critic network Qsk,ak|ϖQ, whose weights are defined as ϖQ. The current output of compensated parameters is generated based on the actor network ask|ϖa, whose weights are defined as ϖa. The weights ϖQ are updated according to the loss function, which is described in equation the following:(39)LϖQ=Es,aQsk,ak|ϖQ−y¯k2,where y¯k=rksk,ak+γpQsk+1,ϖsk|ϖa|ϖQ.The weights of actor network are updated according to the policy gradient theory, which is given in equations (40) and (41).(40)ϖak+1=ϖak+Lan∇ϖaJ,(41)∇ϖaJ=Eπ∇ϖaQπsk,πsk|ϖa|ϖQ|s=sk,a=πsk|ϖa=Eπ∇ϖaQπsk,πsk|ϖQ|∇ϖaπsk|ϖQ,where Lan represents the learning rate of actor network.Moreover, in the DDPG algorithm, two networks are adopted as actor target network and critic target network, which are defined asa′sk|ϖa′ and Q′sk,ak|ϖQ′. The weights of actor target network a′sk|ϖa′ are defined as ϖa′, which are updated according to the following equation:(42)ϖa′k+1=Latϖak+1−Latϖa′k.Lat denotes the learning rate of actor target network.Similarly, the weights of critic target networkQ′sk,ak|ϖQ′ are defined as ϖQ′, which are updated according to the following equation:(43)ϖQ′k+1=LctϖQk+1+1−LctϖQ′k+1.Lct represents the learning rate of critic target network.In order to improve the robustness of the proposed algorithm, an exploration noise is introduced as a compensation of the output of actor network, which can be generated based on(44)ak=πsk|ϖa+Na.Therefore, based on the statement above, the pseudocode of intelligent fault detection filters design can be presented in Algorithm 1.Algorithm 1: Intelligent optimization algorithm for fault detection filters design.
(1)
Construct the fault detection filters for switched systems(2)
Set the variation range of fault detection filters(3)
Stability analysis and calculate the MDADT based on Eq. (15)(4)
Design the fault detection filters based on Theorem3(5)
Randomly generate the initial weights of the actor network and critic network(6)
Initialize the critic target network and actor target network(7)
Initialize the Replay buffer and defineepisode = 0(8)
forepisode = 1 to M do(9)
Initialize a random noise processNa to improve the robustness(10)
Initialize a random state vectors1, then the initial observation can be obtained(11)
fort = 1 to K do(12)
Chose actionak=πsk|ϖa+Na based on the current state sk and noise in the environment(13)
Calculate the reward functionrk and state sk+1(14)
Store the transition pairsk,ak,rk,sk+1 in the Replay buffer R(15)
Sample a random minibatch ofN transitions sm,am,rm,sm+1 from the Replay buffer(16)
Setyk=rksk,ak+γpQsk+1,ϖsk|ϖa|ϖQ(17)
Update the critic network, actor network and target networks(18)
end for(19)
end forRemark 4.
The DDPG algorithm is developed in this paper to improve the transient performance of the fault detection filter. Compared with the traditional method, the stability, robustness, and dynamic performance can be guaranteed simultaneously, in which the robust control theory and nonfragile control theory are introduced to ensure the stability, and the compensation of controller can be viewed as the variation of predesigned controller. Therefore, the stability of closed-loop system can be guaranteed by nonfragile control theory.


## 3.1. Robust Filter Design

The definitions and lemmas are given as follows to make the proof convenient.Definition 1.
[17]. For given switching logic σt and time interval t0,t1, we define Nσ,it0,t1 as the number of switching during the time interval t0,t1. If there exist constants N0i≥0 and τai>0, such that(9)Nσ,it0,t1≥N0i+Tit0,t1τai.
Then,τai is called the MDADT of fast switching and N0i is called the chattering bounds; we set N0i=0 in this paper.Definition 2.
(see [12]). For given switching logic σt, if there exist constant scalars δ>0, ε>0, and equation (10) holds for ω˜t=0, then the switched system in (5) is GUES.(10)et≤δe−εt−t0et0.Lemma 1.
(see [27]). For given matrices S, T and symmetric matrix Y, if there exist constant scalar κ>0, such that(11)Y+κ−1STS+κTTT<0.
then for matrixEi with EiTEi≤I, we have(12)Y+STET+TTETS<0,Theorem 1.
For given constant scalars0<μi<1, λi>0, if there exist positive definite matrices Pi, such that(13)Pi≤μiPj,∀i≠j,(14)A˜iT≤Pi+PiA˜i≤λiPi
then the switched system in equation (5) is GUES if MDADT satisfies the following equation:(15)0≤τai≤τai∗=−lnμσtiλσti.Proof.
We set the Lyapunov function as follows:(16)Vix˜t=x˜TtPix˜t.
Thus, we have(17)V˙it−λiVit=x˜tω˜tTA˜iB˜iTPix˜t+x˜TtPiA˜iB˜ix˜tω˜t−λix˜TtPix˜t=x˜tω˜tTA˜iTPi+PiA˜i−λiPiPiB˜i∗0x˜tω˜t.
Combining equation (14) and ω˜t=0, we can conclude that(18)V˙it−λiVit=x˜TtA˜iTPi+PiA˜i−λiPix˜t≤0.
It is supposed that the switching instants during time interval0,t are set to be t1, t2, …，tk with tk+1 = t; we can derive that(19)Vσtt≤eλσtt−tkVσttk≤μσteλσtt−tkVσtktk−≤μσteλσtt−tkeλσtktk−tk−1Vσtktk−1≤μσtμσtkeλσtt−tkeλσtktk−tk−1Vσtk−1tk−1−⋅⋅⋅≤∏s=1kμσtse∑s=1kλσtsts+1−tsVσ00.
Together with Definition 1, we have(20)Vσtt≤exp∑i=1kNσ,i0,tlnμσti+∑i=1kTi0,tλσtiVσ00≤exp∑i=1kTi0,tτailnμσti+∑i=1kTi0,tλσtiVσ00≤exp∑i=1klnμσtiτai+λσtiTi0,tVσ00.
If the MDADT of switched system satisfies equation (15), we have(21)lnμσtiτai+λσti≤0.
Combining with Definition2, one can conclude that the switched system in equation (5) is GUES when ω˜t=0.Theorem 2.
For given constants0<μi<1, λi>0, γ>0, if there exist positive definite matrices Pi, such that(22)Pi≤μiPj,∀i≠j,A˜iTPi+PiA˜i−λiPiPiB˜iC˜iT∗−γ2ID˜iT∗∗−I≤0.
Then, the switched system with MDADT satisfying equation (15) is GUES with prescribed attenuation performance γd, where γd=γ2μmin−k/2.Proof.
Define the Lyapunov function in equation (16); we can obtain the following equations under zero-initial condition.(23)V˙it−λiVit+r˜Ttr˜t−γ2ω˜Ttω˜t=x˜tω˜tTA˜iTPi+PiA˜i−λiPiPiB˜i∗0x˜tω˜t+x˜tω˜tTC˜iD˜iTC˜iD˜ix˜tω˜t−γ2ω˜Ttω˜t=x˜tω˜tTA˜iTPi+PiA˜i−λiPiPiB˜i∗−γ2I+C˜iD˜iTC˜iD˜ix˜tω˜t.
Based on Schur complement, we have(24)V˙it−λiVit+r˜Ttr˜t−γ2ω˜Ttω˜t=x˜tω˜tTA˜iTPi+PiA˜i−λiPiPiB˜iC˜iT∗−γ2ID˜iT∗∗−Ix˜tω˜t≤0.
According to the statement above, we have(25)Vσtt≤eλσtt−tkVσttk−∫tkteλσtt−sr˜Tsr˜s−γ2ω˜Tsω˜sds≤μσteλσtt−tkVσtktk−−∫tkteλσtt−sr˜Tsr˜s−γ2ω˜Tsω˜sds≤μσteλσtt−tkeλσtktk−tk−1Vσtktk−1−∫tk−1tkeλσttk−sr˜Tsr˜s−γ2ω˜Tsω˜sds−∫tkteλσtt−sr˜Tsr˜s−γ2ω˜Tsω˜sds≤μσtμσtkeλσtt−tkeλσtktk−tk−1Vσtk−1tk−1−−μσteλσtt−tk∫tk−1tkeλσttk−sr˜Tsr˜s−γ2ω˜Tsω˜sds−∫tkteλσtt−sr˜Tsr˜s−γ2ω˜Tsω˜sds⋅⋅⋅≤∏s=1kμσtse∑s=0kλσtsts+1−tsVσ00−∫tkteλσtt−sr˜Tsr˜s−γ2ω˜Tsω˜sds−μσteλσtt−tk∫tk−1tkeλσttk−sr˜Tsr˜s−γ2ω˜Tsω˜sds⋅⋅⋅−∏s=1kμσtse∑s=1kλσtsts+1−ts∫0t1eλσtt1−sr˜Tsr˜s−γ2ω˜Tsω˜sds≤exp∑i=1kNσ,i0,tlnμσti+∑i=1kTi0,tλσtiVσ00−∫0texp∑i=1kNσ,is,tlnμσti+∑i=1kTis,tλσtir˜Tsr˜s−γ2ω˜Tsω˜sds.
Together with the conditionVσtt≥0, one can conclude(26)∫0texp∑i=1kNσ,is,tlnμσti+∑i=1kTis,tλσtir˜Tsr˜s−γ2ω˜Tsω˜sds≤0⇔∫0texp∑i=1kNσ,is,tlnμσti+∑i=1kTis,tλσtir˜Tsr˜sds≤γ2∫0texp∑i=1kNσ,is,tlnμσti+∑i=1kTis,tλσtiω˜Tsω˜sds.
Multiplying both sides of equation (26) byexp−∑i=1kNσ,i0,tlnμσti, we can obtain(27)∫0texp−∑i=1kNσ,i0,slnμσti+∑i=1kTis,tλσtir˜Tsr˜sds≤γ2∫0texp−∑i=1kNσ,i0,slnμσti+∑i=1kTis,tλσtiω˜Tsω˜sds.
Together with equation (21) and the condition in equation (28), we can obtain equation (29).(28)Nσ,i0,slnμσti≤Ti0,sτailnμσti≤−Ti0,sλσti,(29)∫0texp∑i=1kTi0,sλσti+∑i=1kTis,tλσtir˜Tsr˜sds=∫0texp∑i=1kTi0,tλσtir˜Tsr˜sds≤∫0texp−∑i=1kNσ,i0,slnμσti+∑i=1kTis,tλσtir˜Tsr˜sds≤γ2∫0texp−∑i=1kNσ,i0,slnμσti+∑i=1kTis,tλσtiω˜Tsω˜sds≤γ2∫0texp−∑i=1kNσ,i0,tlnμσti+∑i=1kTis,tλσtiω˜Tsω˜sds≤γ2e−∑i=1kNσ,i0,tlnμmin∫0texp∑i=1kTis,tλσtiω˜Tsω˜sds≤γ2μmin−k∫0texp∑i=1kTis,tλσtiω˜Tsω˜sds≤γ2μmin−k∫0texp∑i=1kTi0,tλσtiω˜Tsω˜sds,and it can be inferred that(30)∫0texp∑i=1kTi0,tλσtir˜Tsr˜sds≤γ2μmin−k∫0texp∑i=1kTi0,tλσtiω˜Tsω˜sds∫0tr˜Tsr˜sds≤γ2μmin−k∫0tω˜Tsω˜sds.
Integrating both sides of equation (30) from 0 to ∞, we have(31)∫0∞r˜Tsr˜sds≤γ2μmin−k∫0∞ω˜Tsω˜sds.
We can conclude that the switched system in equation (5) is GUES with prescribed attenuation performance γd=γ2μmin−k/2.Theorem 3.
For given constant scalars0<μi<1, λi>0, and γ>0, if there exist positive definite matrices Pi, such that(32)Pi≤μiPj,∀i≠j,ΦiPiB˜iC˜iTΨ1iΨ2i∗−γ2ID˜iT00∗∗−I00∗∗∗Wi0∗∗∗∗Wi≤0.
Then, the switched system with MDADT satisfying equation (16) is GUES with prescribed attenuation performance γd; the parameters of fault detection filter can be given by(33)Lni=P2i−1X1i,where Pi=P1iP2i∗P3i, Φi=AiTP1i+P1iAi−λP1i,AiTP2i+P2iAi−λP2i−X1iCi,∗,AiTP3i+P3iAi−λP3i−X2iCi−CiTX2iT,, X1i=P2iLni, X2i=P3iLni, Ψ1i=Θ1iTΘ2iT, Ψ2i=Θ3iTΘ4iT, Θ1i=−M1iTP2iT0, Θ2i=0N1iCi, Θ3i=0−M2iTP3iT, Θ4i=0N2iCi, Wi=−κi00−κi−1.Proof.
It can be inferred that equation (14) can be rewritten as(34)Yi+S1iTEiT1i+T1iTEiTS1i+S2iTEiT2i+T2iTEiTS2i≤0,where Yi=A˜1iTPi+PiA˜1i−λiPiPiB˜iC˜iT∗−γ2ID˜iT∗∗−I, A˜1i=Ai00Ai−LniCi, S1i=Θ1i00, T1i=Θ2i00S2i=Θ3i00, T2i=Θ4i00.
Based on Lemma1, we have(35)Yi+κ−1S1iTS1i+κT1iTT1i+κ−1S2iTS2i+κT2iTT2i<0.
Thus, by the aid of Schur complement, we can conclude that the switched system in equation (5) is GUES with prescribed performance γd; the parameters of fault detection filters can be obtained by equation (33).Remark 3.
The learning-based fault detection filter is viewed as the compensation of the robust filter. Therefore, we can obtain the scheduling interval to ensure the stability and prescribed attenuation performance by the aid of nonfragile control theory in Theorem3. The upper bounds on action in DDPG algorithm can be obtained by the predefined scheduling interval.


## 3.2. Intelligent Filter Design

The stability and prescribed attenuation performance are guaranteed by the theorems aforementioned. However, the transient performance needs to be improved. To solve the problem, deep reinforcement learning is applied in the framework of actor-critic. The optimization of parameters of fault detection filter can be viewed as an infinite Markov decision process, which is a series of continuous optimization processes. Thus, the DDPG algorithm is developed in this paper to improve the performance of the filter.The frame of reinforcement learning consists of an agent and the environment. The state atkth time instant is defined as sk, an action ak is chosen by the agent, and then a reward function rk and the state of next step are obtained, where rk is developed to evaluate the performance of state-action pair generated by the agent. The fault detection system is viewed as the environment. We define the additional compensation Ld,σt as the action, which is utilized to maximize the sum of the expected discounted reward function over given future steps. The action and the sum of expected discounted reward function can be described in equations (36) and (37).(36)Rk=re,k+γfre,k+1+γf2re,k+2+⋯+γfKf−kre,Kf=re,k+γfRk+1,(37)ak=ΔLi,where γf∈0,1 is defined as the discount factor and Kf represents the terminal step.The state of the agent is given in the following equation:(38)sk=xkekrkr˜k.The DDPG algorithm is proposed in the framework of deepQ learning and actor-critic. There are two actor networks and two critic networks. The optimal policy is tried for realization based on policy gradient theory in continuous action spaces. The action-value is approximated by employing the critic network Qsk,ak|ϖQ, whose weights are defined as ϖQ. The current output of compensated parameters is generated based on the actor network ask|ϖa, whose weights are defined as ϖa. The weights ϖQ are updated according to the loss function, which is described in equation the following:(39)LϖQ=Es,aQsk,ak|ϖQ−y¯k2,where y¯k=rksk,ak+γpQsk+1,ϖsk|ϖa|ϖQ.The weights of actor network are updated according to the policy gradient theory, which is given in equations (40) and (41).(40)ϖak+1=ϖak+Lan∇ϖaJ,(41)∇ϖaJ=Eπ∇ϖaQπsk,πsk|ϖa|ϖQ|s=sk,a=πsk|ϖa=Eπ∇ϖaQπsk,πsk|ϖQ|∇ϖaπsk|ϖQ,where Lan represents the learning rate of actor network.Moreover, in the DDPG algorithm, two networks are adopted as actor target network and critic target network, which are defined asa′sk|ϖa′ and Q′sk,ak|ϖQ′. The weights of actor target network a′sk|ϖa′ are defined as ϖa′, which are updated according to the following equation:(42)ϖa′k+1=Latϖak+1−Latϖa′k.Lat denotes the learning rate of actor target network.Similarly, the weights of critic target networkQ′sk,ak|ϖQ′ are defined as ϖQ′, which are updated according to the following equation:(43)ϖQ′k+1=LctϖQk+1+1−LctϖQ′k+1.Lct represents the learning rate of critic target network.In order to improve the robustness of the proposed algorithm, an exploration noise is introduced as a compensation of the output of actor network, which can be generated based on(44)ak=πsk|ϖa+Na.Therefore, based on the statement above, the pseudocode of intelligent fault detection filters design can be presented in Algorithm 1.Algorithm 1: Intelligent optimization algorithm for fault detection filters design.
(1)
Construct the fault detection filters for switched systems(2)
Set the variation range of fault detection filters(3)
Stability analysis and calculate the MDADT based on Eq. (15)(4)
Design the fault detection filters based on Theorem3(5)
Randomly generate the initial weights of the actor network and critic network(6)
Initialize the critic target network and actor target network(7)
Initialize the Replay buffer and defineepisode = 0(8)
forepisode = 1 to M do(9)
Initialize a random noise processNa to improve the robustness(10)
Initialize a random state vectors1, then the initial observation can be obtained(11)
fort = 1 to K do(12)
Chose actionak=πsk|ϖa+Na based on the current state sk and noise in the environment(13)
Calculate the reward functionrk and state sk+1(14)
Store the transition pairsk,ak,rk,sk+1 in the Replay buffer R(15)
Sample a random minibatch ofN transitions sm,am,rm,sm+1 from the Replay buffer(16)
Setyk=rksk,ak+γpQsk+1,ϖsk|ϖa|ϖQ(17)
Update the critic network, actor network and target networks(18)
end for(19)
end forRemark 4.
The DDPG algorithm is developed in this paper to improve the transient performance of the fault detection filter. Compared with the traditional method, the stability, robustness, and dynamic performance can be guaranteed simultaneously, in which the robust control theory and nonfragile control theory are introduced to ensure the stability, and the compensation of controller can be viewed as the variation of predesigned controller. Therefore, the stability of closed-loop system can be guaranteed by nonfragile control theory.


## 4. Numerical Example

In this section, simulation results are given to validate the effectiveness of the proposed method. The system matrices are given as follows:(45)A1=−2.23−3.722.832.32,A2=−1.251.262.22−2.22,C1=1001,C2=1001,D1=0.10.1,D2=0.20.2,F1=0.10.2,F2=0.10.1.The eigenvalues ofA1 and A2 can be obtained as follows:(46)λ11=0.045+2.3134i,λ12=0.045−2.3134i,λ21=0.0064,λ22=−3.4764.It can be inferred that the subsystemsA1 and A2 are unstable, respectively. The external disturbance is defined as follows:(47)ωt=0.1exp−0.1tcos−0.1t.The other parameters of switched systems are listed as follows:(48)μ1=0.73,μ2=0.73,λ1=0.62,λ2=0.65.Therefore, the MDADT of switched systems areτ1=0.5076,τ2=0.4842. The prescribed attenuation performance index is set to be γ=0.8. Then the parameter matrices of filters can be obtained in Theorem 3.To validate the effectiveness of the proposed method, we give two numerical examples. The stability criteria of the switched systems in Theorem2 are demonstrated by Example 1. Moreover, the effectiveness of the fault detection filter in Theorem 3 and the DDPG algorithm is illustrated in Example 2.Example 1.
Firstly, we provide the proof that the proposed conditions can ensure the stability of switched systems. As a comparison, a randomly generated switching signal is introduced to show that the switched systems cannot stay stable if all the subsystems cannot share a CLF. The switching logic satisfying equation (20) is given in Figure 1 and the randomly generated switching logic is given in Figure 2. The state response of the proposed switching logic is given in Figure 3 and the state response of randomly generated switching logic is showed in Figure 4. We can see that the switched systems cannot ensure the stability under randomly generated switching logic. However, the stability can be guaranteed according to Theorem 1 in the paper.Figure 1
The switching logic satisfying equation (20).Figure 2
A randomly generated switching signal.Figure 3
The state response of the proposed switching logic.Figure 4
The state response of randomly generated switching logic.Example 2.
To validate the effectiveness of the fault detection filter, the traditional MDADT method and the proposed method are given. The fault signal is set to be(49)fk=0.5,4≤t≤5,0,else.
The results are given in Figures5–12. The residual signal and J(k) of MDADT method are given in Figures 5 and 6; the residual signal and J(k) of the proposed method are showed in Figures 7 and 8. We can see that the detection time of the MDADT method is 0.1 s, whereas, the detection time of the proposed method is 0.06 s. The fault can be detected efficiently, and the transient performance of fault detection filter can be improved by the DDPG algorithm. The state responses of the MDADT method and the proposed method are depicted in Figures 9–12. It can be inferred that the fault detection filter proposed in this paper can track the state response. The response of episodes reward is showed in Figure 13; we can see that the reward function can converge to a neighbor of the origin, which illustrate the effectiveness of the optimization algorithm.
In summary, we can see that the tighter bounds on dwell time and less conservative results are obtained. The stability of switched systems can be guaranteed by the proposed results in this paper. The fast switching strategy is applied, and it allows the subsystems of switched systems to remain unstable. The transient performance of fault detection filters can be improved by the aid of DDPG algorithm, and the stability, robustness, and optimal policy can be guaranteed simultaneously by the method proposed in this paper.Figure 5
The response of residual signal under MDADT method.Figure 6
The response ofJ(k) under MDADT method.Figure 7
The response of residual signal under the proposed method.Figure 8
The response ofJ(k) under the proposed method.Figure 9
The state response of the MDADT method.Figure 10
The state response of the proposed method.Figure 11
The state response of MDADT method under fault case.Figure 12
The state response of the proposed method under fault case.Figure 13
The response of episode reward.


## 5. Conclusions

The problem of fault detection and online scheduling for switched systems with all modes unstable is studied. The observer-based fault detection filter is proposed to generate the residual signal, which consists of two parts: the dynamics-based fault detection filter and the learning-based fault detection filter. By employing MDADT method and MLF method, the stability of the switched systems is guaranteed. The solutions of fault detection filters are given in the form of LMIs. To achieve optimal control policy and improve the transient performance, the DDPG algorithm is utilized as learning-based fault detection filter, in which the output can be viewed as the variation of robust fault detection filter. Therefore, the nonfragile control is provided to ensure the stability of the optimization algorithm. Finally, the simulation results are introduced to demonstrate the effectiveness and superiority of the proposed method.



---

*Source: 8339634-2022-04-04.xml*