# Differential Neural Networks for Identification and Filtering in Nonlinear Dynamic Games

Authors: Emmanuel García; Daishi Alfredo Murano
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

This paper deals with the problem of identifying and filtering a class of continuous-time nonlinear dynamic games (nonlinear differential games) subject to additive and undesired deterministic perturbations. Moreover, the mathematical model of this class is completely unknown with the exception of the control actions of each player, and even though the deterministic noises are known, their power (or their effect) is not. Therefore, two differential neural networks are designed in order to obtain a feedback (perfect state) information pattern for the mentioned class of games. In this way, the stability conditions for two state identification errors and for a filtering error are established, the upper bounds of these errors are obtained, and two new learning laws for each neural network are suggested. Finally, an illustrating example shows the applicability of this approach.


---

## Body

## 1. Introduction

### 1.1. Preliminaries and Motivation

Nowadays, an investigation field that has been developed widely is the design of controllers for a certain group of systems involved in a conflicting interaction, that is to say, when the objective of each implicated system differs and when the known information about this interaction may be distinct for every system (e.g., [1]).More formally, this special class of system groups can be represented by means of the dynamic noncooperative game theory, where the decision making process (or interaction) is called a game and each involved system in it is called a player [2].From the viewpoint of control theory, a dynamic game is controlled by obtaining its equilibrium solution, and in order to do so, a (mathematical) model of this dynamic game is needed. So, several publications about dynamic games (and particularly about continuous-time dynamic games) are based on the complete knowledge of the model that describes its dynamics (see, e.g., [3, 4]). Nevertheless, having a model (or even a partial model) of a continuous-time dynamic game is not always possible.On the other hand, the equilibrium solution of a dynamic game is also based on the information structure that every player has or, in other words, on the available information that each player can use in the control strategy. For example, one can obtain an open-loop Nash equilibrium solution by using the maximum principle technique or a feedback Nash equilibrium solution by utilizing the dynamic programing method (see [2]).According to the above, the aim of an identification process in terms of a dynamic game should be the modeling of such game and the obtaining of its information structure, that is, the guarantee that the control strategy of every player gives an equilibrium solution for a game despite its dynamic uncertainties. In this way, the works of [5, 6] obtain feedback control strategies for differential games modeled through norm-bounded uncertainties; the studies of [7–9] achieve equilibrium solutions for several classes of differential games under a multimodel approach; the analyses of [10, 11] present adaptive algorithms for determining equilibrium solutions without the complete knowledge of the game dynamics, and the work of [12] proposes the obtaining of a suboptimal equilibrium solution where a differential game is approximated by a fuzzy model.However, an identification process could be deficient if there exist undesired perturbations in the game dynamics, that is to say, if the obtained information structure of the game is corrupted by (deterministic) noises. Thereby, some deterministic filtering works have been published in order to solve this issue. For example, [13] describes the concept of adaptive noise canceling for the estimation of signals corrupted by additive perturbations; [14] introduces a finite horizon robust H∞ filtering method that provides a guaranteed bound for the estimation error in the presence of both parameter uncertainty and a known input signal; [15] presents the filtering of the states of a time-varying uncertain linear system with deterministic disturbances of limited power; and [16] derives an optimal filtering formula for linear time-varying discrete systems with unknown inputs.Therefore, according to these preliminary works, the motivation of this paper is to solve the problem of identifying and filtering a class of nonlinear differential games with additive deterministic perturbations, where the mathematical model of this class and the effect (or power) of the noises are completely unknown.


### 1.2. Main Contribution

Since the introduction of continuous-time recurrent neural networks (see [17]), the self-named differential neural networks have proved to be an excellent tool in the identification, state estimation, and control of several systems and of the appointed continuous-time dynamic games.For example, in [18], differential neural networks are used for the identification of dynamical systems; references [19–21] design differential neural network observers for adaptive state estimation; the works of [22, 23] propose neural network controllers for several applications; and [24] shows a compendium of differential neural networks for identification, state estimation, and control of nonlinear systems. Also, [25] treats the state estimation problem for affine nonlinear differential games using a differential neural network observer, and in [26], a nearly optimal Nash equilibrium for classes of deterministic and stochastic nonlinear differential games is obtained using differential neural networks.Moreover, speaking of recurrent neural networks and deterministic filtering, the work of [27] develops a recurrent neural network for robust optimal filter design in a H∞ approach, and in [28], algorithms are presented in order to obtain adaptive filtering in nonlinear dynamic systems approximated by neural networks.Nevertheless, the idea of using differential neural networks for identification and filtering of a class of continuous-time nonlinear dynamic games is a new approach that, as far as the authors know, has not been treated before.Hence, the main contribution of this paper is the proof that it is possible to identify and to filter the states of a certain class of nonlinear differential games through the designing of two differential neural networks. Moreover, this filtered identification process generates a feedback (perfect state) information pattern for the mentioned class of games.More specifically, although the structure of this class is known, its mathematical model is not; that is to say, the only available information of the nonlinear differential game is the control actions of each player. So, by using only this available information, a first differential neural network will be designed for the identification of the nonlinear dynamic game with the undesired perturbations, and, similarly, the second differential neural network will identify the effect of these additive noises in the dynamics of the nonlinear differential game, which means that the perturbations are known but their power is not.According to the above, one of these two differential neural networks will do the complete identification process of the class of nonlinear differential games, and, noticeably, the filtering (or the canceling) of the undesired perturbations will be held by subtracting the state estimates of the two differential neural networks.Finally, it is important to emphasize that these two differential neural networks have the structure of multilayer perceptrons (see [23, 24, 29]), and, by means of Lyapunov’s second method of stability, the learning laws for their synaptic weights are derived.


## 1.1. Preliminaries and Motivation

Nowadays, an investigation field that has been developed widely is the design of controllers for a certain group of systems involved in a conflicting interaction, that is to say, when the objective of each implicated system differs and when the known information about this interaction may be distinct for every system (e.g., [1]).More formally, this special class of system groups can be represented by means of the dynamic noncooperative game theory, where the decision making process (or interaction) is called a game and each involved system in it is called a player [2].From the viewpoint of control theory, a dynamic game is controlled by obtaining its equilibrium solution, and in order to do so, a (mathematical) model of this dynamic game is needed. So, several publications about dynamic games (and particularly about continuous-time dynamic games) are based on the complete knowledge of the model that describes its dynamics (see, e.g., [3, 4]). Nevertheless, having a model (or even a partial model) of a continuous-time dynamic game is not always possible.On the other hand, the equilibrium solution of a dynamic game is also based on the information structure that every player has or, in other words, on the available information that each player can use in the control strategy. For example, one can obtain an open-loop Nash equilibrium solution by using the maximum principle technique or a feedback Nash equilibrium solution by utilizing the dynamic programing method (see [2]).According to the above, the aim of an identification process in terms of a dynamic game should be the modeling of such game and the obtaining of its information structure, that is, the guarantee that the control strategy of every player gives an equilibrium solution for a game despite its dynamic uncertainties. In this way, the works of [5, 6] obtain feedback control strategies for differential games modeled through norm-bounded uncertainties; the studies of [7–9] achieve equilibrium solutions for several classes of differential games under a multimodel approach; the analyses of [10, 11] present adaptive algorithms for determining equilibrium solutions without the complete knowledge of the game dynamics, and the work of [12] proposes the obtaining of a suboptimal equilibrium solution where a differential game is approximated by a fuzzy model.However, an identification process could be deficient if there exist undesired perturbations in the game dynamics, that is to say, if the obtained information structure of the game is corrupted by (deterministic) noises. Thereby, some deterministic filtering works have been published in order to solve this issue. For example, [13] describes the concept of adaptive noise canceling for the estimation of signals corrupted by additive perturbations; [14] introduces a finite horizon robust H∞ filtering method that provides a guaranteed bound for the estimation error in the presence of both parameter uncertainty and a known input signal; [15] presents the filtering of the states of a time-varying uncertain linear system with deterministic disturbances of limited power; and [16] derives an optimal filtering formula for linear time-varying discrete systems with unknown inputs.Therefore, according to these preliminary works, the motivation of this paper is to solve the problem of identifying and filtering a class of nonlinear differential games with additive deterministic perturbations, where the mathematical model of this class and the effect (or power) of the noises are completely unknown.


## 1.2. Main Contribution

Since the introduction of continuous-time recurrent neural networks (see [17]), the self-named differential neural networks have proved to be an excellent tool in the identification, state estimation, and control of several systems and of the appointed continuous-time dynamic games.For example, in [18], differential neural networks are used for the identification of dynamical systems; references [19–21] design differential neural network observers for adaptive state estimation; the works of [22, 23] propose neural network controllers for several applications; and [24] shows a compendium of differential neural networks for identification, state estimation, and control of nonlinear systems. Also, [25] treats the state estimation problem for affine nonlinear differential games using a differential neural network observer, and in [26], a nearly optimal Nash equilibrium for classes of deterministic and stochastic nonlinear differential games is obtained using differential neural networks.Moreover, speaking of recurrent neural networks and deterministic filtering, the work of [27] develops a recurrent neural network for robust optimal filter design in a H∞ approach, and in [28], algorithms are presented in order to obtain adaptive filtering in nonlinear dynamic systems approximated by neural networks.Nevertheless, the idea of using differential neural networks for identification and filtering of a class of continuous-time nonlinear dynamic games is a new approach that, as far as the authors know, has not been treated before.Hence, the main contribution of this paper is the proof that it is possible to identify and to filter the states of a certain class of nonlinear differential games through the designing of two differential neural networks. Moreover, this filtered identification process generates a feedback (perfect state) information pattern for the mentioned class of games.More specifically, although the structure of this class is known, its mathematical model is not; that is to say, the only available information of the nonlinear differential game is the control actions of each player. So, by using only this available information, a first differential neural network will be designed for the identification of the nonlinear dynamic game with the undesired perturbations, and, similarly, the second differential neural network will identify the effect of these additive noises in the dynamics of the nonlinear differential game, which means that the perturbations are known but their power is not.According to the above, one of these two differential neural networks will do the complete identification process of the class of nonlinear differential games, and, noticeably, the filtering (or the canceling) of the undesired perturbations will be held by subtracting the state estimates of the two differential neural networks.Finally, it is important to emphasize that these two differential neural networks have the structure of multilayer perceptrons (see [23, 24, 29]), and, by means of Lyapunov’s second method of stability, the learning laws for their synaptic weights are derived.


## 2. Class of Nonlinear Differential Games

Consider the following continuous-time nonlinear dynamic game given by(1)x˙t=f(xt)+∑i=1N‍gi(xt)uti+Hξt,
where t∈[0,∞); the index i=1,2,3,…,N denotes the number of players; xt∈Rn is the state vector of the game; uti∈Uadmi⊆Rsi denotes the admissible control action vector of each player; the mappings f:Rn→Rn and gi:Rn→Uadmi are unknown nonlinear functions; ξt∈Rn denotes a known deterministic perturbation vector; and H is an unknown constant matrix of adequate dimensions.Similarly, consider now the following set of cost functions (or performance indexes) associated with each player and given by(2)Jti=∫0∞‍θi(xt,uti)dt,
where θi:Rn×Uadmi→R is well-defined for the i player.Moreover, the information structure of each player (denoted byηti) has a standard feedback (perfect state) pattern; that is to say,
(3)ηti={xt}
and a permissible control strategy (or control policy) of the i player is defined by the set of functions ρi(xt) satisfying
(4)ρi:Rn⟶Uadmi.
Nevertheless, the class of nonlinear differential games (1)–(4) is not completely described if the following assumptions are not fulfilled.Assumption 1.
The admissible control actionsuti are measurable and bounded for all time t∈[0,∞); that is,
(5)∥uti∥≤u¯i<∞,
where u¯i∈R are known constants.Assumption 2.
In order to guarantee the global existence and uniqueness of the solution of (1), the unknown nonlinear functions f(·) and gi(·) satisfy the Lipschitz condition; that is, there exist 0<ψ, ψi∈R constants such that the equations
(6)∥f(χ1)-f(χ2)∥≤ψ∥χ1-χ2∥,∥gi(χ1)-gi(χ2)∥≤ψi∥χ1-χ2∥
are fulfilled for all χ1,χ2∈X⊂Rn. Moreover, f(·) and gi(·) satisfy the following linear growth condition:
(7)∥f(χ)∥≤f¯1+f¯2∥χ∥,∥gi(χ)∥≤g¯1i+g¯2i∥χ∥,
where f¯1,f¯2,g¯1i,g¯2i∈R are known constants.Assumption 3.
Under the permissible control strategiesρi(xt), the (feedback) class of nonlinear differential games given by (1)–(4) is quadratically stable; that is, there exists a Lyapunov (maybe unknown) function 0≤Et:Rn→R such that the inequalities
(8)[∂Et∂xt][f(xt)+∑i=1N‍gi(xt)uti+Hξt]≤-λ1∥xt∥2,∥∂Et∂xt∥≤λ2∥xt∥
are satisfied for any known 0<λ1, λ2∈R constants.Assumption 4.
The deterministic perturbationξt is measurable and bounded for all time t∈[0,∞); that is,
(9)∥ξt∥≤ξ¯<∞,
where ξ¯∈R is a known constant.


## 3. Problem Statement

Let the class of continuous-time nonlinear dynamic games (1)–(4) be such that Assumptions 1 to 4 are fulfilled. Then, if one makes the following change of variables:
(10)y˙t=f(xt)+∑i=1N‍gi(xt)uti+Hξt,(11)z˙t=Hξt,
it is clear that the expected or uncorrupted vector state of the class of nonlinear differential games (1)–(4) can be defined as
(12)x¯t=yt-zt.Thereby, and in view of the fact thatf(·), gi(·), and H are unknown, the tackled problem in this paper is to obtain a feedback (perfect state) information pattern ηti={x^t}, given that x^t∈Rn satisfies the following filtering (or noise canceling) equation:
(13)x^t=y^t-z^t
and where y^t,z^t∈Rn are the state estimates of differential equations (10) and (11), respectively.


## 4. Differential Neural Networks Design

In order to solve the problem described above, consider a first differential neural network given by the following equation:(14)y^˙t=W1,tσ(V1,ty^t)+∑i=1N‍W2,tiφi(V2,tiy^t)uti,
where t∈[0,∞); i=1,2,3,…,N denotes the number of players; y^t∈Rn is the vector state of the neural network; the matrices W1,t∈Rn×k, V1,t∈Rk×n, W2,ti∈Rn×ri, and V2,ti∈Rri×n are synaptic weights of the neural network; and σ:Rk→Rk and φi:Rri→Rri×si are activation functions of the neural network.Remark 5.
For simplicity (and from now on)σ≔σ(V1,ty^t) and φi≔φi(V2,tiy^t).According to [29], the differential neural network (14) is classified as multilayer perceptrons and its structure was initially taken from [23, 24]. Also, this differential neural network only uses sigmoid activation functions; that is, σ and φi have a diagonal structure with elements
(15)σp(w)=ap1+e-bpw-cp|w=V1,ty^t,p=1,2,…,k(16)φqqi(wi)=aq1+e-bqwi-cq|wi=V2,tiy^t,q=1,2,…,min⁡(ri,si),
where 0<ap,bp,cp∈R and 0<aq,bq,cq∈R are known constants that manipulate the geometry of the sigmoid function.Thus, in view of the fact thatσ and φi are sigmoid activation functions, they are bounded and they satisfy the following equations:
(17)[σ~-A[y^t-yt]]TΛσ~[σ~-A[y^t-yt]]≤[y^t-yt]TΛσ[y^t-yt],[φ~iuti]TΛφ~i[φ~iuti]≤[u¯i]2[y^t-yt]TΛφi[y^t-yt],(18)σ~′=Dσ[V1,t-V1,0]y^t+νσ,φ~i′=〈Dφi,uti〉[V2,ti-V2,0i]y^t+νφiuti,(19)[νσ]TΛσ~′[νσ]≤lσ[[V1,t-V1,0]y^t]TΛνσ[[V1,t-V1,0]y^t],[νφiuti]TΛφ~′i[νφiuti]≤lφi[u¯i]2[[V2,ti-V2,0i]y^t]TΛνφi[[V2,ti-V2,0i]y^t],
where A, Λσ~, Λσ~′, Λνσ, Λσ, Λφ~i, Λφ~′i, Λνφi, Λφi, lσ, and lφi are known constants of adequate dimensions, and
(20)σ~≔σ(V1,0y^t)-σ(V1,0yt),φ~i≔φi(V2,0iy^t)-φi(V2,0iyt),σ~′≔σ(V1,ty^t)-σ(V1,0y^t),φ~i′≔φi(V2,tiy^t)-φi(V2,0iy^t),Dσ≔∂∂ψσ(ψ)∈Rk×k,Dφi≔∂∂ψφi(ψ)∈Rri.Nevertheless, there are some design conditions that this first differential neural network needs to satisfy.Assumption 6.
According to Assumption2, the approximation or residual error of (14) corresponding to the unidentified dynamics of (10) and that is given by
(21)Ft≔y˙t-[W1,0σ(V1,0yt)+∑i=1N‍W2,0iφi(V2,0iyt)uti],
where W1,0, V1,0, W2,0i, and V2,0i are initial synaptic weights (when t=0), is bounded and satisfies the following inequality:
(22)[Ft]TΛF[Ft]≤F¯1+F¯2[yt]TΛF[yt],
where 0<ΛF∈Rn×n is a known constant and 0<F¯1, F¯2∈R.Remark 7.
The constantsF¯1 and F¯2 in (22) are not knowna priori because of the fact that they depend on the performance of the differential neural network (14); that is to say, the residual error (21) will depend on the number of neurons used in (14) and on its parameters, and therefore F¯1 and F¯2 will depend on this too (see [24]).Assumption 8.
There exist values ofΛσ~, Λσ~′, Λφ~i, Λφ~′i, Λσ, Λφi, ΛF, lσ, lφi, A, and Q, such that they provide a 0<P=[P]T∈Rn×n solution to the algebraic Riccati equation as follows:
(23)PA¯+[A¯]TP+PRP+Q¯=0,
where
(24)A¯=W1,0A,(25)R=[W1,0][[Λσ~]-1+[Λσ~′]-1][W1,0]T+[ΛF]-1R=+∑i=1N‍[W2,0i][[Λφ~i]-1+[Λφ~′i]-1][W2,0i]T,(26)Q¯=2Q+Λσ+∑i=1N‍[u¯i]2Λφi,
where A such that A¯∈Rn×n is Hurwitz and Q is known.On the other hand, consider now a second differential neural network given by the following equation:(27)z^˙t=W3,tα(z^t)+W4,tγ(z^t)ξt,
where t∈[0,∞); z^t∈Rn is the vector state; the matrices W3,t∈Rn×δ and W4,t∈Rn×μ are synaptic weights; and α:Rn→Rδ and γ:Rn→Rμ×n are sigmoid activation functions.Then, similar toσ and φi, the activation functions α(·) and γ(·) satisfy the following inequalities:
(28)[α~-A[z^t-zt]]TΛα~[α~-A[z^t-zt]]≤[z^t-zt]TΛα[z^t-zt],[γ~ξt]TΛγ~[γ~ξt]≤[ξ¯]2[z^t-zt]TΛγ[z^t-zt],
where A, Λα~, Λα, Λγ~, and Λγ are known constants of adequate dimensions, and
(29)α~≔α(z^t)-α(zt),γ~≔γ(z^t)-γ(zt).Thereby, it is easy to confirm that ifα(·)=σ, if γ(·)=φi, if ξt=uti, and if V1,t=V2,ti=I, where i=1, then the differential neural networks (14) and (27) coincide. Hence, two new assumptions (corresponding to Assumptions 6 and 8) must be satisfied for this second differential neural network.Assumption 9.
According (again) to Assumption2, the approximation or residual error of (27) corresponding to the unidentified dynamics of (11) and that is given by
(30)Gt≔z˙t-[W3,0α(zt)+W4,0γ(zt)ξt],
where W3,0 and W4,0 are initial synaptic weights (when t=0), is bounded and satisfies the inequality
(31)[Gt]TΛG[Gt]≤G¯1+G¯2[zt]TΛG[zt],
where 0<ΛG∈Rn×n is a known constant and 0<G¯1, G¯2∈R.Remark 10.
Similar to Remark7, the constants G¯1 and G¯2 in (31) are not knowna priori because they depend on the performance of the differential neural network (27).Assumption 11.
If there exist values ofΛα~, Λγ~, ΛG, W3,0, and W4,0 such that
(32)[W4,0][Λγ~]-1[W4,0]T=∑i=1N‍[W2,0i][[Λφ~i]-1+[Λφ~′i]-1][W2,0i]T,W3,0=W1,0,Λα~=Λσ~+Λσ~′,ΛG=ΛF
and values of Λα and Λγ such that
(33)Q=Λα+[ξ¯]2Λγ-Λσ-∑i=1N‍[u¯i]2Λφi,
then they provide a 0<P=[P]T∈Rn×n solution to the algebraic Riccati equation (23), where
(34)A¯=W3,0A,(35)R=[W3,0][Λα~]-1[W3,0]T+[ΛG]-1+[W4,0][Λφ~i]-1[W4,0]T,(36)Q¯=Q+Λα+[ξ¯]2Λγ.Remark 12.
Notice that the equalities (32) and (33) were chosen in order to guarantee the same solution of the algebraic Riccati equation (23) in Assumptions 8 and 11. Also, notice that the first term of the right-hand side of (26) will always be greater than the first term of the right-hand side of (36); that is, 2Q>Q.


## 5. Main Result on Identification and Filtering

According to the above, the main result on identification and filtering for the class of nonlinear differential games (1)–(4) deals with both the development of an adaptive learning law for the synaptic weights of the differential neural networks (14) and (27) and the inference of a maximum value of identification error for the dynamics (10) and (11).Moreover, the establishment of a maximum value of filtering error between the uncorrupted states and the identified ones is obtained; namely, an error is defined by(37)et≔x^t-x¯t,
where x^t∈Rn is given by noise canceling equation (13) and x¯t∈Rn is given by the expected or uncorrupted vector state (12).More formally, the main obtained result is described in the following three theorems.Theorem 13.
Let the class of continuous-time nonlinear dynamic games (1)–(4) be such that Assumptions 1 to 4 are fulfilled. Also, let the differential neural network (14) be such that Assumptions 6 and 8 are satisfied. If the synaptic weights of (14) are adjusted with the following learning law:
(38)W˙1,t=-2KW1Pεt[σ]TV˙1,t=-KV1[lσ[Λνσ]TV~1,ty^t2[Dσ]T[W1,0]TPεt+lσ[Λνσ]TV~1,ty^t][y^t]T,W˙2,ti=-2KW2iPεt[uti]T[φi]T,V˙2,ti=-KV2i[+lφi[u¯i]2[Λνφi]TV~2,tiy^t2〈Dφi,uti〉[W2,0i]TPεt+lφi[u¯i]2[Λνφi]TV~2,tiy^t][y^t]T,
where
(39)εt≔y^t-yt
denotes the identification error, V~1,t≔V1,t-V1,0, V~2,ti≔V2,ti-V2,0i, and KW1, KV1, KW2i, and KV2i are known symmetric and positive-definite constant matrices, then it is possible to obtain the next maximum value of identification error in average sense KW2i as follows:
(40)limsup⁡t→∞(1t∫0t‍([ετ]T2Q[ετ])dτ)≤F¯1.Proof.
Taking into account the residual error (21), the differential equation (10) can be expressed as
(41)y˙t=W1,0σ(V1,0yt)+∑i=1N‍W2,0iφi(V2,0iyt)uti+Ft.
Then, by substituting (14) and (41) into the derivative of (39) with respect to t and by adding and subtracting the terms [W1,0σ], [W1,0σ(V1,0y^t)], [W2,0iφiuti], and [W2,0iφi(V2,0iy^t)uti], it is easy to confirm that
(42)ε˙t=W~1,tσ+W1,0σ~+W1,0σ~′-Ft+∑i=1N‍W~2,tiφiuti+∑i=1N‍W2,0iφ~iuti+∑i=1N‍W2,0iφ~i′uti,
where W~1,t≔W1,t-W1,0 and W~2,ti≔W2,ti-W2,0i. Now, let the Lyapunov (energetic) candidate function
(43)Lt≔Et+[εt]TP[εt]+12tr⁡([W~1,t]T[KW1]-1[W~1,t])Lt≔+12tr⁡([V~1,t]T[KV1]-1[V~1,t])Lt≔+12∑i=1N‍tr⁡([W~2,ti]T[KW2i]-1[W~2,ti])Lt≔+12∑i=1N‍tr⁡([V~2,ti]T[KV2i]-1[V~2,ti])
be such that the inequalities (8) are fulfilled (see Assumption 3), and let
(44)L˙t≤-λ1∥yt∥2+2[εt]TP[ε˙t]+tr⁡([W˙1,t]T[KW1]-1[W~1,t])+tr⁡([V˙1,t]T[KV1]-1[V~1,t])+∑i=1N‍tr⁡([W˙2,ti]T[KW2i]-1[W~2,ti])+∑i=1N‍tr⁡([V˙2,ti]T[KV2i]-1[V~2,ti])
be the derivative of (43) with respect to t. Then, by substituting (42) into the second term of the right-hand side of (44) and by adding and subtracting the term [2[εt]TPA¯εt], one may get
(45)2[εt]TP[ε˙t]=2[εt]TPW1,0[σ~-Aεt]+2[εt]TPW1,0σ~′+∑i=1N‍2[εt]TPW2,0iφ~iuti+∑i=1N‍2[εt]TPW2,0iφ~i′uti-2[εt]TPFt+2[εt]TPW~1,tσ+∑i=1N‍2[εt]TPW~2,tiφiuti+[εt]T[PA¯+[A¯]TP][εt].
Next, by analyzing the first five terms of the right-hand side of (45) with the following inequality:
(46)[X]TY+[Y]TX≤[X]T[Λ]-1[X]+[Y]TΛ[Y]
which is valid for any pair of matrices X,Y∈RΩ×Γ and for any constant matrix 0<Λ∈RΩ×Ω, where Ω and Γ are positive integers (see [24]), the following is obtained.(i)
Using (46) and (17) in the first term of the right-hand side of (45), then
(47)2[εt]TPW1,0[σ~-Aεt]≤[εt]T[P[W1,0][Λσ~]-1[W1,0]TP][εt]+[εt]TΛσ[εt].(ii)
Substituting (18) into the second term of the right-hand side of (45) and using (46) and (19), then
(48)2[εt]TPW1,0σ~′≤2[εt]TPW1,0DσV~1,ty^t+[εt]T[P[W1,0][Λσ~′]-1[W1,0]TP][εt]+lσ[V~1,ty^t]TΛνσ[V~1,ty^t].(iii)
Using (46) and (17) in the third term of the right-hand side of (45), then
(49)∑i=1N‍2[εt]TPW2,0iφ~iuti≤∑i=1N‍[εt]T[P[W2,0i][Λφ~i]-1[W2,0i]TP][εt]+∑i=1N‍[u¯i]2[εt]TΛφi[εt].(iv)
Substituting (18) into the fourth term of the right-hand side of (45) and using (46) and (19), then
(50)∑i=1N‍2[εt]TPW2,0iφ~i′uti≤∑i=1N‍2[εt]TPW2,0i〈Dφi,uti〉V~2,tiy^t+∑i=1N‍[εt]T[P[W2,0i][Λφ~′i]-1[W2,0i]TP][εt]+∑i=1N‍lφi[u¯i]2[V~2,tiy^t]TΛνφi[V~2,tiy^t].(v)
Using (46) and (22) in the fifth term of the right-hand side of (45), then
(51)-2[εt]TPFt≤[εt]T[P[ΛF]-1P][εt]+F¯1+F¯2[yt]TΛF[yt].So, by substituting (47)–(51) into the right-hand side of (45) and by adding and subtracting the term [[εt]T2Q[εt]], inequality (44) can be expressed as
(52)L˙t≤[εt]T[PA¯+[A¯]TP+PRP+Q¯][εt]-[εt]T2Q[εt]+F¯1+F¯2∥ΛF∥∥yt∥2-λ1∥yt∥2+tr⁡(ΥW1)+tr⁡(ΥV1)+∑i=1N‍tr⁡(ΥW2i)+∑i=1N‍tr⁡(ΥV2i),
where the algebraic Riccati equation in the first term of the right-hand side of (52) is described in (23) and in (24)–(26), and
(53)ΥW1=[W˙1,t]T[KW1]-1[W~1,t]+2σ[εt]TPW~1,tΥV1=[V˙1,t]T[KV1]-1[V~1,t]+2y^t[εt]TPW1,0DσV~1,t+lσy^t[y^t]T[V~1,t]TΛνσ[V~1,t],ΥW2i=[W˙2,ti]T[KW2i]-1[W~2,ti]+2φiuti[εt]TPW~2,ti,ΥV2i=[V˙2,ti]T[KV2i]-1[V~2,ti]+2y^t[εt]TPW2,0i〈Dφi,uti〉V~2,ti+lφi[u¯i]2y^t[y^t]T[V~2,ti]TΛνφi[V~2,ti].
Thereby, by equating (53) to zero, that is
(54)ΥW1=ΥV1=ΥW2i=ΥV2i=0,
and by, respectively, solving for W˙1,t, V˙1,t, W˙2,ti, and V˙2,ti, the learning law given by (38) is obtained. Now, by choosing
(55)∥ΛF∥≤λ1F¯2
and by solving the algebraic Riccati equation (23), one may get
(56)L˙t≤-[εt]T2Q[εt]+F¯1.
Thus, by integrating both sides of (56) on the time interval [0,t], the following is obtained:
(57)∫0t‍([ετ]T2Q[ετ])dτ≤L0-Lt+[F¯1]t≤L0+[F¯1]t,
and by dividing (57) by t, it is easy to verify that
(58)1t∫0t‍([ετ]T2Q[ετ])dτ≤L0t+F¯1.
Finally, by calculating the upper limit as t→∞, the maximum value of identification error in average sense is the one described in (40). This means that the identification error is bounded between zero and (40), and, therefore, this means that εt is stable in the sense of Lyapunov.Theorem 14.
Let the class of continuous-time nonlinear dynamic games (1)–(4) be such that Assumptions 1 to 4 are fulfilled. Also, let the differential neural network (27) be such that Assumptions 9 and 11 are satisfied. If the synaptic weights of (27) are adjusted with the following learning law:
(59)W˙3,t=-2KW3Mϵt[α(z^t)]T,W˙4,ti=-2KW4iMϵt[ξt]T[γ(z^t)]T,
where
(60)ϵt≔z^t-zt
denotes the identification error and KW3 and KW4i are known symmetric and positive-definite constant matrices, then it is possible to obtain the next maximum value of identification error in average sense KW2i as follows:
(61)limsup⁡t→∞(1t∫0t‍([ϵτ]TQ[ϵτ])dτ)≤G¯1.Proof.
Taking into account the residual error (30), the differential equation (11) can be expressed as
(62)z˙t=W3,0α(zt)+W4,0γ(zt)ξt+Gt.
Then, by substituting (27) and (62) into the derivative of (60) with respect to t, and by adding and subtracting the terms [W3,0α(z^)] and [W4,0γ(z^)ξt], it is easy to confirm that
(63)ϵ˙t=W~3,tα(z^)+W3,0α~+W~4,tγ(z^)ξt+W4,0γ~ξt-Gt,
where W~3,t≔W3,t-W3,0 and W~4,t≔W4,t-W4,0. Now, let the Lyapunov (energetic) candidate function
(64)Mt≔Et+[ϵt]TP[ϵt]+12tr⁡([W~3,t]T[KW3]-1[W~3,t])+12tr⁡([W~4,t]T[KW4]-1[W~4,t])
be such that the inequalities (8) are fulfilled (see Assumption 3), and let
(65)M˙t≤-λ1∥zt∥2+2[ϵt]TP[ϵ˙t]M˙t≤+tr⁡([W˙3,t]T[KW3]-1[W~3,t])M˙t≤+tr⁡([W˙4,t]T[KW4]-1[W~4,t])
be the derivative of (64) with respect to t. Then, by substituting (63) into the second term of the right-hand side of (65) and by adding and subtracting the term [2[ϵt]TPA¯ϵt], one may get
(66)2[ϵt]TP[ϵ˙t]=2[ϵt]TPW3,0[α~-Aϵt]+2[ϵt]TPW4,0γ~ξt-2[ϵt]TPGt+2[ϵt]TPW~3,tα(z^)+2[ϵt]TPW~4,tγ(z^)ξt+[ϵt]T[PA¯+[A¯]TP][ϵt].
Next, by analyzing the first three terms of the right-hand side of (66) with the inequality (46), the following is obtained.(i)
Using (46) and (28) in the first term of the right-hand side of (66), then
(67)2[ϵt]TPW3,0[α~-Aϵt]≤[ϵt]T[P[W3,0][Λα~]-1[W3,0]TP][ϵt]+[ϵt]TΛα[ϵt].(ii)
Using (46) and (28) in the second term of the right-hand side of (66), then
(68)2[ϵt]TPW4,0γ~ξt≤[ϵt]T[P[W4,0][Λγ~]-1[W4,0]TP][ϵt]+[ξ¯]2[ϵt]TΛγ[ϵt].(iii)
Using (46) and (31) in the third term of the right-hand side of (66), then
(69)-2[ϵt]TPGt≤[ϵt]T[P[ΛG]-1P][ϵt]+G¯1+G¯2[zt]TΛG[zt].So, by substituting (67)–(69) into the right-hand side of (66) and by adding and subtracting the term [[ϵt]TQ[ϵt]], inequality (65) can be expressed as
(70)M˙t≤[ϵt]T[PA¯+[A¯]TP+PRP+Q¯][ϵt]-[ϵt]TQ[ϵt]+G¯1+G¯2∥ΛG∥∥zt∥2-λ1∥zt∥2+tr⁡(ΨW3)+tr⁡(ΨW4),
where the algebraic Riccati equation in the first term of the right-hand side of (70) is described in (23) and in (34)–(36), and
(71)ΨW3=[W˙3,t]T[KW3]-1[W~3,t]+2α(z^)[ϵt]TPW~3,t,(72)ΨW4=[W˙4,t]T[KW4]-1[W~4,t]+2γ(z^)ξt[ϵt]TPW~4,t.
Thereby, by equating (71) and (72) to zero (ΨW3=ΨW4=0) and by, respectively, solving for W˙3,t and W˙4,t, the learning law given by (59) is obtained. Now, by choosing
(73)∥ΛG∥≤λ1G¯2
and by solving the algebraic Riccati equation (23), one may get
(74)M˙t≤-[ϵt]TQ[ϵt]+G¯1.
Thus, by integrating both sides of (74) on the time interval [0,t], the following is obtained:
(75)∫0t‍([ϵτ]TQ[ϵτ])dτ≤M0-Mt+[G¯1]t≤M0+[G¯1]t,
and by dividing (75) by t, it is easy to verify that
(76)1t∫0t‍([ϵτ]TQ[ϵτ])dτ≤M0t+G¯1.
Finally, by calculating the upper limit as t→∞, the maximum value of identification error in average sense is the one described in (61).Theorem 15.
Let the class of continuous-time nonlinear dynamic games (1)–(4) be such that Assumptions 1 to 4 are fulfilled. Also, let the differential neural networks (14) and (27) be such that the Assumptions 6 and 8 and Assumptions 9 and 11 are satisfied. If the synaptic weights of (14) and (27) are, respectively, adjusted with the learning laws (38) and (59), then it is possible to obtain the following maximum value of filtering error in average sense:
(77)limsup⁡t→∞(1t∫0t‍([eτ]TQ[eτ])dτ)≤F¯1-G¯1hhhhhhhhhhhhhhhhhhhhhhhhifF¯1>G¯1,(78)limsup⁡t→∞(1t∫0t‍([eτ]TQ[eτ])dτ)=0hhhhhhhhhhhhhhhhhhhifF¯1≤G¯1,
where et is given by (37).Proof.
Consider the following Lyapunov (energetic) candidate function:(79)Φt≔[et]TP[et],
where P is the solution of the algebraic Riccati equation (23) with A¯, R, and Q¯ defined by (24)–(26) or (34)–(36). Then, by calculating the derivative of (79) with respect to t and by considering the equations (37), (13), and (12), it can be verified that
(80)Φ˙t=2[et]TP[ε˙t-ϵ˙t],
and, taking into account (42), (63), and the proofs of Theorems 13 and 14, one may get
(81)Φ˙t≤-[et]T2Q[et]+F¯1-[-[et]TQ[et]+G¯1]≤-[et]T[2Q-Q][et]+F¯1-G¯1≤-[et]TQ[et]+F¯1-G¯1.
Thus, by integrating both sides of (81) on the time interval [0,t], the following is obtained:
(82)∫0t‍([eτ]TQ[eτ])dτ≤Φ0-Φt+[F¯1-G¯1]t≤Φ0+[F¯1-G¯1]t,
and by dividing (82) by t, it is easy to confirm that
(83)1t∫0t‍([eτ]TQ[eτ])dτ≤Φ0t+F¯1-G¯1.
By calculating the upper limit as t→∞, equation (83) can be expressed as
(84)limsup⁡t→∞(1t∫0t‍([eτ]TQ[eτ])dτ)≤F¯1-G¯1
and, finally, it is clear that
(i)
ifF¯1>G¯1, the maximum value of filtering error in average sense is the one described in (77);
(ii)
ifF¯1≤G¯1, the maximum value of filtering error in average sense is the one described in (78) given that the left-hand side of the inequality (84) cannot be negative.
Hence, the theorem is proved.


## 6. Illustrating Example

Example 16.
Consider a 2-player nonlinear differential game given by(85)x˙t=f(xt)+∑i=12‍gi(xt)uti+Hξt
subject to the change of variables (10) and (11) (i=2), where the control actions of each player are
(86)ut1=[0.5sawtooth(0.5t)H(t)],ut2=[H(t)0.5sawtooth(0.5t)].
The undesired deterministic perturbations are
(87)ξt=[sin(10t)sin(10t)]
and H(t) denotes the Heaviside step function or unit step signal. Then, under Assumptions 1–4, 6, 8, 9 and 11, it is possible to obtain a filtered feedback (perfect state) information pattern ηti={x^t}.Remark 17.
As told before, the functionsf(·) and gi(·) as well as the matrix H are unknown, but in order to make an example of (85) and its simulation, the values of these “unknown” parameters will be shown in the Appendix of this paper.
Thereby, consider now a first differential neural network given by(88)y^˙t=W1,tσ(V1,ty^t)+∑i=12‍W2,tiφi(V2,tiy^t)uti,
where
(89)W1,0=[0.50.1020.20.50.60],V1,0=[10.30.50.20.51.310.6]T,W2,01=[0.50.100.2],W2,02=[1.30.80.81.3],V2,01=[10.10.12],V2,02=[0.50.70.90.1]
and the activation functions are
(90)σp(w)=4.51+e-0.5w-3|w=V1,ty^t,p=1,2,3,4,φpq1(w1)=4.51+e-0.5w1-3|w1=V2,t1y^t,p=q=1,2,φpq2(w2)=4.51+e-0.5w2-3|w2=V2,t2y^t,p=q=1,2.
By proposing the values described in Assumption 8 as
(91)Λσ=Λφ1=Λφ2=[130013],Λνφ1=Λνφ2=[0.8000.8],Λφ~1=Λφ~2=Λφ~′1=Λφ~′2=[130013]-1,ΛF=[0.1000.1]-1,Λσ~=Λσ~′=[13000013000013000013]-1,Λνσ=[0.800000.800000.800000.8],A=[-1.3963-2.26320.0473-6.16060.426-7.4451-6.15330.8738],Q=[100010],lσ=lφ1=lφ2=1.3,u¯1=u¯2=5
and by choosing KW1=100 and KV1=KW21=KW22=KV21=KV22=1, the solution of the algebraic Riccati equation (23) results in
(92)P=[2.0524-0.6374-0.63743.1842].
Then, by applying the learning law (38) described in Theorem 13, the value of the identification error in average sense (40) on a time period of 25 seconds is
(93)limsup⁡t→25(1t∫0t‍([ετ]T2Q[ετ])dτ)=0.03145.Remark 18.
Even though0.03145 is the value of F¯1 on the time interval [0,25], it is not the global minimum value that F¯1 can take since t does not tend to infinity. So, in order to find this global minimum value, it would be required to simulate an arbitrarily large time.On the other hand, let a second differential neural network given by(94)z^˙t=W3,tα(z^t)+W4,tγ(z^t)ξt
be such that the equalities (32) and (33) are fulfilled; that is,
(95)W3,0=[0.50.1020.20.50.60],W4,0=[1001],Λα=[17300173],Λγ=[200020],Λα~=[26000026000026000026]-1,Λγ~=[67.3454.654.661.62]-1,ΛG=[0.1000.1]-1,ξ¯=5,
and the activation functions are
(96)αp(w)=4.51+e-0.5w-3|w=z^t,p=1,2,3,4,γpq(w)=4.51+e-0.5w1-3|w=z^t,p=q=1,2.
By choosing KW3=100000 and KW4=1, the solution of the algebraic Riccati equation (23) results in the matrix (92), and by applying the learning law (59) described in Theorem 14, the value of the identification error in average sense (61) on a time period of 25 seconds is
(97)limsup⁡t→25(1t∫0t‍([ϵτ]TQ[ϵτ])dτ)=0.00006645.Remark 19.
As in Remark18, in order to find the global minimum value of G¯1, it would be required to simulate an arbitrarily large time.Finally, taking into account the identification errors (93) and (97), the value of the filtering error in average sense (77)-(78) on a time period of 25 seconds is
(98)limsup⁡t→25(1t∫0t‍([eτ]TQ[eτ])dτ)=0.03139.
The simulation of this example was made using the MATLAB and Simulink platforms and its results are shown in Figures 1–4.Comparison betweenyt and y^t for the 2-player nonlinear differential game (85).
(a)
(b)Comparison betweenzt and z^t for the 2-player nonlinear differential game (85).
(a)
(b)Comparison betweenx¯t and x^t for the 2-player nonlinear differential game (85).
(a)
(b)Comparison betweenxt and x^t for the 2-player nonlinear differential game (85).
(a)
(b)As is seen in Figure1, the differential neural network (88) can perform the identification of the continuous-time nonlinear dynamic game (85), where y1,t and y2,t (or, similarly, x1,t and x2,t) denote the state variables with undesired perturbations and y^1,t and y^2,t indicate their state estimates.On the other hand, according to Figure2, the differential neural network (94) identifies the dynamics of the additive deterministic noises or, in other words, the dynamics of z˙=Hξt. Thus, z1,t and z2,t represent the state variables of the above differential equation and z^1,t and z^2,t are their state estimates.In this way, Figure3 shows the performance of the filtering process (13) in the nonlinear differential game (85); that is to say, it shows the comparison between the expected or uncorrupted state variables x¯1,t and x¯2,t and the filtered state estimates x^1,t and x^2,t.Finally, Figure4 also exhibits the described filtering process but comparing the real state variables x1,t and x2,t with filtered state estimates x^1,t and x^2,t.


## 7. Results Analysis and Discussion

Although the differential neural networks (14) and (27) can perform the identification of the differential equations (10) and (11), it is important to remember that their performance depends on the number of neurons used and on the proposition of all the constant values (or free parameters) that were described in Assumptions 8 and 11. Therefore, the values of F¯1 and G¯1 (at a fixed time) can change according to this fact.In other words, due to the fact that the differential neural networks are an approximation of a dynamic system (or game), there always will be a residual error that will depend on this approximation.Thus, in the particular case shown in Example16, the design of (88) was made using only eight neurons, four for the layer of perceptrons without any relation to the players and two for the layer of perceptrons of each player. Similarly, for the design of (94) (which is subject to the equalities (32)-(33)), six neurons were used.On the other hand, it is important to mention that (14) and (27) will operate properly only if Assumptions 1–4, 6, 8, 9 and 11 are satisfied; that is to say, there is no guarantee that these differential neural networks perform good identification and filtering processes if, for example, the class of nonlinear differential games (1)–(4) has a stochastic nature.Finally, although this paper presents a new approach for identification and filtering of nonlinear dynamic games, it should be emphasized that there exist other techniques that might solve the problem treated here, for example, the cited publications in Section1.


## 8. Conclusions and Future Work

According to the results of this paper, the differential neural networks (14) and (27) solve the problem of identifying and filtering the class of nonlinear differential games (1)–(4).However, there is no guarantee that (14) and (27) identify and filter the states of (1)–(4) if Assumptions 1–4, 6, 8, 9 and 11 are not met.Thereby, the proposed learning laws (38) and (59) obtain the maximum values of identification error in average sense (40) and (61), that is to say, the maximum value of filtering error (77)-(78).Nevertheless, these errors depend on both the number of neurons used in the differential neural networks (14) and (27) and the proposed values applied in their design conditions.On the other hand, according to the simulation results of the illustrating example, the effectiveness of (14) and (27) is shown and the applicability of Theorems 13, 14, and 15 is verified.Finally, speaking of future work in this research field, one can analyze and discuss the use of (14) and (27) for the obtaining of equilibrium solutions in the class of nonlinear differential games (1)–(4), that is to say, the use of differential neural networks for controlling nonlinear differential games.



---

*Source: 306761-2014-06-11.xml*