# An Online Distributed Game Optimal Control for Heavy Haul Trains with Limited Communication

Authors: Kai Gao; Zhiwu Huang; Jun Peng; Heng Li; Weirong Liu
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

For heavy haul trains, it is difficult to get global information due to the limited range
of communication. This paper proposed a novel distributed optimal control based on game
strategy, in which the global optimization is achieved by equilibrizing subsystems’ performance
just utilizing local information. To online solve the game control, an efficient multivariable
extremum seeking algorithm was adapted to approximate the partial differential equation
deduced by optimal condition. The convergence of the proposed approximate algorithm
is proved by constructing a fictitious Lie bracket system using Lyapunov function. Finally,
the proposed distributed optimal control is valuated rigorously by case study according to the
configuration of Daqin railway in China.


---

## Body

## 1. Introduction

Heavy haul trains are used broadly in many countries with high demand for transporting mineral, petroleum, coals, and so on. Essentially, heavy haul trains are distributed powered networked system constituted with many locomotives and wagons. The basic control problem for heavy haul trains is tracking to the target speed profile while considering some performance index, including the traveling time, energy consumption, and in-train forces. Then, the optimal controller should be designed to achieve the performance.The optimal problem has received many attentions recently in the field of train’s operating control [1–5]. Energy consumption and traveling time focused optimal control was studied in these works for either passenger trains or ordinary freight trains. But much larger dynamic in-train forces were imposed in heavy haul trains due to unreasonable control of the distributed power, undulating grades, and also the lager train lengths. The couplings used in heavy haul trains wear out due to large in-train forces [1]. So the optimization techniques used for passenger trains or ordinary freight trains may not be directly applicable to heavy haul trains.Researches on optimal control of heavy haul trains were done about travelling time, in-train forces, and energy consumption. An optimal open-loop offline controller is designed for cruise control of heavy haul trains [6]. It is derived that distributed controller can be more useful to minimize the in-train forces. But static error always exists for the reason of open loop. Based on this, the authors did further research on closed-loop optimal controller of heavy haul trains with consideration of in-train forces, speed tracking error, and energy consumption. It concluded that the 2-2 mode (full distributed) controller’s performance about in-train forces is much better than 1-1 mode (centralized). A similar conclusion was derived in [7]. In these literatures, a LQR (linear quadratic regulator) optimal method is used to optimize the feedback gain using global information [6, 7]. The global information is gathered by communication networks laid on the train, for example, Lonworks (used in electronically controlled pneumatic (ECP)), GSM-R, or LTE-R. But the global information is not always available during the traveling due to the range of communication, disturbance, delay, and communication failure. Therefore, design of a distributed optimal controller for each powered locomotive and wagon is more practical and intriguing using available local information about the whole system.Local performance index should be constructed first when designing distributed optimal controllers for heavy haul trains. The subsystems in the train work in a team to track a desired speed profile. At the same time, each of them tries to minimize the energy consumption and in-train forces at the cost of increasing others, which is very complex and difficult. Game strategy is an efficient method to solve these types of problems with conflict benefits [8]. Distributed optimal control problem was solved by game strategy for multiagent system in some works [9–11].In [9], a noncooperative game was designed to distributed optimization with fixed local communication. The authors design estimators to estimate other players’ states which make the controller compute heavily. Since there is information exchange among the players, there is no need to design so many estimators. A most related work is done in [10], which brings together cooperative control, reinforcement learning, and game theory to solve multiplayer differential games on communication graph topologies. It formulates graphical games for dynamic systems and provides policy iteration and learning algorithms along with proof of convergence to the Nash equilibrium. Since there are communications, the subsystems cooperate to track the target speed.A differential game in the sense of cooperation is constructed in [12] and Pareto equilibrium is further studied for cooperative game [13]. Analysis of convergence to Pareto equilibrium was done and the necessary and sufficient conditions were derived. It showed that if the dynamic system is controllable then all Pareto candidates can be obtained by solving the necessary conditions of a weighted sum optimal control problem. However, this is done offline and global information is required.In this paper, we made an improvement on these works. We propose a novel distributed optimal control based on game strategy, in which the global optimization is achieved by equilibrizing subsystems’ performance just utilizing local information. To online solve the game, an efficient multivariable extremum seeking algorithm was adapted to approximate the partial differential equation deduced by optimality condition. The proposed controller is applied to heavy haul trains with second order interconnected model.The main contribution of this paper includes four aspects. Firstly, a distributed optimal controller using local information with conflict performance index is proposed for heavy haul trains with a second order interconnected model. A system level global target was attained by optimizing the local performance with proper communication topology. Secondly, a cooperative differential game is constructed to solve the distributed optimal problem with conflicting individual objectives. We proved that a Pareto equilibrium would be reached when the communication topology of the subsystem in heavy haul trains was strongly connected. Thirdly, we proved that the cooperative game can be solved only by local information, while, in most existing literatures, global information is necessary for solving the game. Lastly, we develop an efficient multivariable extremum seeking algorithm, which was proposed to approximate the partial differential equation deduced by solving the cooperative game.The remainder of this paper is organized as follows. In Section2, the model of heavy haul trains is given, including kinetic model and communication connections model. In Section 3, the distributed optimal control for heavy haul trains is formulated and then a cooperative game is constructed. In Section 4, the stability and convergence of formulated cooperative game are analyzed. In Section 5, a multivariable extremum seeking algorithm is designed to approximate the partial differential equation deduced by solving the cooperative game. In Section 6, a simulation scenario is set based on application. At last, the conclusion is given.


## 2. Model of Heavy Haul Trains

In this section, the formulation of distributed optimal control problem and the train model used in this paper is summarized.There are two types of model of heavy haul trains discussed in the existing literatures, namely, point-mass model used in [3, 14] and spring-mass model used in [7, 15]. Spring-mass model is mostly used when considering the complex in-train dynamics and the effectiveness is validated against experimental data collected on a train operated by Spoornet on its COALlink in South Africa [16]. The wired Lonworks communication or wireless GSM-R and LTE-R communication is used to operate heavy haul trains more efficiently with the development of communication based train control system. In our paper, the spring-mass model of heavy haul trains is used with considering the communication connections among the cars (for ease of presentation, in what follows both locomotives and wagons in heavy haul trains are referred to as cars), which is more realistic.


### 2.1. Kinetic Model of Heavy Haul Trains

The longitudinal dynamic characters of heavy haul trains can be described by the following set of equations [7, 16]:
(1)x˙1=v1,v˙1=u1m1-F1Cm1,x˙i=vi,v˙i=uimi-FiCmi+Fi-1Cmi,x˙n=vn,v˙n=unmn+Fn-1Cmn,
where i=2,…,n-1 and n (positive integer) is the number of cars in heavy haul trains. When xi, vi, and mi denote the displacement with respect to an inertial frame, the speed, and the mass of the ith car, respectively, ui is the traction force or braking force added to the ith car. FiC is the in-train force between the ith and (i+1)th car and can be formulated as follows:
(2)FiC=kixi-xi+1+dix˙i-x˙i+1,
where ki is the coupler’s spring constant of the ith car and di is the coupler’s damp constant of the ith car.The in-train forces can be viewed as the coupling force between the cars and we can rewrite (1) as follows:
(3)z→˙i=A→z→i+∑j=i-1i+1‍cijΘz→j+B→ui,
where the letter x→ is used to denote a vector. z→i=xiviT∈R2 is the ith car’s states vector, the matrix A→=01;00, the matrix B→=01T, cij is coupling coefficient, and cij=1 when i≠j, cij=-2 when i=j, Θ=00;k/md/m is the coupling matrix where we suppose that all the ki and di are equal, if the types of couplers in the train are the same actually, and ui is the control input.


### 2.2. Model of Communication Connections among the Cars

As a benefit from the communication network equipment laid on heavy haul trains, more flexible control strategy can be designed. Lonworks is the most popular network protocol used in heavy haul trains equipped with ECP/iDP systems as shown in Figure1(a). However, heavy haul trains need to marshal when loading and unloading freight, and then the wireless network equipped on the train will be more flexible, such as LTE-R (Long Term Evolution-Railway), as shown in Figure 1(b).Two types of network equipped on heavy haul trains. In (a)Ci (i∈n, n is the number of cars in the train) is the ith controller equipped on the cars. They can communicate with each other by wired Lonworks protocol. Ji is the conjunction box between the cars. (b) It is a wireless network configuration of heavy haul trains. The physical connections are ignored in the figure.
(a)
Wired network equipped on heavy haul trains
(b)
Wireless network equipped on heavy haul trainsRemark 1.
It is obvious that there are some drawbacks of configuration as shown in Figure1(a). If one of the nodes fails, the whole network will be broken. Moreover, frequent marshalling of the train makes it difficult and expensive to maintain. Configuration (b) is much more flexible and can provide larger bandwidth. LTE-R has already been laid out in Shuohuang Railway in China.A direction graph is used to describe the communication among subsystems. According to the trains model equation (1), there are n vertexes v={v1,v2,…,vn} in the Graph G(v,ε,A), which is used to describe the communication among the cars (a node in the graph) similarly as [17]. The set of edges or arcs is ε⊆v×v. An edge from node j to node i is denoted by (vj,vi), which means that node i receives the information from node j. The associated adjacency matrix A=aij∈Rn×n, where aij=1 if (vj,vi)∈ε, means node i can receive information from node j, aij=0 otherwise. Particulalry, aii=0; that is to say, there is no self-loop in the Graph. Node j is called a neighbor of node i if (vj,vi)∈ε. The set of neighbors of node i is denoted as Ni=vj∈v:(vj,vi)∈ε. The degree of node i is defined as di=∑j∈Ni‍aij. N∖{i} is used to denote all of the cars except i. If there is a sequence of nodes v1,v2,…,vj such that (vi,vi+1)∈ε, i∈{1,2,…,j-1}, then there is a path from v1 to vj. A direction graph is said to have a spanning tree, if there is a node ir (called the root), such that there is a directed path from the root to every other node in the graph. In the communication graph G, if there is a path from node i to node j, we denote that j is reachable from i. If there is at least a path between any two nodes i and j, the graph is strongly connected [18].


### 2.3. Desired Speed Profile

The heavy haul trains travel according to a desired speed profile and the speed profile can be expressed as(4)x˙0=v0,v˙0=u0,
where x0, v0, and u0 are the desired displacement, speed, and input, respectively, and z→0=x0v0T.Usually, we can view the desired speed profile as the operator’s command and can be scheduled offline due to experience or online heuristic or close-loop global optimality of energy consumption, traveling time, and so on. However how to design the desired speed profile is not our focus in this paper. If the desired speed profile is scheduled appropriately, and the heavy haul trains can track the profile well under our proposed controller, the trains can travel with favorable performance.


## 2.1. Kinetic Model of Heavy Haul Trains

The longitudinal dynamic characters of heavy haul trains can be described by the following set of equations [7, 16]:
(1)x˙1=v1,v˙1=u1m1-F1Cm1,x˙i=vi,v˙i=uimi-FiCmi+Fi-1Cmi,x˙n=vn,v˙n=unmn+Fn-1Cmn,
where i=2,…,n-1 and n (positive integer) is the number of cars in heavy haul trains. When xi, vi, and mi denote the displacement with respect to an inertial frame, the speed, and the mass of the ith car, respectively, ui is the traction force or braking force added to the ith car. FiC is the in-train force between the ith and (i+1)th car and can be formulated as follows:
(2)FiC=kixi-xi+1+dix˙i-x˙i+1,
where ki is the coupler’s spring constant of the ith car and di is the coupler’s damp constant of the ith car.The in-train forces can be viewed as the coupling force between the cars and we can rewrite (1) as follows:
(3)z→˙i=A→z→i+∑j=i-1i+1‍cijΘz→j+B→ui,
where the letter x→ is used to denote a vector. z→i=xiviT∈R2 is the ith car’s states vector, the matrix A→=01;00, the matrix B→=01T, cij is coupling coefficient, and cij=1 when i≠j, cij=-2 when i=j, Θ=00;k/md/m is the coupling matrix where we suppose that all the ki and di are equal, if the types of couplers in the train are the same actually, and ui is the control input.


## 2.2. Model of Communication Connections among the Cars

As a benefit from the communication network equipment laid on heavy haul trains, more flexible control strategy can be designed. Lonworks is the most popular network protocol used in heavy haul trains equipped with ECP/iDP systems as shown in Figure1(a). However, heavy haul trains need to marshal when loading and unloading freight, and then the wireless network equipped on the train will be more flexible, such as LTE-R (Long Term Evolution-Railway), as shown in Figure 1(b).Two types of network equipped on heavy haul trains. In (a)Ci (i∈n, n is the number of cars in the train) is the ith controller equipped on the cars. They can communicate with each other by wired Lonworks protocol. Ji is the conjunction box between the cars. (b) It is a wireless network configuration of heavy haul trains. The physical connections are ignored in the figure.
(a)
Wired network equipped on heavy haul trains
(b)
Wireless network equipped on heavy haul trainsRemark 1.
It is obvious that there are some drawbacks of configuration as shown in Figure1(a). If one of the nodes fails, the whole network will be broken. Moreover, frequent marshalling of the train makes it difficult and expensive to maintain. Configuration (b) is much more flexible and can provide larger bandwidth. LTE-R has already been laid out in Shuohuang Railway in China.A direction graph is used to describe the communication among subsystems. According to the trains model equation (1), there are n vertexes v={v1,v2,…,vn} in the Graph G(v,ε,A), which is used to describe the communication among the cars (a node in the graph) similarly as [17]. The set of edges or arcs is ε⊆v×v. An edge from node j to node i is denoted by (vj,vi), which means that node i receives the information from node j. The associated adjacency matrix A=aij∈Rn×n, where aij=1 if (vj,vi)∈ε, means node i can receive information from node j, aij=0 otherwise. Particulalry, aii=0; that is to say, there is no self-loop in the Graph. Node j is called a neighbor of node i if (vj,vi)∈ε. The set of neighbors of node i is denoted as Ni=vj∈v:(vj,vi)∈ε. The degree of node i is defined as di=∑j∈Ni‍aij. N∖{i} is used to denote all of the cars except i. If there is a sequence of nodes v1,v2,…,vj such that (vi,vi+1)∈ε, i∈{1,2,…,j-1}, then there is a path from v1 to vj. A direction graph is said to have a spanning tree, if there is a node ir (called the root), such that there is a directed path from the root to every other node in the graph. In the communication graph G, if there is a path from node i to node j, we denote that j is reachable from i. If there is at least a path between any two nodes i and j, the graph is strongly connected [18].


## 2.3. Desired Speed Profile

The heavy haul trains travel according to a desired speed profile and the speed profile can be expressed as(4)x˙0=v0,v˙0=u0,
where x0, v0, and u0 are the desired displacement, speed, and input, respectively, and z→0=x0v0T.Usually, we can view the desired speed profile as the operator’s command and can be scheduled offline due to experience or online heuristic or close-loop global optimality of energy consumption, traveling time, and so on. However how to design the desired speed profile is not our focus in this paper. If the desired speed profile is scheduled appropriately, and the heavy haul trains can track the profile well under our proposed controller, the trains can travel with favorable performance.


## 3. Control Problem Formulation

### 3.1. Distributed Optimal Control with Neighbor’s Information

Due to the different control requirements caused by undulating grades of the railway and unpredictable communication failure among cars in the train, distributed controllers are designed by neighbor’s information (available information from cars in the train, and global information is not necessary), with considering some performance index to track the desired speed profile. The disagreement of cari to his neighbors and the desired speed profile is denoted in
(5)η→i=∑j∈Ni‍aijz→i-z→j+li0z→i-z→0,
where aij is defined in the section of communication model and li0=1 if the car i can directly receive the operator’s command.The controllers in the train cooperate to achieve a system level objective. So, the objective of our work is to designui in (1) with optimizing the local performance index Ji as designed in
(6)Ji=∫0∞‍Liη→i,ui,tdt=∫0∞‍η→iQiiη→iT+uiR→iiuiT+∑j∈Ni‍ujR→ijujTdt,
where Qii, R→ii, and R→ij are weight coefficients. In the performance index Ji, Qii is the weight of tracking desired speed and the in-train forces and R→ii is the weight of energy consumption and R→ij is the weight of energy consumption of neighbor cars. To minimize the performance index Ji means optimization about the train’s travel time, in-train forces, and energy consumption [7]. The significant difference is Ji defined in (6) depending only on local neighbors’ information and the performance index may be coupled and conflict. We can see that decreasing the input of one car will result in increasing inputs of others, which makes solving of optimal control problem very difficult. The distributed optimal control problem discussed in this paper will be formulated as a cooperative game problem, which is capable of solving problems of many individuals with cooperative and/or conflict interests.


### 3.2. Applying Cooperative Game Strategy to Heavy Haul Trains

As discussed in the section of distributed optimal control problem, it is converted to a cooperative game. The detailed description and definition will be given as follows.A four-tuplez→iϕiuiJi is used to describe the state of a car in the train. ui is a controller’s decision value, namely, control input, considering the neighbors’ states. u-i denotes the decision values of i’s neighbors. Ui is used to denote the set of available decisions for car i. A tuple u→=(u1,u2,…,un)=ΠiUi∈U is denoted as a joint value. A function Vi(ηi(t)) is used to denote the cost of taking decision ui. The cost function is defined as (7), which coincides with performance index Ji:
(7)Viη→it=∫t∞‍η→iQiiη→iT+uiRiiuiT+∑j∈Ni‍ujRijujTdt.Assumption 2.
Assume thatVi(η→i(t)) is continuous and first-order derivative and from the expression equation (7), it is an integral of quadratic form function, then Vi is nonnegative.
The neighbors’ states are available to make a decision for cari. It is always impossible to minimize the performance index for a single car without increasing that of other cars. It implies that there exists an equilibrium for the whole system, which coincides with the characters of Pareto optimality. The Pareto optimality equilibrium for cooperative game is defined in [12], with global information of the system.Lemma 3.
Global Pareto optimality:u* are Pareto optimal solutions for the formulated cooperative game, if and only if, for ∀i∈n, u* minimizes Ji(u) on the constrained set:
(8)Ui≜u∈U∣Jju*≤Jju,∀j∈N∖i.However, when only partial information is available to design the decision during game, Lemma3 needs to be extended in the sense of local Pareto optimality equilibrium. If the communication topology of the system meets some conditions, the global Pareto optimality equilibrium will be reached on the basis of local Pareto optimality equilibrium. The definition of local Pareto optimality equilibrium is given in Definition 4.Definition 4.
Local Pareto optimality: subsystemi’s Pareto best response to neighbours’ fixed u-i,i∈Ni is the rule ui* such that minimizing Ji and for all j∈Ni, the following inequation holds, Jj(ui*,u-i*)≤Jj(ui,u-i).The objective of the paper is to design a decision making algorithm while playing the cooperative game. The game converges to a global Pareto optimality when every player is achieving local Pareto optimality.For a distributed optimal problem formulated in our paper, if the designed decision making ruleui, ∀i to play the game is continuous and ui(0)=0 can stabilize the disagreement η→i of every car locally with minimizing Ji, and the value function Vi(η→i(t)) is finite, we say that a global optimization is reached under distributed optimal controllers by cooperative game.Designui to minimize the value function Vi(η→i(t)) subject to the state transition function of the system equation (3) and the initial states z→i(0) with states bound γ_i≤ui≤γ¯i.By Lemma3 and conclusion from [13], this game can be solved by a group of constrained optimal equations. Construct a Hamiltonian function with boundary conditions and γ_i≤ui≤γ¯i as
(9)Hiui,u-i,η→i,z→0=ξitϕiz→i,z→j,ui+Li+ujR→jjujT+ζiui-γ¯i+κiγ_i-ui,
where ϕi(z→i,z→j,ui)=z→˙i and z→˙i is denoted in (3), Li is defined in (6), and ζi and κi are positive real numbers [19]. To minimize the local performance index equation (6) with constrains, by Pontryagin’s minimum principle and Karush-Kuhn-Tucker condition, the following equations should be satisfied:
(10)dHi(·)dt=0,z→˙i*=ϕiz→i*,ξ˙it=-∂Hi∂η→i,∂Hi∂ui=0,z→0=z→0,
where
(11)ξ˙it=-A→ξit-2Q→iiη→i,(12)∂Hi∂ui=ξi(t)(di+li0)B→T+2uiR→ii-κi+ζi⟹ui=-12ξi(t)(di+li0)R→ii-1B→T-κiR→ii-11→2+ζiR→ii-11→2
if κi=ζi, (12) can be written as
(13)ui=-12ξitdi+li0R→ii-1B→T.


## 3.1. Distributed Optimal Control with Neighbor’s Information

Due to the different control requirements caused by undulating grades of the railway and unpredictable communication failure among cars in the train, distributed controllers are designed by neighbor’s information (available information from cars in the train, and global information is not necessary), with considering some performance index to track the desired speed profile. The disagreement of cari to his neighbors and the desired speed profile is denoted in
(5)η→i=∑j∈Ni‍aijz→i-z→j+li0z→i-z→0,
where aij is defined in the section of communication model and li0=1 if the car i can directly receive the operator’s command.The controllers in the train cooperate to achieve a system level objective. So, the objective of our work is to designui in (1) with optimizing the local performance index Ji as designed in
(6)Ji=∫0∞‍Liη→i,ui,tdt=∫0∞‍η→iQiiη→iT+uiR→iiuiT+∑j∈Ni‍ujR→ijujTdt,
where Qii, R→ii, and R→ij are weight coefficients. In the performance index Ji, Qii is the weight of tracking desired speed and the in-train forces and R→ii is the weight of energy consumption and R→ij is the weight of energy consumption of neighbor cars. To minimize the performance index Ji means optimization about the train’s travel time, in-train forces, and energy consumption [7]. The significant difference is Ji defined in (6) depending only on local neighbors’ information and the performance index may be coupled and conflict. We can see that decreasing the input of one car will result in increasing inputs of others, which makes solving of optimal control problem very difficult. The distributed optimal control problem discussed in this paper will be formulated as a cooperative game problem, which is capable of solving problems of many individuals with cooperative and/or conflict interests.


## 3.2. Applying Cooperative Game Strategy to Heavy Haul Trains

As discussed in the section of distributed optimal control problem, it is converted to a cooperative game. The detailed description and definition will be given as follows.A four-tuplez→iϕiuiJi is used to describe the state of a car in the train. ui is a controller’s decision value, namely, control input, considering the neighbors’ states. u-i denotes the decision values of i’s neighbors. Ui is used to denote the set of available decisions for car i. A tuple u→=(u1,u2,…,un)=ΠiUi∈U is denoted as a joint value. A function Vi(ηi(t)) is used to denote the cost of taking decision ui. The cost function is defined as (7), which coincides with performance index Ji:
(7)Viη→it=∫t∞‍η→iQiiη→iT+uiRiiuiT+∑j∈Ni‍ujRijujTdt.Assumption 2.
Assume thatVi(η→i(t)) is continuous and first-order derivative and from the expression equation (7), it is an integral of quadratic form function, then Vi is nonnegative.
The neighbors’ states are available to make a decision for cari. It is always impossible to minimize the performance index for a single car without increasing that of other cars. It implies that there exists an equilibrium for the whole system, which coincides with the characters of Pareto optimality. The Pareto optimality equilibrium for cooperative game is defined in [12], with global information of the system.Lemma 3.
Global Pareto optimality:u* are Pareto optimal solutions for the formulated cooperative game, if and only if, for ∀i∈n, u* minimizes Ji(u) on the constrained set:
(8)Ui≜u∈U∣Jju*≤Jju,∀j∈N∖i.However, when only partial information is available to design the decision during game, Lemma3 needs to be extended in the sense of local Pareto optimality equilibrium. If the communication topology of the system meets some conditions, the global Pareto optimality equilibrium will be reached on the basis of local Pareto optimality equilibrium. The definition of local Pareto optimality equilibrium is given in Definition 4.Definition 4.
Local Pareto optimality: subsystemi’s Pareto best response to neighbours’ fixed u-i,i∈Ni is the rule ui* such that minimizing Ji and for all j∈Ni, the following inequation holds, Jj(ui*,u-i*)≤Jj(ui,u-i).The objective of the paper is to design a decision making algorithm while playing the cooperative game. The game converges to a global Pareto optimality when every player is achieving local Pareto optimality.For a distributed optimal problem formulated in our paper, if the designed decision making ruleui, ∀i to play the game is continuous and ui(0)=0 can stabilize the disagreement η→i of every car locally with minimizing Ji, and the value function Vi(η→i(t)) is finite, we say that a global optimization is reached under distributed optimal controllers by cooperative game.Designui to minimize the value function Vi(η→i(t)) subject to the state transition function of the system equation (3) and the initial states z→i(0) with states bound γ_i≤ui≤γ¯i.By Lemma3 and conclusion from [13], this game can be solved by a group of constrained optimal equations. Construct a Hamiltonian function with boundary conditions and γ_i≤ui≤γ¯i as
(9)Hiui,u-i,η→i,z→0=ξitϕiz→i,z→j,ui+Li+ujR→jjujT+ζiui-γ¯i+κiγ_i-ui,
where ϕi(z→i,z→j,ui)=z→˙i and z→˙i is denoted in (3), Li is defined in (6), and ζi and κi are positive real numbers [19]. To minimize the local performance index equation (6) with constrains, by Pontryagin’s minimum principle and Karush-Kuhn-Tucker condition, the following equations should be satisfied:
(10)dHi(·)dt=0,z→˙i*=ϕiz→i*,ξ˙it=-∂Hi∂η→i,∂Hi∂ui=0,z→0=z→0,
where
(11)ξ˙it=-A→ξit-2Q→iiη→i,(12)∂Hi∂ui=ξi(t)(di+li0)B→T+2uiR→ii-κi+ζi⟹ui=-12ξi(t)(di+li0)R→ii-1B→T-κiR→ii-11→2+ζiR→ii-11→2
if κi=ζi, (12) can be written as
(13)ui=-12ξitdi+li0R→ii-1B→T.


## 4. Stability and Convergence of Cooperative Game

Theorem 5.
Let cari in the heavy haul trains be reachable for all i∈n, and let every car i be in local Pareto optimality as Definition 4 to all of its neighbors j∈Ni. Then all the cars are in global Pareto optimality as Lemma 3 if and only if the graph is strongly connected.Proof.
If every cari is in local Pareto optimality to its neighbours j∈Ni, then follow the Definition 4, we have ∀i, and Ji is minimized at Jj(ui,u-i)≤Jj(ui*,u-i*), where j∈Ni. We analyze the problem from a viewpoint of propagation. Choose any car labeled i, its neighbours are Ni, and not all the cars are included in this group. Then for this group Gri, the Pareto optimal is reached. Because the graph is strongly connected, for the group Gri, there is at least a car k which is reachable for other cars not in group Gri. It is the same for the group Grk which is different from Gri. Then the Pareto optimality is propagated from Gri to Grk. The Pareto optimality is propagated to other cars until there is no any car left in the graph. Then we can say that the global Pareto optimality is reached.
Necessity. If the graph is not strongly connected, there is at least a pair of i and j between which there is no path. Then the Pareto optimalilty related with i cann’t propagate to j, and then the global Pareto optimality is not achieved.
Sufficiency. When the graph is strongly connected, the global Pareto optimality is not achieved and only some local Pareto optimality is achieved, for example, Gri, Grj, and Grk. That is to say, i∈Ni, i∉Nj, and i∉Nk. By the view of Pareto optimality propagation, we can deduce that there is no path from i to j or k, so this is contradictory to the graph that is strongly connected.Theorem 6 (stability of cooperative distributed optimal controller).
Let Assumption2 hold. To design a distributed cooperative optimal controller through game, according to (13), aiming at effecting on state disagreement and deviation to the desired profile, ui should be related with the variety of ηi, that is ∂Vi/∂ηi. Given ξi(t)=∂Vi/∂ηi, ∀i, under the Pareto optimal ui*, the heavy haul trains are asymptotically stable and terminally converge to the desired speed profile.Proof.
To analyze the stability of cooperative distributed optimal control, we utilize the Lyapunov function. Choose the value functionVi as a Lyapunov function, Vi≥0. The time derivative of Vi is as follows:
(14)V˙i=∂ViT∂η→iη→˙i=-Li-∑j∈Ni‍η→jQjjη→jT+ujR→jjujT+∑k∈Nj‍ukR→jkukT,
where Li is designated in (6). The terms in V˙i are quadratic form, and then we can derive that V˙i<0. So the system is asymptotically stable and η→i→0. That is to say z→i→z→j→z→0, the states converge to the desired profile.


## 5. Multivariable Extremum Seeking Algorithm for Cooperative Game

To drive the cooperative game to a Pareto optimal equilibrium, the decision rule equation (13) must be computable. According to Theorem 6, ui is in the form of
(15)ui=-12∂Vi∂η→idi+li0R→ii-1B→T.Substitutingui into (9), we derive a partial differential equation. It is hard to get an analytical solution for such an equation. Extremum seeking algorithm provides a numerical perturbation based method to steer an unknown dynamical system to the optimality. Motivated by [20, 21], a multivariable extremum seeking estimator is designed to solve the Pareto optimality. A basic multivariable extremum seeking algorithm is given in [22], which is based on perturbation and periodic (sinusoidal) excitation signals are primarily used to probe the nonlinearity and couplings. According to the equations of ui and η→i, block diagram of the multivariable extremum seeking algorithm for solving Pareto optimality is given as Figure 2, where hi is a vector to pick Ji whose ith element is 1, and other elements are 0. f→i is a vector of [1,2(di+li0)-1B→-1R→ii], and then the estimator of (ui,η→i) can be derived as (u^i,η→^i). a, ωi, and ki are real numbers. According to the extremum seeking algorithm shown as Figure 2, one has
(16)u^˙i=akiJisinωit+aωicos⁡ωit,η→^˙i=akiρiJisinωit+aωiρicos⁡ωit,ρi=2di+li0-1B→-1R→ii.Figure 2
Block diagram of multivariable extremum seeking algorithm for solving Pareto optimal.Remark 7.
In our designed extremum seeking algorithm,Ji is not measured directly but can be derived by measuring other states according to (6), which depends on local available states. ωi affects the frequency of perturbation and avoiding of reaching a local optimality for nonconvex system. It is insignificant to set ωi uniformly as ω for simplify. ki is set similarly. By the multivariable extremum seeking algorithm, we aim at forcing the solutions (u^i,η→^i), ∀i∈n finally converge to the Pareto optimal u^i*, η→^i*, ∀i∈n, and then the distributed optimal problem is solved.Theorem 8.
Consider a distributed optimal problem aiming at tracking the desired profile which would be solved by a game, to minimize the performance indexJi, ∀i∈n as defined in (6), Lemma 3. Through a multivariable extremum seeking algorithm described in Figure 2 and (16), the Pareto optimality can be solved online at Ji*, ui* and ηi*. If ωi is large sufficiently, the Pareto optimal solution (u^i*,η→^i*), ∀i∈n, is globally uniformly asymptotically stable for system as described in (16).Proof.
Motivated by [21], a Lie brackets based analysis method is developed. Firstly, we consider the stability of ui. According to (16), the overall states can be written as
(17)u→=f→ν1+g→ν2,
where u→=u1u2⋯unT, f→=ak1J1ak2J2⋯aknJnT, and g→=aω1aω2⋯aωnT. ν1=sinωit and ν2=cos⁡ωit are fictitious control input. Then an input affine system is constructed.
The Lie bracket operation of vectorf→ and g→ is defined as follows:
(18)f→,g→=-a2∑i=1n‍kiωi∇uiJi,
where ∇uiJi is the gradient of Ji at the direction of ui. Then a Lie bracket system can be defined according to (18):
(19)u→˙′=0+ψ122πf→,g→,
where ψ12=∫-ππ‍∫0θ‍sinτcos⁡θdτdθ, calculating the integral, and ψ12=π. Then the Lie bracket system can be written as
(20)u→˙i′=-12a2∑i=1n‍kiωi∇uiJi.
Construct a global Lyapunov function asVL=∑i=1nJi-∑i=1nJi*. It is obvious that VL≥0 because Ji* is minimized. The time derivative of VL is as follows:
(21)V˙L=∑i=1n‍∇uiJiu˙i′.
Substitute (20) into (21),
(22)V˙L=-∑i=1n‍12a2∑i=1n‍kiωi∇uiJi2≤0
when VL=0 and V˙L=0, we can see that Ji will converge to Pareto optimal equilibrium Ji* for all i and the optimal control ui* will be achieved at the same time. So the system is globally uniformly asymptotically stable.


## 6. Results and Discussion

To validate the performance of proposed distributed cooperative optimal controller, a simulation scenario based on field application is set. A segment of track, desired speed profile Figure3, and simulation parameters of trains are given in Table 1, which comes from heavy haul trains running on Daqin railway in China. Without loss of generality, the number of locomotives of heavy haul trains is set to be four in the simulation, which is a 1+2+1 (namely, one locomotive + some wagons + two locomotives + some wagons + one locomotive) mode usually used in Daqin. Traction or braking of the train mainly act on locomotive and the dynamics of wagons are not considered for simplicity and treat them as rigid body in our simulation. But the method can be easily applied to more cars scenario.Table 1
The simulation parameters.
Parameter
Value
Unit
Massmi
292000
kg
Locomotive length
20
m
Coupler lengthLi,i-1
1
m
k
i
100
×
10
6
Nm−1
d
i
100
×
10
4
Nsm−1
Max traction
450
kN
Max brake
382
kN
Number of wagons
200
—
Number of locomotives
4
—
Start point
0.2
km
End point
12
kmDesired speed profile.
(a)
(b)The communication is limited due to the long distance among the cars, so the cars can only get some information of the whole train. By Theorem6, a strongly connected communication scenario is set as Figure 4, where the locomotives in the heavy haul trains are labeled 1, 2, 3, and 4. The car labeled 0 describes the order imposed on the heavy haul trains. The order may come from the operator or an automatic driving system. It is easy to validate that the graph is strongly connected according to the definition in Section 2.Figure 4
Communication topology of simulation system.The desired speed profilev0(t) is given in Figure 3, and the initial speeds of the locomotives are set to be
(23)v1(0)=59.40km/h,v2(0)=60.12km/h,v3(0)=59.44km/h,v4(0)=59.29km/h.When the distributed optimal controller is used, the velocities of the locomotives response as shown in Figure5. The lateral axis displays the train’s traveling distance during simulation. The vertical axis displays the speed response of locomotives under the controller designed in our paper. We can see that the locomotives in the heavy haul trains can track desired speed profile very well under the distributed controller with local performance index. That is because the controller aims to minimize a performance index Ji where η→i contains a term about tracking the desired speed. When the desired speed changes, there is a transient process before the locomotives reach the new desired speed. A partial enlarged figure in Figure 5 (from 7.5 km to 8.5 km) shows the detailed speed dynamics.Figure 5
Velocity response under distributed optimal controller.The cars’ velocity deviation to the desired profile under distributed optimal controller is shown in Figure6. The lateral axis displays the train’s traveling distance during simulation. The vertical axis displays the velocity deviation to the desired profile under the controller designed in our paper. We can see that the velocity deviation becomes very large when the desired speed changes but under the function of proposed controller, the deviation converges to nearly zero gradually. Because of the influence of gravitation on the ramp of railway, there is small speed fluctuation.Figure 6
Velocity deviation to the desired profile under distributed optimal controller.Figure7 shows the resulted control inputs for each locomotive under the designed controller. The lateral axis displays the train’s traveling distance during simulation. The vertical axis displays the control input including traction force and brake force under the controller designed in our paper. As shown in Table 1, the maximum of traction force is 450 kN and the maximum of brake force is −382 kN. It is obvious that all of the control inputs are within the normal range. The ultimate value of performance index J (sum of J1, J2, J3, and J4) of distributed optimal controller is a little bigger than 600 (Figure 8). From the definition of (6), the velocity deviation, position deviation, and control input are optimized.Figure 7
The control inputs under distributed optimal controller.Figure 8
Performance index of distributed optimal controller.


## 7. Conclusions

Optimization control is an important topic in train’s operation. Generally, traveling time, energy consumption, and in-train forces are considered in the performance index. Different from other types of trains, heavy haul trains are multipowered networked system with great length. In-train forces have to be considered in optimal control of heavy haul trains due to safety and cost of maintenance. Main focus has been on designing optimal controller by global information of the whole train and more recently on distributed control designed by partial information without considering the performance optimization. This paper studies the distributed optimal problem with individual performance index, which is designed by neighbors’ information. The distributed optimization problem is very difficult due to couplings and conflicts among the indexes. Nonetheless, in this paper, the distributed optimization problem is formulated as a cooperative game problem. We derived the condition for achieving global Pareto optimality equilibrium in this cooperative game. An online multivariable extremum seeking algorithm is used to approximate the partial differential equation during designing the game rules. Rigorous valuation of the proposed algorithm is done against application to Daqin railway. It is revealed that the designed controller can make the train track the desired speed profile with optimization of proposed performance index.



---

*Source: 158695-2015-03-02.xml*