# Calculus of Variations and Nonlinear Optimization Based Algorithm for Optimal Control of Hybrid Systems with Controlled Switching

Authors: Hajer Bouzaouache
Journal: Complexity
 (2017)

---

## Abstract

This paper investigates the optimal control problem of a particular class of hybrid dynamical systems with controlled switching. Given a prespecified sequence of active subsystems, the objective is to seek both the continuous control input and the discrete switching instants that minimize a performance index over a finite time horizon. Based on the use of the calculus of variations, necessary conditions for optimality are derived. An efficient algorithm, based on nonlinear optimization techniques and numerical methods, is proposed to solve the boundary-value ordinary differential equations. In the case of linear quadratic problems, the two-point boundary-value problems can be avoided which reduces the computational effort. Illustrative examples are provided and stress the relevance of the proposed nonlinear optimization algorithm.


---

## Body

## 1. Introduction

The optimization of hybrid dynamical systems has been widely investigated in the last years [1–7] because such systems can be used to model a wide range of real-world processes in many application fields, such as automotive systems, communication networks, chemical processes, robotics, air-traffic management systems, automated highway systems, embedded systems, and electrical circuit systems, etc [8–11].The focus of this paper is on the optimal control of a particular class of hybrid dynamical systems called switched systems. The behaviour of interest of such systems is described by a set of time-driven continuous-state subsystems and a switching law specifying the active subsystem at each time instant. A switching happens when an event signal is received. This signal may be an external generated signal or an internal signal generated if a condition on the time evolution, the states, and/or the inputs is satisfied. Consequently, we call a switching triggered by an external event an externally forced switching. So, according to the nature of the switching signal, switched systems may be classified into switched systems with externally forced switching or switched systems with internally forced switching.In the last decade the switched systems have been extensively studied [12–16] and the corresponding optimal control has never been as relevant as it is nowadays. Due to its significance in theory and applications many theoretical results and numerical algorithms have appeared in the literature [16–20]. Most of the available theoretical results are concerned with the study of necessary and/or sufficient conditions for a trajectory to be optimal by means of the Pontryagin maximum principle [21, 22], the dynamic programming approach [23], or the calculus of variations [24]. See [25] for a brief survey on recent progress in computational methods of the optimal control of switched systems.Modeling switched system depends on the different dynamics of its subsystems described by indexed differential or difference equations. Otherwise, if there is no external control influence on the system, we call it an autonomous switched system. To address the optimal control problem of autonomous switched systems, we have to focus on deriving the optimal sequence of switching times, and even if this sequence is the sole control influence, its determination remains a challenging task. In previous work [10], we have investigated the optimal control problem for autonomous switched systems with autonomous and/or controlled switches. The obtained results were considered to solve a time-optimal control problem for a nonlinear chemical process subject to state constraints.For nonautonomous switched systems, it is necessary to consider the continuous input together with switching times and sequences. Similar to the optimization for autonomous switched systems, optimization techniques are needed to find the optimal solutions for nonautonomous switched systems. But, despite the relevant contributions to find numerical solutions to such problems by employing the established theoretical conditions, effective algorithms still remain to be developed.Inspired by what is cited above, the main contribution of this paper is to solve the optimal control problem for nonautonomous switched systems with controlled switching. So, given a prespecified sequence of active subsystems, the objective is to seek both the continuous control input and the discrete switching instants that minimize a performance index over a finite time horizon. By using the calculus of variations, we derive necessary conditions for optimality. A computational algorithm, based on nonlinear optimization techniques and numerical methods for solving boundary-value ordinary differential equations, is then proposed. We also show that, in the case of linear quadratic problems, we can avoid dealing with two-point boundary-value problems and therefore reduce the computational effort.This paper is organized as follows. In Section2, the description of the studied switched systems is introduced and the corresponding optimal control problem is formulated. The main results of nonlinear optimization problem are presented in Section 3. An algorithm for computing the optimal continuous control input as well as the full information of the switching sequence and time instants is also proposed. In Section 4, we focus on quadratic optimal control problems for linear switched systems. Numerical simulations are performed in Section 5 to demonstrate the efficiency of the derived results and algorithm. Finally, some concluding remarks and suggested future work are given in Section 6.


## 2. Problem Statements and Preliminaries

In this paper, we focus on continuous-time switched systems whose dynamics are described fort∈t0,tf by(1)x˙t=fαtxt,ut,t,xt0=x0,where x∈Rn is the continuous-state vector, u∈Rm is the continuous control input vector, fi:Rn×Rm×R→Rni=1N is a set of continuously differentiable functions describing the dynamics of the N subsystems, and α:t0,tf→I=1,…,N is a piecewise constant function of time, named switching signal, specifying the active subsystem at each time instant.For such switched systems, we can control the state trajectory evolution by appropriately choosing the continuous control inputu and the switching signal α. Given a prespecified switching sequence that indicates the order of active subsystems, the control task is reduced to the computation of the continuous control and the discrete switching instants. A switching sequence in t0,tf is defined as(2)σ=t0,i0,t1,i1,…,tr,irwith(3)t0<t1<⋯<tr<tr+1=tf,ik∈I,k=0,…,r,where t1,…,tr are the switching instants and ik is the number of the active subsystems during the time interval tk,tk+1.The optimal control problem considered in the present paper can be formulated as follows.


### 2.1. Problem 1

Given a continuous-time switched system whose dynamics are governed by (1) and (2) for a fixed time interval t0,tf, the objective is to find the continuous control u∗ and the switching instants tk∗ that minimize the quadratic performance index(4)Ju,t1,…,tr=∑k=1r+1∫tk-1tkMkxt,ut,tdt+∑k=1rNkξk,tk+Kxf,tfwith(5)Mkxt,ut,t=12Cxt-xdtTQkCxt-xdt+12uTtRkut,(6)Nkξk,tk=12Cξk-xdtkTPkCξk-xdtk,(7)Kxf,tf=12Cxtf-xdtfTSCxtf-xdtf,(8)ξk=xtk;k=1,…,r.Qk,Pk,Rk, and S are symmetric matrices with Qk≥0,Pk≥0,S≥0, and Rk>0.xd is a desired trajectory over t0,tf and Nk are costs associated with the switches.In order to solve this problem, we will resort to the calculus of variations.


### 2.2. Basic Concepts

Based on the optimal control problem of continuous systems and the calculus of variations, the variational problem can be stated as follows. LetE be a family of trajectories qt defined on some interval t0,tf by(9)E=q:t0,tf⟶Rnof  classC1∣qt0=q0,qtf=qf.The problem is to find q∗(t)∈E that minimize a given cost functional defined as(10)Jq=∫t0tfLqt,q˙t,tdt+Kqf,tf,where L and K are real-valued continuously differentiable functions with respect to their arguments.In order to solve the above problem, we need to express the variation of the cost functionalJ, denoted by δJ, in terms of independent increments in all of its arguments. The optimal trajectory is then characterized by imposing the stationary condition δJ=0. Let q^(t)=q(t)+δq(t) be a neighboring perturbed trajectory of q(t), evolving in the time interval t0+δt0,tf+δtf, as illustrated in Figure 1. δt0,δq0,δtf, and δqf are small changes in the trajectory at the initial and final instants.Figure 1
Perturbations of trajectory.The variation ofJ is given by(11)δJ=Jq+δq-Jq.From the calculus of variations, we can obtain the expression of δJ as(12)δJ=∫t0tf∂L∂qt-ddt∂L∂q˙tTδqtdt+Ltf-∂L∂q˙tfTq˙tf+∂K∂tfqf,tfδtf+-Lt0+∂L∂q˙t0Tq˙t0δt0+∂K∂qfqf,tf+∂L∂q˙tfTδqf-∂L∂q˙t0Tδq0.If we introduce the Hamiltonian function H(t) and the conjugate moment p(t) as defined below(13)Ht≜-Lt+∂L∂q˙tTq˙t,pt≜∂L∂q˙tthe variation of J may be rewritten in the following form:(14)δJ=∫t0tf∂L∂qt-p˙tTδqtdt+-Htf+∂K∂tfqf,tfδtf+Ht0δt0+∂K∂qfqf,tf+ptfTδqf-pt0Tδq0.Setting to zero the coefficients of the independent increments δq,δt0,δq0,δtf, and δqf yields necessary conditions for a trajectory to be optimal. The obtained results will be used in the following section to solve the hybrid control problem.


## 2.1. Problem 1

Given a continuous-time switched system whose dynamics are governed by (1) and (2) for a fixed time interval t0,tf, the objective is to find the continuous control u∗ and the switching instants tk∗ that minimize the quadratic performance index(4)Ju,t1,…,tr=∑k=1r+1∫tk-1tkMkxt,ut,tdt+∑k=1rNkξk,tk+Kxf,tfwith(5)Mkxt,ut,t=12Cxt-xdtTQkCxt-xdt+12uTtRkut,(6)Nkξk,tk=12Cξk-xdtkTPkCξk-xdtk,(7)Kxf,tf=12Cxtf-xdtfTSCxtf-xdtf,(8)ξk=xtk;k=1,…,r.Qk,Pk,Rk, and S are symmetric matrices with Qk≥0,Pk≥0,S≥0, and Rk>0.xd is a desired trajectory over t0,tf and Nk are costs associated with the switches.In order to solve this problem, we will resort to the calculus of variations.


## 2.2. Basic Concepts

Based on the optimal control problem of continuous systems and the calculus of variations, the variational problem can be stated as follows. LetE be a family of trajectories qt defined on some interval t0,tf by(9)E=q:t0,tf⟶Rnof  classC1∣qt0=q0,qtf=qf.The problem is to find q∗(t)∈E that minimize a given cost functional defined as(10)Jq=∫t0tfLqt,q˙t,tdt+Kqf,tf,where L and K are real-valued continuously differentiable functions with respect to their arguments.In order to solve the above problem, we need to express the variation of the cost functionalJ, denoted by δJ, in terms of independent increments in all of its arguments. The optimal trajectory is then characterized by imposing the stationary condition δJ=0. Let q^(t)=q(t)+δq(t) be a neighboring perturbed trajectory of q(t), evolving in the time interval t0+δt0,tf+δtf, as illustrated in Figure 1. δt0,δq0,δtf, and δqf are small changes in the trajectory at the initial and final instants.Figure 1
Perturbations of trajectory.The variation ofJ is given by(11)δJ=Jq+δq-Jq.From the calculus of variations, we can obtain the expression of δJ as(12)δJ=∫t0tf∂L∂qt-ddt∂L∂q˙tTδqtdt+Ltf-∂L∂q˙tfTq˙tf+∂K∂tfqf,tfδtf+-Lt0+∂L∂q˙t0Tq˙t0δt0+∂K∂qfqf,tf+∂L∂q˙tfTδqf-∂L∂q˙t0Tδq0.If we introduce the Hamiltonian function H(t) and the conjugate moment p(t) as defined below(13)Ht≜-Lt+∂L∂q˙tTq˙t,pt≜∂L∂q˙tthe variation of J may be rewritten in the following form:(14)δJ=∫t0tf∂L∂qt-p˙tTδqtdt+-Htf+∂K∂tfqf,tfδtf+Ht0δt0+∂K∂qfqf,tf+ptfTδqf-pt0Tδq0.Setting to zero the coefficients of the independent increments δq,δt0,δq0,δtf, and δqf yields necessary conditions for a trajectory to be optimal. The obtained results will be used in the following section to solve the hybrid control problem.


## 3. Main Results

In this section, we will consider the case of a single switching, but the proposed approach and used methods can be straightforwardly applied to the case of several subsystems and more than one switching. The first problem is then reduced to the following problem.


### 3.1. Problem 2

A continuous-time switched system is given whose dynamics are governed by(15)x˙t=f1xt,ut,tfort∈t0,t1,x˙t=f2xt,ut,tfort∈t1,tf,xt0=x0,where x0,t0, and tf are fixed.Find the continuous controlu∗ and the switching instant t1∗ that minimize the quadratic performance index(16)Ju,t1=∫t0t1M1xt,ut,tdt+Nξ,t1+∫t1tfM2xt,ut,tdt+Kxf,tfwith(17)Mkxt,ut,t=12Cxt-xdtTQkCxt-xdt+12uTtRkut;k=1,2,Nξ,t1=12Cξ-xdt1TPCξ-xdt1,Kxf,tf=12Cxtf-xdtfTSCxtf-xdtf.Qk,P,Rk, and S are symmetric matrices with Qk≥0,P≥0,S≥0, and Rk>0.To solve this problem, we introduce a costate variableλ(t)∈Rn also called Lagrange multiplier to adjoin the system subject to constraints (15) to J (16). The augmented performance index is thus(18)J~u,t1=∫t0t1M1xt,ut,t+λTtx˙t-f1xt,ut,tdt+∫t1tfM2xt,ut,t+λTtx˙t-f2xt,ut,tdt+Nξ,t1+Kxf,tfwhich can be written as (19)J~u,t1=∫t0t1L1xt,ut,x˙t,tdt+Nξ,t1+∫t1tfL2xt,ut,x˙t,tdt+Kxf,tfwith(20)Lkxt,ut,x˙t,t=Mkxt,ut,t+λTtx˙t-fkxt,ut,tk=1,2.Note that the performance index J~ (19) is a sum of two cost functionals having the same form as (10) with q(t)=x(t)u(t)T. The variation δJ~ can therefore be obtained by using the results developed in Section 2.2. According to (13), the Hamiltonian function H(t) and the conjugate moment p(t) are expressed by(21)Hk=-Lk+x˙T∂Lk∂x˙+u˙T∂Lk∂u˙=-Mk+λTfk,k=1,2,(22)pt=∂Lk∂u˙=0,∂Lk∂x˙=λt.Using (14), we can write(23)δJ~=∫t0t1∂L1∂xt-λ˙tTδxt+∂L1∂utTδutdt+∫t1tf∂L2∂xt-λ˙tTδxt+∂L2∂utTδutdt+-H1t1-+H2t1++∂N∂t1ξ,t1δt1+λt1-+∂N∂ξξ,t1-λt1+Tδξ+∂K∂xfxf,tf+λtfTδxf.Since(24)∂Lk∂x=∂Mk∂x-∂fk∂xTλ=-∂Hk∂x,k=1,2,∂Lk∂u=∂Mk∂u-∂fk∂uTλ=-∂Hk∂u,k=1,2it follows that(25)δJ~=∫t0t1-∂H1∂xt-λ˙tTδxt+-∂H1∂utTδutdt+∫t1tf-∂H2∂xt-λ˙tTδxt+-∂H2∂utTδutdt+-H1t1-+H2t1++∂N∂t1ξ,t1δt1+λt1-+∂N∂ξξ,t1-λt1+Tδξ+∂K∂xfxf,tf+λtfTδxf.According to the Lagrange theory, a necessary condition for a solution to be optimal is δJ~=0. Setting to zero the coefficients of the independent increments δx(t),δu(t),δt1,δξ, and δxf yields the costate equation defined as(26)-∂H1∂xt-λ˙t=0fort∈t0,t1,-∂H2∂xt-λ˙t=0fort∈t1,tf,λt1--λt1++∂N∂ξξ,t1=0,∂K∂xfxf,tf+λtf=0,the gradient of the cost functional with respect tou(27)∇Ju=-∂H1∂ut=0fort∈t0,t1,∇Ju=-∂H2∂ut=0fort∈t1,tf,and the gradient of the cost functional with respect to the switching instant t1(28)∇Jt1=-H1t1-+H2t1++∂N∂t1ξ,t1=0.Taking into account (17) and (21), the costate equation (26) is rewritten(29)λ˙t=-∂f1∂xx,u,tTλt+CTQ1Cxt-xdtfort∈t0,t1,λ˙t=-∂f2∂xx,u,tTλt+CTQ2Cxt-xdtfort∈t1,tf,λt1-=λt1+-CTPCξ-xdt1,λtf=-CTSCxtf-xdtf.The gradient of the cost functional with respect tou (27) will be described by(30)∇Ju=R1ut-∂f1∂ux,u,tTλt=0fort∈t0,t1,∇Ju=R2ut-∂f2∂ux,u,tTλt=0fort∈t1,tfand the gradient of the cost functional with respect to the switching instant t1 (28) will be expressed as(31)∇Jt1=12Cξ-xdt1TQ1Cξ-xdt1+uTt1-R1ut1--Cξ-xdt1TQ2Cξ-xdt1-uTt1+R2ut1+-λTt1-f1t1-+λTt1+f2t1+=0.Considering linear controlled systems, we get from (30)(32)u∗t=R1-1∂f1∂ux,tTλtfort∈t0,t1,u∗t=R2-1∂f2∂ux,tTλtfort∈t1,tf.Substituting (32) into the costate equation (29) and the state equation (15) yields the Hamiltonian system expressed by(33)x˙λ˙=f1x,R1-1∂f1∂uTλ,t-∂f1∂xTλ+CTQ1Cx-xdfort∈t0,t1,xt0=x0;λt1-=λt1+-CTPCξ-xdt1,x˙λ˙=f2x,R2-1∂f2∂uTλ,t-∂f2∂xTλ+CTQ2Cx-xdfort∈t1,tf,xt1+=xt1-;λtf=-CTSCxtf-xdtf.To determine the hybrid optimal control (u∗,t1∗), we have to solve (33) and (31). Analytical resolution of the above equations is a difficult task, so we need to resort to the following:(i)
Numerical methods for solving boundary-value ordinary differential equations: the Hamiltonian system consists of two boundary-value ordinary differential equations whose solutions must satisfy conditions specified at the boundaries of the time intervalst0,t1 and t1,tf. To find these solutions, we can use, for example, the shooting method [26] consisting in replacing the boundary-value problem by an initial-value problem. More details about this method will be further provided.(ii)
Nonlinear optimization algorithms: to locate the optimal switching instantt1∗, we shall use nonlinear optimization techniques which are abundant in the literature [20, 27, 28]. These methods allow finding the instant t1 that satisfies the stationary condition (31).Therefore, the hybrid optimal control (u∗,t1∗) can be found by the implementation of the algorithm detailed in the following subsection.


### 3.2. Algorithm

See Algorithm1.Algorithm 1
Step 1. Chose an initial Tk=t1k for k=0;
Step 2. Compute the corresponding continuous control
(i) Solve the Hamiltonian system using the shooting method
(a) Guess the unspecified initial conditionsλ0(t0),x0(t1+)
andλ0(t1+);
(b) Integrate the Hamiltonian system (33) forward from t0
totf;
(c) Using the resulting values ofx(t1-),λ(t1-) and λ(tf), evaluate
the error function:
E=x0t1+-xt1-;λt1--λ0t1++CTPCξ-xdt1;
-CTSCxtf-xdtf-λtf;
(d) Adjust the value ofλ0(t0),x0(t1+) and λ0(t1+), using a
numerical method for solving nonlinear equations, to bring
the functionE closer to zero;
(ii) Calculate the continuous optimal control (32);
Step 3. Calculate J (16);
Step 4. Calculate GRADk=∇Jt1k (31);
Step 5. Find the value of Tk+1 by using a nonlinear optimization
technique;k=k+1;
Step 6. Repeat Steps 2, 3, 4 and 5 until the criterion
GRADk2<ε is satisfied.Remark 1.
Note that, at each iterationk, we have to solve a boundary-value problem to find the continuous control for a fixed switching instant. Numerical methods used for solving such problem are generally iterative, which may lead to heavy computational time.Remark 2.
For the case of linear switched systems with quadratic performance index, the present work will show that dealing with two-point boundary-value problems can be avoided, and therefore the computational effort can be reduced.


## 3.1. Problem 2

A continuous-time switched system is given whose dynamics are governed by(15)x˙t=f1xt,ut,tfort∈t0,t1,x˙t=f2xt,ut,tfort∈t1,tf,xt0=x0,where x0,t0, and tf are fixed.Find the continuous controlu∗ and the switching instant t1∗ that minimize the quadratic performance index(16)Ju,t1=∫t0t1M1xt,ut,tdt+Nξ,t1+∫t1tfM2xt,ut,tdt+Kxf,tfwith(17)Mkxt,ut,t=12Cxt-xdtTQkCxt-xdt+12uTtRkut;k=1,2,Nξ,t1=12Cξ-xdt1TPCξ-xdt1,Kxf,tf=12Cxtf-xdtfTSCxtf-xdtf.Qk,P,Rk, and S are symmetric matrices with Qk≥0,P≥0,S≥0, and Rk>0.To solve this problem, we introduce a costate variableλ(t)∈Rn also called Lagrange multiplier to adjoin the system subject to constraints (15) to J (16). The augmented performance index is thus(18)J~u,t1=∫t0t1M1xt,ut,t+λTtx˙t-f1xt,ut,tdt+∫t1tfM2xt,ut,t+λTtx˙t-f2xt,ut,tdt+Nξ,t1+Kxf,tfwhich can be written as (19)J~u,t1=∫t0t1L1xt,ut,x˙t,tdt+Nξ,t1+∫t1tfL2xt,ut,x˙t,tdt+Kxf,tfwith(20)Lkxt,ut,x˙t,t=Mkxt,ut,t+λTtx˙t-fkxt,ut,tk=1,2.Note that the performance index J~ (19) is a sum of two cost functionals having the same form as (10) with q(t)=x(t)u(t)T. The variation δJ~ can therefore be obtained by using the results developed in Section 2.2. According to (13), the Hamiltonian function H(t) and the conjugate moment p(t) are expressed by(21)Hk=-Lk+x˙T∂Lk∂x˙+u˙T∂Lk∂u˙=-Mk+λTfk,k=1,2,(22)pt=∂Lk∂u˙=0,∂Lk∂x˙=λt.Using (14), we can write(23)δJ~=∫t0t1∂L1∂xt-λ˙tTδxt+∂L1∂utTδutdt+∫t1tf∂L2∂xt-λ˙tTδxt+∂L2∂utTδutdt+-H1t1-+H2t1++∂N∂t1ξ,t1δt1+λt1-+∂N∂ξξ,t1-λt1+Tδξ+∂K∂xfxf,tf+λtfTδxf.Since(24)∂Lk∂x=∂Mk∂x-∂fk∂xTλ=-∂Hk∂x,k=1,2,∂Lk∂u=∂Mk∂u-∂fk∂uTλ=-∂Hk∂u,k=1,2it follows that(25)δJ~=∫t0t1-∂H1∂xt-λ˙tTδxt+-∂H1∂utTδutdt+∫t1tf-∂H2∂xt-λ˙tTδxt+-∂H2∂utTδutdt+-H1t1-+H2t1++∂N∂t1ξ,t1δt1+λt1-+∂N∂ξξ,t1-λt1+Tδξ+∂K∂xfxf,tf+λtfTδxf.According to the Lagrange theory, a necessary condition for a solution to be optimal is δJ~=0. Setting to zero the coefficients of the independent increments δx(t),δu(t),δt1,δξ, and δxf yields the costate equation defined as(26)-∂H1∂xt-λ˙t=0fort∈t0,t1,-∂H2∂xt-λ˙t=0fort∈t1,tf,λt1--λt1++∂N∂ξξ,t1=0,∂K∂xfxf,tf+λtf=0,the gradient of the cost functional with respect tou(27)∇Ju=-∂H1∂ut=0fort∈t0,t1,∇Ju=-∂H2∂ut=0fort∈t1,tf,and the gradient of the cost functional with respect to the switching instant t1(28)∇Jt1=-H1t1-+H2t1++∂N∂t1ξ,t1=0.Taking into account (17) and (21), the costate equation (26) is rewritten(29)λ˙t=-∂f1∂xx,u,tTλt+CTQ1Cxt-xdtfort∈t0,t1,λ˙t=-∂f2∂xx,u,tTλt+CTQ2Cxt-xdtfort∈t1,tf,λt1-=λt1+-CTPCξ-xdt1,λtf=-CTSCxtf-xdtf.The gradient of the cost functional with respect tou (27) will be described by(30)∇Ju=R1ut-∂f1∂ux,u,tTλt=0fort∈t0,t1,∇Ju=R2ut-∂f2∂ux,u,tTλt=0fort∈t1,tfand the gradient of the cost functional with respect to the switching instant t1 (28) will be expressed as(31)∇Jt1=12Cξ-xdt1TQ1Cξ-xdt1+uTt1-R1ut1--Cξ-xdt1TQ2Cξ-xdt1-uTt1+R2ut1+-λTt1-f1t1-+λTt1+f2t1+=0.Considering linear controlled systems, we get from (30)(32)u∗t=R1-1∂f1∂ux,tTλtfort∈t0,t1,u∗t=R2-1∂f2∂ux,tTλtfort∈t1,tf.Substituting (32) into the costate equation (29) and the state equation (15) yields the Hamiltonian system expressed by(33)x˙λ˙=f1x,R1-1∂f1∂uTλ,t-∂f1∂xTλ+CTQ1Cx-xdfort∈t0,t1,xt0=x0;λt1-=λt1+-CTPCξ-xdt1,x˙λ˙=f2x,R2-1∂f2∂uTλ,t-∂f2∂xTλ+CTQ2Cx-xdfort∈t1,tf,xt1+=xt1-;λtf=-CTSCxtf-xdtf.To determine the hybrid optimal control (u∗,t1∗), we have to solve (33) and (31). Analytical resolution of the above equations is a difficult task, so we need to resort to the following:(i)
Numerical methods for solving boundary-value ordinary differential equations: the Hamiltonian system consists of two boundary-value ordinary differential equations whose solutions must satisfy conditions specified at the boundaries of the time intervalst0,t1 and t1,tf. To find these solutions, we can use, for example, the shooting method [26] consisting in replacing the boundary-value problem by an initial-value problem. More details about this method will be further provided.(ii)
Nonlinear optimization algorithms: to locate the optimal switching instantt1∗, we shall use nonlinear optimization techniques which are abundant in the literature [20, 27, 28]. These methods allow finding the instant t1 that satisfies the stationary condition (31).Therefore, the hybrid optimal control (u∗,t1∗) can be found by the implementation of the algorithm detailed in the following subsection.


## 3.2. Algorithm

See Algorithm1.Algorithm 1
Step 1. Chose an initial Tk=t1k for k=0;
Step 2. Compute the corresponding continuous control
(i) Solve the Hamiltonian system using the shooting method
(a) Guess the unspecified initial conditionsλ0(t0),x0(t1+)
andλ0(t1+);
(b) Integrate the Hamiltonian system (33) forward from t0
totf;
(c) Using the resulting values ofx(t1-),λ(t1-) and λ(tf), evaluate
the error function:
E=x0t1+-xt1-;λt1--λ0t1++CTPCξ-xdt1;
-CTSCxtf-xdtf-λtf;
(d) Adjust the value ofλ0(t0),x0(t1+) and λ0(t1+), using a
numerical method for solving nonlinear equations, to bring
the functionE closer to zero;
(ii) Calculate the continuous optimal control (32);
Step 3. Calculate J (16);
Step 4. Calculate GRADk=∇Jt1k (31);
Step 5. Find the value of Tk+1 by using a nonlinear optimization
technique;k=k+1;
Step 6. Repeat Steps 2, 3, 4 and 5 until the criterion
GRADk2<ε is satisfied.Remark 1.
Note that, at each iterationk, we have to solve a boundary-value problem to find the continuous control for a fixed switching instant. Numerical methods used for solving such problem are generally iterative, which may lead to heavy computational time.Remark 2.
For the case of linear switched systems with quadratic performance index, the present work will show that dealing with two-point boundary-value problems can be avoided, and therefore the computational effort can be reduced.


## 4. Quadratic Optimization

In this section, we consider the problem of minimizing a quadratic criterion subject to switched linear subsystems. For this special class, we can obtain a closed-loop continuous control within each time intervaltk,tk+1. As for the previous section, we consider the case of a single switching. The proposed approach can be straightforwardly applied to the case of several subsystems and more than one switching. So, we will try to find a more attractive solution to Problem 2 with(34)fixt,ut,t=Aixt+Biuti=1,2,where Ai∈Rn×n and Bi∈Rn×m.According to (32), the continuous control is given as(35)u∗t=R1-1B1Tλtfort∈t0,t1,u∗t=R2-1B2Tλtfort∈t1,tf.Using (33) and (31), the Hamiltonian system can be written as(36)x˙λ˙=A1x+B1R1-1B1TλCTQ1Cx-xd-A1Tλfort∈t0,t1,(37)xt0=x0,λt1-=λt1+-CTPCξ-xdt1,(38)x˙λ˙=A2x+B2R2-1B2TλCTQ2Cx-xd-A2Tλfort∈t1,tf,(39)xt1+=xt1-,(40)λtf=-CTSCxtf-xdtf.The gradient of the cost functional with respect to the switching instant t1 is then defined as(41)∇Jt1=12Cξ-xdt1TQ1Cξ-xdt1+uTt1-R1ut1--Cξ-xdt1TQ2Cξ-xdt1-uTt1+R2ut1+-λTt1-A1ξ+B1ut1-+λTt1+A2ξ+B2ut1+=0.In order to solve the Hamiltonian system, we use the sweep method [19]. Thus, assume that λ(t) and x(t) satisfy a linear relation like (40) for all t∈t1,tf(42)λt=-Ktxt+Vt∀t∈t1,tf.If we can find the matrices K(t) and V(t), then our assumption is valid.By differentiating (42) with respect to time, we get(43)λ˙t=-K˙txt-Ktx˙t+V˙t.According to (38), one obtains(44)λ˙=-K˙x-KA2x+B2R2-1B2Tλ+V˙=-A2Tλ+CTQ2Cx-xd.Taking into account (42), it follows that(45)-K˙-KA2+KB2R2-1B2TK-A2TK-CTQ2Cx-KB2R2-1B2TV+V˙+A2TV+CTQ2xd=0.This condition holds for all state trajectories x(t), which implies (46)-K˙-KA2+KB2R2-1B2TK-A2TK-CTQ2C=0fort∈t1,tf,(47)V˙+A2T-KB2R2-1B2TV+CTQ2xd=0fort∈t1,tf.The matrices K(t) and V(t) are determined by solving (46) and (47) with final conditions deduced from relation (40) and they are written as(48)Ktf=CTSC,Vtf=CTSxdtf.Our assumption, (42), was then valid.Since(49)λt1+=-Kt1+ξ+Vt1+we can rewrite (37) as(50)λt1-=-Kt1+-CTPCξ+Vt1++CTPxdt1.Note that λ(t1-) is linear in ξ. Hence, to solve the Hamiltonian system in the time interval t0,t1, we make the same assumption as (42)(51)λt=-Ktxt+Vt∀t∈t0,t1and then, by using (36), we establish the needed equations for the determination of the matrices K(t) and V(t) in the time interval t0,t1(52)-K˙-KA1+KB1R1-1B1TK-A1TK-CTQ1C=0fort∈t0,t1,(53)V˙+A1T-KB1R1-1B1TV+CTQ1xd=0fort∈t0,t1with final conditions deduced from (50) (54)Kt1-=Kt1++CTPC,Vt1-=Vt1++CTPxdt1.Substituting (42) and (51) into (35), we get(55)u∗t=-R1-1B1TKtxt+R1-1B1TVtfort∈t0,t1,u∗t=-R2-1B2TKtxt+R2-1B2TVtfort∈t1,tf.Note that u∗ is an affine state feedback. The closed-loop system is therefore governed by(56)x˙t=A1-B1R1-1B1TKtxt+B1R1-1B1TVtfort∈t0,t1,x˙t=A2-B2R2-1B2TKtxt+B2R2-1B2TVtfort∈t1,tf,xt0=x0,xt1+=xt1-.In order to compute the optimal continuous control for a fixed switching instant, we need to solve the matrix Riccati equations (46)–(52) and the auxiliary equations (47)–(53). The latter are integrated backward in time to get the matrices K(t) and V(t). The hybrid optimal control is then determined by the implementation of Algorithm 1 with Step 2 modified in Algorithm 2.Algorithm 2:Modified Step 2 of Algorithm 1 for solving switched linear quadratic optimal control problems.
Step 2. Compute the corresponding continuous control
(a) Solve the matrix Riccati equations (46)–(52)
backward fromtf to t0;
(b) Solve the auxiliary equations (47)–(53) backward
fromtf to t0;
(c) Solve the state equation (56) forward;
(d) Calculateλ(t1+) (42) and λ(t1-) (51);
(e) Calculate the continuous optimal control (55);


## 5. Simulation Results

To illustrate the validity of the proposed result and the efficiency of the algorithms, two examples are considered in this section. The former concerns the optimization of a nonlinear switched system. The latter deals with quadratic optimal control problem for linear switched system. The computation was performed using MATLAB 6.5 on a Celeron 2 GHz PC with 256 Mo of RAM.


### 5.1. First Illustrative Numerical Example

Consider a nonlinear switched system described bySubsystem 1(57)f1x,u,t=x1+usin⁡x1-x2-ucos⁡x2active  fort∈t0,t1,Subsystem 2(58)f2x,u,t=x2+usin⁡x2-x1-ucos⁡x1active  fort∈t1,t2,Subsystem 3(59)f3x,u,t=-x1-usin⁡x1x2+ucos⁡x2active  fort∈t2,tfwith t0=0,tf=3 and x0=23T.The objective is to find the continuous controlu∗ and the switching times t1∗ and t2∗ that minimize the following performance index:(60)J=12x1tf-12+12x2tf+12+12∫t0tfx1t-12+x2t+12+u2tdt.By the implementation of Algorithm 1 with T0=t10t20T=12T and using the steepest descent method to locate the switching instants, after 6 iterations taking about 93.78 seconds of CPU time, we find t1∗=0.2303,t2∗=1.0255, and J∗=5.5247.The optimal control and the corresponding state trajectory are shown in Figure2. Figure 3 shows the plot of the cost functional J for different 0<t1<t2<3.Figure 2
Continuous control input and optimal state trajectory.Figure 3
CostJ for different t1,t2.


### 5.2. Second Illustrative Example

Let us consider a linear switched system, described by three subsystems asSubsystem 1(61)A1=-200-1B1=10active  for0,Subsystem 2(62)A2=0.55.3-5.30.5B2=1-1active  fort∈t1,t2,Subsystem 3(63)A3=1001.5B3=01active  fort∈t2,tfwith t0=0,tf=3 and x(0)=44T.The problem is to find the continuous controlu∗ and the switching instants t1∗ and t2∗ that minimize the quadratic performance index (64)J=12x1tf+4.14372+12x2tf-9.35692+12∫t0tfu2tdt.By the implementation of the modified Algorithm 1 (see Algorithm 2) with T0=t10t20T=0.81.8T and using the Broyden-Fletcher-Goldfarb-Shanno method to locate the switching instants, after 5 iterations taking about 47.51 seconds of CPU time, we find t1∗=1.0002,t2∗=2.0002, and J∗=5.8776e-7. The optimal control and the corresponding state trajectory are shown in Figure 4. Figure 5 shows the plot of the cost functional J for different 0<t1<t2<3.Figure 4
Control input and optimal state trajectory.Figure 5
CostJ for different t1,t2.By examining Figures3 and 5, we can notice that the function J is not convex. Since the nonlinear optimization techniques lead generally to local minimums, the initial point of Algorithm 1 must be adequately chosen to reach the global minimum. The solutions presented for the both examples were evidently the optimal ones.


## 5.1. First Illustrative Numerical Example

Consider a nonlinear switched system described bySubsystem 1(57)f1x,u,t=x1+usin⁡x1-x2-ucos⁡x2active  fort∈t0,t1,Subsystem 2(58)f2x,u,t=x2+usin⁡x2-x1-ucos⁡x1active  fort∈t1,t2,Subsystem 3(59)f3x,u,t=-x1-usin⁡x1x2+ucos⁡x2active  fort∈t2,tfwith t0=0,tf=3 and x0=23T.The objective is to find the continuous controlu∗ and the switching times t1∗ and t2∗ that minimize the following performance index:(60)J=12x1tf-12+12x2tf+12+12∫t0tfx1t-12+x2t+12+u2tdt.By the implementation of Algorithm 1 with T0=t10t20T=12T and using the steepest descent method to locate the switching instants, after 6 iterations taking about 93.78 seconds of CPU time, we find t1∗=0.2303,t2∗=1.0255, and J∗=5.5247.The optimal control and the corresponding state trajectory are shown in Figure2. Figure 3 shows the plot of the cost functional J for different 0<t1<t2<3.Figure 2
Continuous control input and optimal state trajectory.Figure 3
CostJ for different t1,t2.


## 5.2. Second Illustrative Example

Let us consider a linear switched system, described by three subsystems asSubsystem 1(61)A1=-200-1B1=10active  for0,Subsystem 2(62)A2=0.55.3-5.30.5B2=1-1active  fort∈t1,t2,Subsystem 3(63)A3=1001.5B3=01active  fort∈t2,tfwith t0=0,tf=3 and x(0)=44T.The problem is to find the continuous controlu∗ and the switching instants t1∗ and t2∗ that minimize the quadratic performance index (64)J=12x1tf+4.14372+12x2tf-9.35692+12∫t0tfu2tdt.By the implementation of the modified Algorithm 1 (see Algorithm 2) with T0=t10t20T=0.81.8T and using the Broyden-Fletcher-Goldfarb-Shanno method to locate the switching instants, after 5 iterations taking about 47.51 seconds of CPU time, we find t1∗=1.0002,t2∗=2.0002, and J∗=5.8776e-7. The optimal control and the corresponding state trajectory are shown in Figure 4. Figure 5 shows the plot of the cost functional J for different 0<t1<t2<3.Figure 4
Control input and optimal state trajectory.Figure 5
CostJ for different t1,t2.By examining Figures3 and 5, we can notice that the function J is not convex. Since the nonlinear optimization techniques lead generally to local minimums, the initial point of Algorithm 1 must be adequately chosen to reach the global minimum. The solutions presented for the both examples were evidently the optimal ones.


## 6. Conclusion

Based on nonlinear optimization techniques and numerical methods for solving boundary-value ordinary differential equations, we proposed an algorithm for solving optimal control problems for switched systems with externally forced switching. We assumed that the switching sequence is fixed, and therefore the control variables are only the continuous control input and the discrete switching instants. The effectiveness of the presented algorithm was demonstrated through simulation results.The obtained results will be extended to the optimal control of interconnected switched systems. Otherwise parametric uncertainties and input disturbances are often present in real-life applications. So, analysis procedures and control synthesis algorithm for hybrid systems if additive disturbances and/or parametric uncertainties are present are topics that are starting to deserve the attention of researchers [5]. Indeed, uncertainty in hybrid system can be present in the vector fields describing the flow of the system and/or in the switching transition law. It can be of parametric nature or caused by time-varying perturbations of the vector field, switching delays. Thus, the robustness analysis will be investigated and can be handled in our future works.



---

*Source: 5308013-2017-08-10.xml*