# Optimal Control for a Family of Systems in Novel State Derivative Space Form with Experiment in a Double Inverted Pendulum System

Authors: Yuan-Wei Tseng; Jer-Guang Hsieh
Journal: Abstract and Applied Analysis
 (2013)

---

## Abstract

Optimal control for a family of systems in novel state derivative space form, abbreviated as SDS systems in this study, is proposed. The first step in deriving optimal control laws for SDS systems is to form an augmented cost functional. It turns out that novel differential Lagrange multipliers must be used to adjoin SDS system constraints (namely, the dynamical equations of the control system) to the integrand of the original cost functional which is a function of state derivatives. This not only eases our derivation but also makes our derivation parallel to that for systems in standard state space form. We will show via a real electric circuit that optimal control for a class of descriptor systems with impulse modes can easily be carried out using our design method. It will be shown that linear quadratic regulator (LQR) design for linear time-invariant SDS systems using state derivative feedback can be obtained via an algebraic Riccati equation. Furthermore, this optimal state derivative feedback may also be implemented using an equivalent state feedback. This is useful in real situations when only states but not the state derivatives are available for measurement. The LQR design for a double inverted pendulum system is implemented to illustrate the use of our method.


---

## Body

## 1. Introduction

Optimal control for systems in standard state space form has long been developed [1–4]. The optimization problem can be stated as follows:(1a)
minimize
J
0
(
u
)
=
ϕ
(
x
(
t
f
)
,
t
f
)
+
∫
t
0
t
f
F
(
x
,
u
,
t
)
d
t
(1b)
subject
to
x
˙
=
f
(
x
,
u
,
t
)
.
m
m
m
m
m
m
m
m
m
m
m
m
mNote that the integrand in the cost functionalJ0(u) is a function of states x and controls u. However, in some applications, the integrand of an appropriate cost functional for the problem at hand is not only a function of states and controls but also a function of the state derivatives x˙. For instance, to improve people’s comfort and safety when ridding a vehicle or an airplane, acceleration during the course of riding must be taken into account. It is then desirable to minimize the integral of a function of the acceleration during the course. Since acceleration is the derivative of the velocity, which is usually a state variable, we may also consider the cost functional of the form
(2)J0(u)=ϕ(x(tf),tf)+∫t0tfF(x˙,u,t)dt.Note also that not every control system can be modeled in standard state space form or can easily be handled if it is expressed in standard state space form. For instance, consider the following system:(3)Ex˙=Fx+Nu.
If the coefficient matrix in state derivatives, E, is singular, then the system cannot be expressed in standard state space form. Such systems are called generalized state space systems [5], descriptor systems [6], singular systems [7], or semistate systems [8]. If E is nearly singular, then it is not easy to handle such a system using the standard control design.Extensive applications of descriptor systems arise in many areas of engineering such as electrical networks [8], aerospace systems [9], smart structures [10], and chemical processes [11]. Descriptor systems also appear in other areas such as the dynamic Leontief model for economic production sectors [12] and biological complex systems [13]. A comprehensive review is available in [14]. In previous studies, descriptor systems are further categorized as impulse-free systems and systems with impulse modes [14]. If the descriptor system has impulsive modes, the integration part of the cost functional may become infinite. In this situation, further investigation of impulse controllability and the impulse mode elimination has to be analyzed in control designs [15]. Therefore, descriptor systems with impulse modes are considered to be difficult in control designs. On the other hand, control designs for impulse-free descriptor systems can be carried out in easier ways. Variational calculus for impulse-free descriptor systems was derived by Jonckheere [16]. Since then, many optimal control algorithms were proposed for impulse-free descriptor systems [14]. Roughly speaking, the available control design algorithms for descriptor systems [15, 17–20] are much more complex than those for the systems in standard state space form. Consequently, there are barricades for engineers without strong mathematical background to apply those sophisticated control algorithms.If the matrixF in (3) is nonsingular, it can be expressed in reciprocal state space (RSS) form [10, 21] as follows:
(4)x=Ax˙+Bu.
Since the system eigenvalues are the reciprocals of the eigenvalues of matrix A in (4), the name of reciprocal state space form was given by the first author of this paper. In this form, one can easily carry out control designs such as pole placement [22], eigenstructure assignment [22], and linear quadratic regulator (LQR) designs [22] using state derivative feedback. Therefore, as long as a descriptor system with impulsive mode can be expressed in RSS form, control design can be easily performed by applying state derivative feedback.Motivated by the analysis above, instead of the systems in standard state space form, a family of control systems in novel state derivative space form, abbreviated as SDS systems, is investigated in this study. The dynamic equation of an SDS system can be described as(5)x=f(x˙,u,t).


## 2. Optimality Conditions

The optimal control problem in this study can be stated as follows:(6a)
minimize
J
0
(
u
)
=
ϕ
(
x
(
t
f
)
,
t
f
)
+
∫
t
0
t
f
F
(
x
˙
,
u
,
t
)
d
t
(6b)
subject
to
x
=
f
(
x
˙
,
u
,
t
)
.
m
m
m
m
m
m
m
m
m
m
m
m
mFurther, we assume that the terminal timetf is free and the final states are free but must be constrained to the surface defined by Ψ[x(tf)]=0. Notice that the integrand of the cost functional may also include the state x; namely, F=F(x,x˙,u,t). Simply using the system equation (6b), this function F can be reduced to a function that depends only on x˙, u, and t.To handle this optimization problem, we first form an augmented cost functional as follows:(7)J=ϕ(x(tf),tf)+νTΨ[x(tf)]+∫t0tf[F(x˙,u,t)+λ˙T(t)(f(x˙,u,t)-x)]dt.Here,ν is a vector of usual Lagrange multipliers for the final state constraints. Note that novel differential Lagrange multipliers λ˙, instead of the usual λ, have been used to adjoin SDS system constraints (6b) to the integrand of the original cost functional which is a function of state derivatives. This not only eases our derivation but also makes our derivation parallel to that for systems in standard state space form.The Hamiltonian is defined by(8)H(x˙,u,λ˙,t)=F(x˙,u,t)+λ˙T(t)f(x˙,u,t).
Substituting (8) to (7), we have
(9)J=ϕ(x(tf),tf)+νTΨ[x(tf)]+∫t0tf(H-λ˙Tx)dt.
Now consider the following small perturbations:
(10)x⟶x+δx,u⟶u+δu,tf=tf+dtf.
To minimize (9), the variation of (9) must be zero. That is,
(11)0=ΔJ=[∂ϕ∂x+νT∂Ψ∂x]t=tfdx(tf)+∂ϕ∂tfdtf+∫t0tfδ(H-λ˙Tx)dt+∂∂tf[∫t0tf(H-λ˙Tx)dt]dtf.
The individual terms of variation in (11) can be obtained as follows:
(12)δH=∂H∂x˙δx˙+∂H∂uδu+δλ˙T∂H∂λ˙T=∂H∂x˙δx˙+∂H∂uδu+δλ˙Tf,δ(-λ˙Tx)=-δλ˙T∂(λ˙Tx)∂λ˙T-∂(λ˙Tx)∂xδx=-δλ˙Tx-λ˙Tδx.Because of the system constraint (6b), we have -δλ˙Tx+δλ˙Tf=0. Adding (12) yields
(13)δ(H-λ˙Tx)=∂H∂x˙δx˙+∂H∂uδu-λ˙Tδx,(14)∴∫t0tfδ(H-λ˙Tx)dt=∫t0tf(∂H∂x˙δx˙+∂H∂uδu-λ˙Tδx)dt.
Integrating by parts for the term -λ˙Tδx, we obtain
(15)-∫t0tfλ˙Tδxdt=-∫t0tfλ˙Tdtδx=-∫t0tf(dλdt)Tdtδx=-∫t0tfdλTδx=-λTδx|t0tf+∫t0tfλTd(δx)=-λTδx|t0tf+∫t0tfλTd(δx)dtdt=-λT(tf)δx(tf)+λT(t0)δx(t0)+∫t0tfλTδx˙dt.
Substituting (15) back to (14), we have
(16)∫t0tfδ(H-λ˙Tx)dt=∫t0tf[(∂H∂x˙+λT)δx˙+∂H∂uδu]dt-λT(tf)δx(tf)+λT(t0)δx(t0).
It can be shown [23] that
(17)dx(tf)=δx(tf)+x˙(tf)dtf.
Therefore, in (16), we have
(18)-λT(tf)δx(tf)=-λT(tf)dx(tf)+λT(tf)x˙(tf)dtf.
In (11), we have
(19)∂ϕ∂tfdtf+∂∂tf[∫t0tf(H-λ˙Tx)dt]dtf=(H+∂ϕ∂tf-λ˙Tx)dtf.
Applying (14)–(19) to (11), one obtains
(20)ΔJ=[∂ϕ∂x+νT∂Ψ∂x-λT]t=tfdx(tf)+∫t0tf[(∂H∂x˙+λT)δx˙+∂H∂uδu]dt+λT(t0)δx(t0)+(H+∂ϕ∂tf-λ˙Tx+λTx˙)dtf=0.
Since the final state x(tf) is free, in addition to SDS system constraint in (6b), from (20), necessary conditions of the optimization problem in (6a) and (6b) can be stated as follows:
(21)∂H∂x˙=-λT,(22)∂H∂u=0,(23)λT(t0)δx(t0)=0(initialconditions),(24)λT(tf)=[∂ϕ∂x+νT∂Ψ∂x]t=tf(terminalconditions).
From (6b) and (8), the “transversality condition” [23] for tf can be determined via the following equation:
(25)[H+∂ϕ∂tf-λ˙Tx+λTx˙]dtf=[F+λ˙Tf+∂ϕ∂tf-λ˙Tx+λTx˙]dtf=[F+λTx˙+∂ϕ∂tf]dtf=0.In the following simple example, we wish to illustrate the point that when the system under consideration can be expressed in both standard state space (SSS) form and state derivative space (SDS) form, optimal designs are equivalent.Example 1.
Suppose that we wish to find a controlu to minimize the following cost functional:
(26)J=tf+∫0tf(u2+x˙)dt
subject to
(27)x=-12x˙+u=f(x˙,u)(inSDSform),(28)x˙=-2x+2u=f~(x,u)(inSSSform),x(0)=3, and tf is free.
Let us first solve this problem using the SDS approach. The Hamiltonian is given by(29)H(x˙,u,λ˙,t)=(u2+x˙)+λ˙(-12x˙+u).
Necessary conditions (21) and (22) give
(30)∂H∂x˙=1-12λ˙=-λ,∂H∂u=2u+λ˙=0;
that is,
(31)λ˙-2λ-2=0,u=-λ˙2.
Since the initial condition x(0)=3 is given, we have δx(0)=0. Hence, (23) is satisfied. Since Ψ=0 and the ϕ is independent of x, the terminal condition (24) gives
(32)λ(tf)=[∂ϕ∂x]t=tf=0.
Since tf is free, the transversality condition (25) now becomes
(33)[F+λx˙+∂ϕ∂tf]=u2+x˙+λx˙+1=u2+(λ+1)x˙+1=0.
In summary, (27) and (31)–(33) constitute the optimality conditions for the system (27) in SDS form.
For comparison, we then solve this problem again for the system (28) in standard state space form using the standard optimization techniques. The Hamiltonian is given by
(34)H~(x˙,u,λ,t)=(u2+x˙)+λ~(-2x+2u)=(u2-2x+2u)+λ~(-2x+2u).
Necessary conditions for the optimality [23] are given by
(35)∂H~∂x=-2-2λ~=-λ~˙,∂H~∂u=2u+2+2λ~=2u+λ~˙=0;
that is,
(36)λ~˙-2λ~-2=0,u=-λ~˙2.
The terminal condition is given by
(37)λ~(tf)=[∂ϕ∂x]t=tf=0.
The transversality condition is given by
(38)[H~+∂ϕ∂tf]=(u2-2x+2u)+λ~(-2x+2u)+1=(u2+x˙)+λ(-2x+2u)+1=u2+x˙+λx˙+1=u2+(λ+1)x˙+1=0.
It is not surprising to find that necessary conditions (31)–(33) for the system (27) in SDS form and those necessary conditions (36)–(38) for the system (28) in standard state space form are identical. This means that optimal control design for systems that can be expressed in both SDS and standard state space forms is the same.
Solving (27) and (31)–(33), we have
(39)x(t)=3.081·e-2t-0.081·e2t,λ(t)=0.162·e2t-1,u(t)=-0.162·e2t,tf=0.909.


## 3. Descriptor Circuit System with Impulse Modes

The following circuit shown in Figure1 is an example of a descriptor system with impulse mode [8]. It is generally considered to be difficult to carry out the optimal control design for such a system. However, we will find that, simply transforming the dynamic equations of the system into the SDS form in the first place, the optimal control design becomes as straightforward as the usual optimal control design approach used for standard state space systems.Figure 1
Descriptor circuit system with impulse mode.Example 2.
Consider the descriptor system of the circuit shown on the left-hand side of Figure1, and its equivalent alternating current circuit is shown on the right. The dynamic equations for the descriptor system are given as
(40)[C000]x˙=[0110]x+[01]u,x=[vCiE]=[x1x2].
In the figure, vC is the capacitor voltage, iE is the emitter current, and α is the common-base current gain of the bipolar junction transistor. Suppose that we wish to make the changing rates of vC and iE to be as small as possible in the time interval [0,1], that is, keeping the values of vC and iE to be constant as possible. Then the cost functional can be selected as
(41)J=∫01(u2+x˙12+x˙22)dt.
The system may be transformed into the SDS form:(42)x=[00C0]x˙+[-10]u.
The Hamiltonian is given by
(43)H=u2+x˙12+x˙22+λ˙1(-u)+λ˙2(Cx˙1).
Necessary conditions (21) and (22) give
(44)∂H∂x˙1=2x˙1+Cλ˙2=-λ1,∂H∂x˙2=2x˙2=-λ2,∂H∂u=2u-λ˙1=0.
Thus, the optimal control is given by
(45)u=λ˙12.
We may put (42) and (44) in matrix form:
(46)X=[x1x2λ1λ2]=[00-0.50C000-200C0-200][x˙1x˙2λ˙1λ˙2]=A-X˙.
Note that the matrix A- is nonsingular for any capacitance C≠0. For illustrational purpose, we let C=1. The matrix equation (46) can now be rewritten as
(47)X˙=A--1X=[0100000-0.5-20000210]X=AX.
The general solution of (47) is given by
(48)X=[x1x2λ1λ2]=k1[0.32240.2535-0.8203-0.3986]e0.7862t+k2[0.3224-0.25350.8203-0.3986]e-0.7862t+k3[-0.2535cos(1.272t)-0.3224sin(1.272t)-0.3986sin(1.272t)-0.8203cos(1.272t)]+k4[-0.2535sin(1.272t)0.3224cos(1.272t)0.3986cos(1.272t)-0.8203sin(1.272t)],
where k1,…,k4 are constants. The terminal conditions (24) imply that λ1(1)=λ2(1)=0. These, together with the initial states x1(0) and x2(0), can be used to determine the constants k1,…,k4. If we let x1(0)=1 and x2(0)=0, then
(49)k1=0.6756,k2=0.2140,k3=-2.8134,k4=-0.3629.
The optimal control given by (45) can then be determined.


## 4. LQR Design for SDS Systems

In this section, we consider the LQR problem for linear time-invariant SDS systems. Suppose that the optimization problem can be stated as follows:(50a)minimizeJ=12∫t0tf(x˙TQx˙+uTRu)dt(50b)subjecttox=Ax˙+Bu.mmmmmmmmFor simplicity, we assume thatQ and R are symmetric positive definite matrices. The Hamiltonian is given by
(51)H(x˙,u,λ˙,t)=12(x˙TQx˙+uTRu)+λ˙T(Ax˙+Bu).
Necessary conditions (21) and (22) imply that
(52)∂H∂x˙=x˙TQ+λ˙TA=-λT,uTR+λ˙TB=0;
that is,
(53)λ=-Qx˙-ATλ˙,u=-R-1BTλ˙.
For a linear time-invariant SDS system, one can let
(54)λ=Sx,
where S is a constant matrix to be determined. By (53) and (54), we have
(55)u=-R-1BTλ˙=-R-1BTSx˙.
Substituting (55) into (50b), the closed-loop system becomes
(56)x=Ax˙-BR-1BTx˙=(A-BR-1BTS)x˙.
By (53), (54), and (56), we have
(57)S(A-BR-1BTS)x˙=Sx=λ=-Qx˙-ATλ˙=-Qx˙-ATSx˙;
that is,
(58)(SA+ATS-SBR-1BTS+Q)x˙=0.
Since this is true for any x˙, we obtain the following Riccati equation for SDS systems:
(59)SA+ATS-SBR-1BTS+Q=0.
By (55), the optimal state derivative feedback control is given by
(60)u=-Kx˙,K=R-1BTS,
and the closed-loop system becomes
(61)x=(A-BK)x˙.The optimal state derivative feedback control lawu=-Kx˙ may also be implemented using an equivalent state feedback u=-K-x. This is useful in real situations when only states but not the state derivatives are available for measurement. This can be proved as follows. The closed-loop system after applying u=-K-x becomes
(62)x=Ax˙-BK-x,
implying
(63)(I+BK-)x=Ax˙,
or
(64)x=(I+BK-)-1Ax˙.
In view of (61), we set
(65)(I+BK-)-1A=(A-BK).
Then, we have
(66)A=(I+BK-)(A-BK)=A+BK-A-BK-BK-BK,
or
(67)B(K-A-K-K-BK)=0.
Now let
(68)K-A-K-K-BK=0,
implying
(69)K-=K(A-BK)-1.


## 5. Controlling a Double Inverted Pendulum

Consider the double inverted pendulum system shown in Figure2 [24, 25], where θ1 is the angle of inclination of the first arm and θ2 is the angle of inclination of the second arm relative to the first arm. This experiment facility is controlled by Simulink interface [24]. As shown in the figure, there are two subcontrol systems in this double inverted pendulum.Figure 2
Double inverted pendulum.Our goal is to keep the first arm in a vertical position pointing downward and second arm in a vertical position but pointing upward. This means that we wish that(70)θ1=-90°,θ2=180°,θ˙1=0,θ˙2=0.
For this system, an initial control is usually needed so that θ2 approximately lies between 170° and 190° [25, 26]. An appropriate linearized model, accurate enough for θ2∈[170°,190°], is given by [24]:
(71)x˙=[00100001-15.8868.0562-0.0405082.08395.9987-0.02030]x+[002.16491.8943]u,
where xT=[θ1θ2θ˙1θ˙2] and u is the input voltage. Suppose that we wish to minimize the following cost functional:
(72)J=12∫t0tf(x˙TQx˙+uTRu)dt,
where Q=R=I. The controllable system (71) can be converted in SDS form:
(73)x=Ax˙+Bu=[-0.00170-0.04390.00370.001700.03750.007310000100]x˙+[0.0881-0.095000]u.
Using the method proposed in the last section, the optimal state derivative feedback control law (60) is given by u=-Kx˙, where
(74)K=[-10.9858-16.5454-0.1404-19751].
An equivalent state feedback control law is given by u=-K-x, where
(75)K-=[52.500962.62983.59225.4327].
We will use the state feedback control law in our real implementation through the Simulink control interface given in [24]. Typical simulated trajectories for θ1 and θ2 are shown in Figures 3 and 4, respectively. It is seen that the control goal is achieved. The full-length video taken during the experiment is available in [27].Figure 3
Trajectory ofθ1.Figure 4
Trajectory ofθ2.


## 6. Conclusion

Optimal control for SDS systems has been investigated in this paper. Necessary conditions for optimality were derived by the use of novel differential Lagrange multipliers. In the past, optimal control design for descriptor systems with impulse modes is not an easy task. We have shown via a real electric circuit that optimal control for a class of descriptor systems with impulse modes can easily be carried out using our design method. The optimal LQR design for linear time-invariant SDS systems using state derivative feedback can be obtained via an algebraic Riccati equation. Moreover, this optimal state derivative feedback may also be implemented using an equivalent state feedback. This is useful in real situations when only states are available for measurement. Note that our derivation and design method parallel those for systems in standard state space form. The LQR design for a double inverted pendulum system has been implemented to illustrate the use of our method.



---

*Source: 715026-2013-09-04.xml*