# Iterative Learning Control with Forgetting Factor for Urban Road Network

Authors: Tianyi Lan; Fei Yan; Hui Lin
Journal: Journal of Control Science and Engineering
 (2017)

---

## Abstract

In order to improve the traffic condition, a novel iterative learning control (ILC) algorithm with forgetting factor for urban road network is proposed by using the repeat characteristics of traffic flow in this paper. Rigorous analysis shows that the proposed ILC algorithm can guarantee the asymptotic convergence. Through iterative learning control of the traffic signals, the number of vehicles on each road in the network can gradually approach the desired level, thereby preventing oversaturation and traffic congestion. The introduced forgetting factor can effectively adjust the control input according to the states of the system and filter along the direction of the iteration. The results show that the forgetting factor has an important effect on the robustness of the system. The theoretical analysis and experimental simulations are given to verify the validity of the proposed method.


---

## Body

## 1. Introduction

Urban road intersection is the meeting center of traffic flow in urban network. Whether the vehicles can drive safely and orderly through the intersection or not has an important influence on the patency of the road network. The signal control of the intersection ensures the efficient operation of traffic flow in urban network and prevents traffic congestion.At present, many signal control methods for urban road intersection have been developed by researchers based on different theories. Some corresponding signal control systems have been developed and put into use, such as SCOOT [1], SCATS [2], UTOPIA/SPOT [3], RHODES [4], OPAC [5], and PRODYN [6]. According to the different theories, there are several methods, for instance, the optimal control [7, 8], model predictive control [9, 10], and agent based method [11]. However, most of them are still in the theoretical research stage. Based on the optimal control theory, Aboudolas et al. proposed a signal control method for urban road intersection, which can calculate the signal timing of large-scale urban road network with the advantages of simple structure and small amount of calculation [7, 8]. But the method performs inappropriately when the traffic state is oversaturated. In [9, 10], Van den Berg et al. introduced a signal control method for urban road intersection based on model predictive control, which can calculate the signal timing online rolling and operate easily; however, tedious calculation is necessary for the large-scale urban road network. Negenborn et al. proposed a signal control method for urban road intersection based on the agent based method, which can work out the urban traffic signal control in distributed manner and avoid the complex calculation of the centralized control [11]. However, this method is difficult to achieve in practice.From the macro point of view, traffic flow has obvious characteristics of repeatability. Therefore, how to make use of the repetitive nature of traffic flow to control the intersection is important for improving urban traffic conditions. For the system with characteristics of repeatability, iterative learning control, which is proposed by Arimoto et al. [12], can deal with the control problems of complex nonlinear dynamic, time-varying, and unknown system in a simple way, and it has been widely used.In the field of traffic control, Hou and Xu applied the iterative learning control method to expressway ramp control [13] and further proposed the hybrid ramp control method by combing the iterative learning control and feedback control to enhance robustness of the system [14]. However, there is little research on the ILC for urban road intersection control [15–17]. Therefore, the iterative learning control algorithm with forgetting factor for urban road intersection is discussed in this work. The introduced forgetting factor can effectively adjust the control input according to the states of the system and filter along the direction of the iteration. So it can accelerate the speed of convergence, smooth the tracking error curve, track the desired trajectory better, and reduce the impact of internal interference in the system on the convergence. The robustness of iterative learning control with forgetting factor is proven by rigorous analysis. Finally, the effectiveness of the method is verified by numerical simulation.


## 2. Traffic Flow Modeling

Leto
1 and o
2 be the two adjacent intersections, the corresponding road is l, d
l denotes the input flow of road l, and t
l is the output flow, which is shown in Figure 1.Figure 1
Signalized intersections.Then the number of vehiclesx
l on road l at time k is (1)
x
l
k
+
1
=
x
l
k
+
T
d
l
k
-
t
l
k
,where T is the control cycle and k
=
0,1
,
2
,
…
,
K is the sampling time.The input flowd
l
(
k
) at road l is the sum of traffic from the upper roads; that is,(2)
d
l
k
=
∑
w
∈
I
o
1
τ
w
,
l
t
w
k
,where I
o
1 is the road collection for traffic into the intersection o
1 and τ
w
,
l is the rate for traffic flow through road w into road l.Letg
l
(
k
) be the effective green light time for road l at time k; then the output flow t
l
(
k
) is (3)
t
l
k
=
g
l
k
C
S
l
,where g
l
(
k
)
=
∑
i
∈
V
l
u
o
2
,
i
(
k
), u
o
2
,
i
(
k
) is the green time for phase i at intersection o
2, and V
l is the phase set with right of way for road l. S
l is the saturated flow for road l; C is the signal cycle. Submitting (2) and (3) into (1), we can get the state function of road l as follows:(4)
x
l
k
+
1
=
x
l
k
+
T
∑
w
∈
I
o
1
τ
w
,
l
S
w
C
∑
i
∈
V
w
u
o
1
,
i
k
-
S
z
C
∑
i
∈
V
l
u
o
2
,
i
k
.Therefore, for different phases at intersectiono, the green time u
o
,
i, the loss time l
o, and signal cycle C should satisfy the following equation:(5)
∑
i
∈
F
o
u
o
,
i
k
+
l
o
=
C
.Considering the pedestrian crossing time,u
o
,
i should also satisfy (6)
u
o
,
i
m
i
n
≤
u
o
,
i
k
≤
u
o
,
i
m
a
x
,where u
o
,
i
m
i
n and u
o
,
i
m
a
x are the minimum and maximum of u
o
,
i, respectively.


## 3. State Space Model and Assumptions

### 3.1. State Space Model

Define the state equation (4) for all roads of the urban network, which combines with the output equation. The state space model of urban network is shown as follows:(7)
x
k
+
1
=
A
x
k
+
B
u
k
,
y
k
+
1
=
C
x
k
+
1
,where x
(
k
) is the state vector, u
(
k
) is the control vector, y
(
k
) is the output vector, and state matrix A and output matrix C are the identity matrix, respectively. The elements of input matrix B contain the characteristic parameters of road network such as phase, cycle, turning saturation flow rate, and other parameters.According to the actual requirement, the control inputu
(
k
) must be in a reasonable range; that is to say, u
(
k
) must satisfy formula (6) and u
(
k
)
∈
[
u
m
i
n
(
k
)
,
u
m
a
x
(
k
)
]; this condition can also be illustrated by the saturation function u
(
k
), which is shown as follows:(8)
s
a
t
u
k
=
u
m
i
n
k
,
u
k
≤
u
m
i
n
k
,
u
k
,
u
m
i
n
k
<
u
k
<
u
m
a
x
k
,
u
m
a
x
k
,
u
k
≥
u
m
a
x
k
.Therefore, when the input is limited, the state space equation of (7) is (9)
x
k
+
1
=
A
x
k
+
B
s
a
t
u
k
,
y
k
+
1
=
C
x
k
+
1
.


### 3.2. Assumptions

Assumption 1.
In the process of iteration, the initial condition should be satisfied:(10)
x
n
0
=
x
d
0
,
y
n
0
=
y
d
0
,
∀
n
,where x
d
(
0
) and y
d
(
0
) are the initial values for the desired state and desired output, respectively. n is the number of iterations.Assumption 2.
For the given desired outputy
d
(
k
)
(
k
∈
[
0
,
K
]
), there exists the control input u
d
(
k
)
(
k
∈
[
0
,
K
]
) which satisfies(11)
x
d
k
+
1
=
A
x
d
k
+
B
u
d
k
,
y
d
k
+
1
=
C
x
d
k
+
1
,where x
d
(
k
)
(
k
∈
[
0
,
K
]
) is the corresponding desired state of y
d
(
k
)
(
k
∈
[
0
,
K
]
).


### 3.3. Control Objective

Through the iterative learning control of green time, the number of vehicles on[
0
,
K
] can approach the reasonable expectation in the urban network, and then traffic congestion can be avoided by preventing the excessive saturated traffic flow.


## 3.1. State Space Model

Define the state equation (4) for all roads of the urban network, which combines with the output equation. The state space model of urban network is shown as follows:(7)
x
k
+
1
=
A
x
k
+
B
u
k
,
y
k
+
1
=
C
x
k
+
1
,where x
(
k
) is the state vector, u
(
k
) is the control vector, y
(
k
) is the output vector, and state matrix A and output matrix C are the identity matrix, respectively. The elements of input matrix B contain the characteristic parameters of road network such as phase, cycle, turning saturation flow rate, and other parameters.According to the actual requirement, the control inputu
(
k
) must be in a reasonable range; that is to say, u
(
k
) must satisfy formula (6) and u
(
k
)
∈
[
u
m
i
n
(
k
)
,
u
m
a
x
(
k
)
]; this condition can also be illustrated by the saturation function u
(
k
), which is shown as follows:(8)
s
a
t
u
k
=
u
m
i
n
k
,
u
k
≤
u
m
i
n
k
,
u
k
,
u
m
i
n
k
<
u
k
<
u
m
a
x
k
,
u
m
a
x
k
,
u
k
≥
u
m
a
x
k
.Therefore, when the input is limited, the state space equation of (7) is (9)
x
k
+
1
=
A
x
k
+
B
s
a
t
u
k
,
y
k
+
1
=
C
x
k
+
1
.


## 3.2. Assumptions

Assumption 1.
In the process of iteration, the initial condition should be satisfied:(10)
x
n
0
=
x
d
0
,
y
n
0
=
y
d
0
,
∀
n
,where x
d
(
0
) and y
d
(
0
) are the initial values for the desired state and desired output, respectively. n is the number of iterations.Assumption 2.
For the given desired outputy
d
(
k
)
(
k
∈
[
0
,
K
]
), there exists the control input u
d
(
k
)
(
k
∈
[
0
,
K
]
) which satisfies(11)
x
d
k
+
1
=
A
x
d
k
+
B
u
d
k
,
y
d
k
+
1
=
C
x
d
k
+
1
,where x
d
(
k
)
(
k
∈
[
0
,
K
]
) is the corresponding desired state of y
d
(
k
)
(
k
∈
[
0
,
K
]
).


## 3.3. Control Objective

Through the iterative learning control of green time, the number of vehicles on[
0
,
K
] can approach the reasonable expectation in the urban network, and then traffic congestion can be avoided by preventing the excessive saturated traffic flow.


## 4. Iterative Learning Control with Forgetting Factor for Urban Network

The definition of norms used in this paper is given as follows:(12)
g
k
λ
=
s
u
p
k
∈
0
,
K
a
-
λ
k
g
k
,where λ
>
0 and a
>
1.In the paper,· denotes the infinite norm, thus, for the matrix W with size s
×
t, whose elements denote w
i
,
j,(13)
W
=
m
a
x
1
≤
i
≤
t
∑
j
=
1
s
w
i
,
j
.For system (9), the iterative learning control law with forgetting factor in this paper is designed as follows:(14)
u
n
+
1
k
=
1
-
α
s
a
t
u
n
k
+
α
u
0
k
+
β
e
n
k
+
1
,where e
n
(
k
)
=
y
d
(
k
)
-
y
n
(
k
) is the tracking error for the nth iteration, β is the iterative learning gain matrix, and α is the iterative forgetting factor for the system which satisfies 0
<
α
<
1.Before investigating learning properties under constraints, we first introduce an important Property:(15)
u
d
k
-
s
a
t
u
n
k
≤
u
d
k
-
u
n
k
.Proof.
Whenu is a scalar, we can easily obtain that (16)
u
d
k
-
s
a
t
u
n
k
2
-
u
d
k
-
u
n
k
2
=
s
a
t
u
n
k
-
u
n
k
s
a
t
u
n
k
+
u
n
k
-
2
u
d
k
.For limited u, there are three possible cases:(1)
u
n
(
k
)
∈
[
u
m
i
n
(
k
)
,
u
m
a
x
(
k
)
].
(2)
u
n
(
k
)
>
u
m
a
x
(
k
).
(3)
u
n
(
k
)
<
u
m
i
n
(
k
).
(1) Whenu
n
(
k
)
∈
[
u
m
i
n
(
k
)
,
u
m
a
x
(
k
)
], u
n
(
k
)
=
s
a
t
[
u
n
(
k
)
]; then (15) is equal to zero, so the “=” relationship of Property holds.
(2) Whenu
n
(
k
)
>
u
m
a
x
(
k
), s
a
t
[
u
n
(
k
)
]
=
u
m
a
x
(
k
), and (17)
s
a
t
u
n
k
-
u
n
k
=
u
m
a
x
k
-
u
n
k
<
0
,
s
a
t
u
n
k
+
u
n
k
-
2
u
d
k
>
0
.From (17), it is obvious that (16) is less than zero.
(3) Whenu
n
(
k
)
<
u
m
i
n
(
k
), s
a
t
[
u
n
(
k
)
]
=
u
m
i
n
(
k
); thus(18)
s
a
t
u
n
k
-
u
n
k
=
u
m
i
n
k
-
u
n
k
>
0
,
s
a
t
u
n
k
+
u
n
k
-
2
u
d
k
<
0
.
From (18) it can be concluded that (16) is less than zero.
From the above discussions, we can see that the Property holds whenu is a scalar. Below we give the proof of the Property for vector-valued u.(19)
u
d
k
-
s
a
t
u
n
k
2
=
∑
j
=
1
N
u
j
,
d
k
-
s
a
t
u
j
,
n
k
2
≤
∑
j
=
1
N
u
j
,
d
k
-
u
j
,
n
k
2
=
u
d
k
-
u
n
k
2
.This means that Property still holds.The robustness theorem of iterative learning control is given as follows.Theorem 3.
For system (9) which satisfies Assumptions 1 and 2, if there exists a matrix β which satisfies (
1
-
α
)
I
-
β
C
B
<
1, then the tracking error converges to bounded range for [
0
,
K
] with the iterative learning control law (14); that is,(20)
l
i
m
n
→
∞
s
u
p
k
∈
0
,
K
e
n
k
λ
≤
b
ξ
1
-
ρ
.Proof.
Define(21)
δ
x
n
k
=
x
d
k
-
x
n
k
,
δ
u
n
k
=
u
d
k
-
u
n
k
.The state equation for the nth iteration is (22)
x
n
k
+
1
=
A
x
n
k
+
B
s
a
t
u
n
k
,
y
n
k
+
1
=
C
x
n
k
+
1
.From Assumption 2, we have (23)
e
n
k
+
1
=
y
d
k
+
1
-
y
n
k
+
1
=
C
A
x
d
k
+
B
u
d
k
-
A
x
n
k
-
B
s
a
t
u
n
k
=
C
A
x
d
k
-
x
n
k
+
B
u
d
k
-
s
a
t
u
n
k
=
C
A
x
d
k
-
x
n
k
+
C
B
u
d
k
-
s
a
t
u
n
k
.From formula (14), we have(24)
δ
u
n
+
1
k
=
1
-
α
u
d
k
-
s
a
t
u
n
k
+
α
u
d
k
-
u
0
k
-
β
e
n
k
+
1
.Submitting (23) into (24), we have(25)
δ
u
n
+
1
k
=
1
-
α
u
d
k
-
s
a
t
u
n
k
+
α
u
d
k
-
u
0
k
-
β
C
A
x
d
k
-
x
n
k
+
B
u
d
k
-
s
a
t
u
n
k
=
1
-
α
u
d
k
-
s
a
t
u
n
k
+
α
u
d
k
-
u
0
k
-
β
C
A
x
d
k
-
x
n
k
-
β
C
B
u
d
k
-
s
a
t
u
n
k
=
1
-
α
I
-
β
C
B
u
d
k
-
s
a
t
u
n
k
+
α
δ
u
0
k
-
β
C
A
δ
x
n
k
.Taking the norm on both sides of formula (25), we have(26)
δ
u
n
+
1
k
≤
1
-
α
I
-
β
C
B
u
d
k
-
s
a
t
u
n
k
+
α
δ
u
0
k
+
β
C
A
δ
x
n
k
≤
1
-
α
I
-
β
C
B
u
d
k
-
u
n
k
+
α
δ
u
0
k
+
β
C
A
δ
x
n
k
≤
1
-
α
I
-
β
C
B
δ
u
n
k
+
α
δ
u
0
k
+
ε
δ
x
n
k
,where ε
=
β
C
A.(27)
δ
x
n
k
=
x
d
k
-
x
n
k
=
A
x
d
k
-
1
+
B
u
d
k
-
1
-
A
x
n
k
-
1
-
B
s
a
t
u
n
k
-
1
=
A
x
d
k
-
1
-
x
n
k
-
1
+
B
u
d
k
-
1
-
s
a
t
u
n
k
-
1
.Taking the norm on both sides of formula (27), we have(28)
δ
x
n
k
≤
A
δ
x
n
k
-
1
+
B
u
d
k
-
1
-
s
a
t
u
n
k
-
1
≤
A
δ
x
n
k
-
1
+
B
δ
u
n
k
-
1
≤
A
2
δ
x
n
k
-
2
+
A
B
δ
u
n
k
-
2
+
B
δ
u
n
k
-
1
≤
A
3
δ
x
n
k
-
3
+
A
2
B
δ
u
n
k
-
3
+
A
B
δ
u
n
k
-
2
+
B
δ
u
n
k
-
1
≤
A
k
δ
x
n
0
+
∑
j
=
0
k
-
1
A
k
-
j
-
1
B
δ
u
n
j
.Taking the λ-norm on both sides of (28) with Assumption 1, we have (29)
δ
x
n
k
≤
∑
j
=
0
k
-
1
a
k
-
j
-
1
b
δ
u
n
j
,where a
=
A and b
=
B.
Takingλ-norm on both sides of (29), we have(30)
δ
x
n
k
λ
≤
s
u
p
k
∈
0
,
K
a
-
λ
k
∑
j
=
0
k
-
1
a
k
-
j
-
1
b
δ
u
n
j
≤
a
-
1
b
s
u
p
k
∈
0
,
K
∑
j
=
0
k
-
1
a
-
λ
k
a
λ
j
a
k
-
j
s
u
p
k
∈
0
,
K
a
-
λ
j
δ
u
n
j
≤
a
-
1
b
δ
u
n
k
λ
s
u
p
k
∈
0
,
K
∑
j
=
0
k
-
1
a
λ
-
1
j
-
k
≤
δ
u
n
k
λ
1
-
a
1
-
λ
K
a
λ
-
a
b
.Taking λ-norm on both sides of (26), we have(31)
δ
u
n
+
1
k
λ
≤
1
-
α
I
-
β
C
B
δ
u
n
k
λ
+
α
δ
u
0
k
λ
+
ε
δ
x
n
k
λ
.Submitting (30) into (31), we have(32)
δ
u
n
+
1
k
λ
≤
1
-
α
I
-
β
C
B
+
1
-
a
1
-
λ
K
a
λ
-
a
b
ε
δ
u
n
k
λ
+
α
δ
u
0
k
λ
.From the convergence condition,(33)
1
-
α
I
-
β
C
B
<
1
.When λ tends to infinity, we have(34)
1
-
α
I
-
β
C
B
+
1
-
a
1
-
λ
K
a
λ
-
a
b
ε
≤
ρ
<
1
.Form (32), we can also obtain the following result:(35)
δ
u
n
+
1
k
λ
≤
ρ
δ
u
n
k
λ
+
ε
1
≤
ρ
n
+
1
δ
u
0
k
λ
+
ε
1
1
+
ρ
n
+
1
1
-
ρ
,where ε
1
=
α
δ
u
0
(
k
)
λ.
Whenn
→
∞, the following result is obtained: (36)
l
i
m
n
→
∞
δ
u
n
+
1
k
λ
≤
ε
1
1
-
ρ
.From (30) and (36), we get(37)
δ
x
n
k
λ
≤
ε
1
1
-
ρ
·
1
-
a
1
-
λ
K
a
λ
-
a
b
.When λ tends to infinity, we have ε
1
→
0; therefore(38)
l
i
m
n
→
∞
δ
x
n
+
1
k
λ
=
0
.We can also get the following result:(39)
l
i
m
n
→
∞
e
n
+
1
k
λ
=
0
.Thus, we can complete the proof of the theorem.


## 5. Simulation Research

In order to validate the effectiveness of the proposed iterative learning control algorithm with forgetting factor for urban network, the following simulation results are given by using VISSIM. Figure2 shows the test network, which is composed of 36 links and 9 intersections. Each link has two-way lanes. VISSIM is employed to simulate the real traffic environment for simulations and provides the traffic measurements of the states to MATLAB for calculating the new input signals by the ILC-based control laws. Different from the methods in [15–17], the new input signals are calculated by the proposed iterative learning control law (14) in the simulation.Figure 2
The simulation network.The input flow of the network for the starting link is shown in Figure3.Figure 3
The input flow for the network.We compare the proposed method with fixed timing (FT) method to verify the effectiveness of the proposed method. The result of fixed timing method is calculated by the classical Webster optimization program. In order to minimize the initial iterative error, the initial inputu
0
(
k
) for (14) is selected as the fixed timing plan.Selecting the vehicle average delay time (TDT) and the number of stops (AVS) as the evaluation indexes, the simulation results are shown in Table1.Table 1
Simulation results.
Control strategy
Fixed timing method
Iterative learning control
Evaluation index
TDT(s)
AVS
TDT(s)
AVS
1
93.06
1.69
82.30
1.56
2
92.73
1.72
80.96
1.59
3
96.15
1.81
83.77
1.65Table1 shows that the traffic conditions under iterative learning control algorithm with forgetting factor are obviously improved, compared to the fixed timing control. The maximum total delay time decreases from 96.15 h to 83.77 h, as low as 12.9%. The maximum average vehicle stop time reduces from 1.81 to 1.65; the decrease rate is 8.8%. The result shows that iterative learning control algorithm with forgetting factor has strong robustness.The variation curves of total network delay time and average number of vehicle stops are illustrated in Figures4 and 5, respectively.Figure 4
Total delay time for two control strategies.Figure 5
The average number of vehicle stops for two control strategies.Figures4 and 5 show that there is no significant change for the total delay time and average vehicle stop times at the initial stages due to the lighter traffic flow. But with the increasing of traffic flow, compared with iterative learning control, the total delay time and average vehicle stop times increase significantly under the fixed timing control strategy after 1200 s. The results show that the iterative learning control algorithm with forgetting factor can control traffic flow by adjusting the green time according to the traffic conditions. Then the traffic conditions can be improved by reducing the total delay time and the number of vehicle stops.


## 6. Conclusions

This work proposes an iterative learning control algorithm with forgetting factor for urban road intersection. The robustness of iterative learning control algorithm is proven by using mathematical analysis. The number of vehicles on different roads in the network approaches the reasonable expectations by using the iterative learning control of the traffic signal. The result ensures that the green time is used completely and prevents traffic jams. The results show that the forgetting factor has an important effect on the robustness of the system. The effectiveness of the algorithm is verified by the theoretical analysis and numerical simulations.



---

*Source: 9269187-2017-02-28.xml*