The 18th summer course of Behavior Modeling
Final presentation

マルコフ決定過程に基づく経路選択
行動のパラメータ推定—自動車・自転車交通施策の検討—
Evaluation of car/bicycle traffic measureswith a link choice model

University of Tokyo team A
Takuya Iizuka (M1)

Kenta Ishii (M1)
Shoma Dehara (M1)
Miho Yonezawa (M1)

1. Background2◆ Area: 松山市 Matsuyama city
Population: 512479 (2018.1.1.)

Area: 429.06 m2• Many people use private car.
• City projects are underway toincrease activity in the central city.

http://udcm.jp/project/

3

2. Basic Analysis
Representative Mode Choice inMatsuyama (n=7107)◆ Mode Choice• Data: Matsuyama PP(2007 Feb.19 – Mar.23)•High rate of Car & Bicycle useBicycle
17%

Walk
15%

Other1%

Car53%• Car & Bicycle paths are overlapping.

→By providing bicycle lanes,
traffic accidents can be suppressed !!

Taxi
0%

Bus
1%
Train
2%

Motorcycle
11%

Car
Motorcycle
Train
Bus
Taxi
Bicycle※経路情報が得られたトリップを抽出

4

2. Basic Analysis◆ Traffic Volume in the center of Matsuyama
Dogo Onsen

JR StationCity HallCenter StationCar Trip
• Most part of the center of Matsuyama,
the car & bicycle trips are separated.

• At some roads, car & bicycle trips areoverlapping!!

Dogo Onsen

JR Station

City Hall

Center Station

Bicycle Trip

5

2. Basic Analysis

Car & Bicycle traffic of each link
The smaller the traffic of the car,the more traffic of the bicycle.

On links with heavy car traffic,
sidewalks are maintained,increasing bicycle traffic.

3. Target◆ For Simulation• Characteristics of each link (length, width, etc.) affect travelers’ behavior.
→ We adopt Link Base Route Choice Model for analysis.

◆ Our Goal
• To clarify what is important element in the route choice behavior of car & bicycle• To simulate transport policy and to verify the sensitivity of each parameter6

74. Model◆ Estimation
Link based route choice model

Different Estimation method
Behavior model;
RL modelcompareInverse Reinforcement
Learning (IRL)parameter

link length
lanes
right turn dummy

84. Model◆ Sequential Route Choice Model: Recursive Logit model (RL) (Fosgerau et al., 2013)

Graph: 𝐺 = 𝐴, 𝜈absorbingstate

𝐴: set of links
𝜈: set of nodes• Utility Maximization problem

𝑣𝑛 𝑎 𝑘 + 𝜇𝜀𝑛 𝑎 + 𝛽𝑉𝑛𝑑 (𝑎)The value function is defined by the
Bellman equation (Bellman, 1957);

An instantaneous utility

𝑉𝑛𝑑 𝑘 = 𝐸 max 𝑣𝑛 𝑎 𝑘 + 𝜇𝜀𝑛 𝑎 + 𝛽𝑉𝑛𝑑 (𝑎)At each current state 𝑘, a traveler
chooses an action 𝑎 (next link).
𝜀𝑛 𝑎 : error term (i.i.d. Gumbel distribution)
𝜇: scale parameter
𝛽: discount rate∀𝑘 ∈ 𝐴An expected downstream utility:value function
from the selected state 𝑎 to thedestination link 𝑑𝑎∈𝐴(𝑘)Link choice probability

𝑃𝑛𝑑 𝑎 𝑘 =1(𝑣𝑛 𝑎 𝑘 +𝛽𝑉𝑛𝑑 (𝑎))𝜇𝑒σ𝑎′∈𝐴(𝑘)1′ 𝑘 +𝛽𝑉 𝑑 (𝑎′ ))(𝑣𝑎𝑛𝑛𝑒𝜇94. Compared IRL with RL◆ Bellman equation∞

𝑉 𝜋 𝑠 = 𝐸𝜋 ෍ 𝛾 𝑘 𝑟𝑡+𝑘+1 |𝑠𝑡 = 𝑠𝑘=0∞= 𝐸𝜋 𝑟𝑡+1 + 𝛾 ෍ 𝛾 𝑘 𝑟𝑡+𝑘+2 |𝑠𝑡 = 𝑠
𝑘=0∞𝑎 ℛ 𝑎 + 𝛾𝐸 ෍ 𝛾 𝑘 𝑟= ෍ 𝜋 𝑠, 𝑎 ෍ 𝒫𝑠𝑠′𝑠𝑠′
𝜋𝑡+𝑘+2 |𝑠𝑡+1 = 𝑠′𝑎𝑠′𝑘=0= 𝑄(𝑠, 𝑎)
𝑎 ℛ 𝑎 + 𝛾𝑉 𝜋 𝑠′
= ෍ 𝜋 𝑠, 𝑎 ෍ 𝒫𝑠𝑠′𝑠𝑠′𝑎𝑠′

𝛾: discount rate (0 < 𝛾 ≤ 1)
𝑎ℛ𝑠𝑠′
: expected reward(= 𝐸{𝑟𝑡+1 |𝑠𝑡 = s, 𝑎𝑡 = 𝑎, 𝑠𝑡+1 = s′ })𝑟𝑡+1𝑠𝑡Transition
state𝑎~𝜋(𝑠, 𝑎)

𝑠𝑡+1𝑎𝒫𝑠𝑠′= Pr{𝑠𝑡+1 = s′ |𝑠𝑡 = s, 𝑎𝑡 = 𝑎}

10

4. Compared IRL with RL◆ The estimation method : Recursive Logit model (RL) -NPL

Reward (Instantaneous utility): 𝑟𝑡 = 𝜽𝑻 𝑿

Parameter𝜽Valuefunction

Choiceprobability

Likelihood

Convergence
test
NoYes

The algorithm for calculating fixed point of value function 𝑉estimatedParameter 𝜽∗Convergence test෍ 𝑉𝑡 𝜽∗ − 𝑉𝑡 (𝜽) + ෍ 𝜽𝑻 − 𝜽 < 𝛿𝑡𝑡11

4. Compared IRL with RL◆ The estimation method : Max entropy - Inversed Reinforced Learning (IRL)

Reward: 𝑟𝑡 = 𝜽𝑻 𝑿
Reinforced Learning

Parameter𝜽Reward𝑟𝑡Policy(𝑄 value)

Likelihood
𝐿𝐿

Convergence
testNoYesProblem
max σ𝑖 log 𝑃 (𝜻𝑖 |𝜽)𝜽st. 𝑄𝑡 = 𝑟𝑡 + 𝛾𝑄𝑡+1

where 𝜻𝑖 is the path of expert
𝑿 is the feature relating to link

estimated
Parameter 𝜽∗12

5. Estimation Result◆ RL estimation (car)◆ IRL estimation (car)𝛽 = 0.47 (given)

𝛽 = 0.47 (given)

Variables

Parameters

t-Value

Variables

Parameters

t-Value

Link Length-0.03-1.33

Link Length-0.07-9.72**

Right-Turn-0.80-6.49**

Right-Turn-1.02-8.53**

Lanes

0.37

2.76**

Lanes

-0.37

-5.64**

L(0)-1179.29

L(0)-2080.67

LL-1147.00

LL-1117.10

Rho-Square

0.03

Rho-Square

0.46

Adjusted Rho-Square

0.02

Adjusted Rho-Square

0.46

13

5. Estimation Result◆ Recursive Logit estimation (bicycle)

Variables

Parameters

t-Value

Link Length-0.00-6.21**

Right-Turn-0.19-3.67**Car Traffic-14.37

-0.14

β0.00

15.15**

L(0)-4093.90

LL-3861.56

Rho-Square

0.06

Adjusted Rho-Square0.06

5. Simulation and Evaluation
Network Policy

𝐺 = 𝑙𝑖𝑛𝑘, 𝑛𝑜𝑑𝑒, 𝑙𝑎𝑛𝑒

Car Assignment

𝑣𝑐𝑎𝑟 = 𝜃1 ∙ 𝐿𝑒𝑛𝑔𝑡ℎ + 𝜃2 ∙ 𝑅𝑖𝑔ℎ𝑡𝑡𝑢𝑟𝑛 + 𝜃3 ∙ 𝐿𝑎𝑛𝑒𝑠

Car traffic

Bicycle Assignment

𝑣𝑏𝑖𝑐𝑦𝑐𝑙𝑒 = 𝜃4 ∙ 𝐿𝑒𝑛𝑔𝑡ℎ + 𝜃5 ∙ 𝑅𝑖𝑔ℎ𝑡𝑡𝑢𝑟𝑛 + 𝜃6 ∙ 𝐶𝑎𝑟𝑇𝑟𝑎𝑓𝑓𝑖𝑐

14

15

5. simulation←Bicycle traffic

PolicyReduce the lanes of large
bicycle traffic links
Private car/bicycle user’s logsum value with/without policy

Without policy

With policy(rode lanes are reduced)

Private car user-2639-2638

Bicycle user-9297

-1147

16

6. Future works◆ Policies decided by Two-stage optimization
To decide the policyby calculating the fixed point of demand of cars and bicycles

Variables is changed

Policychange

Demandchange
Consumer surplus

20

4. Frame & Model◆ Estimation◆ Policy Simulation

Link based route choice model

Upper Problem: traffic network

Different Estimation method• reduction of vehicle lanes(pedestrian/bicycle only)

Behavior model;
RL modelcompareInverse Reinforcement
Learning (IRL)parameter

link lengthlanes
right turn dummy

traffic volumeof each link

network

Lower Problem: route choice behavior
CarBicycle

Assign each OD volume