September 21-23, 2019
The 18th Summer Course for Behavior Modeling in Transportation Networks@The University of Tokyo

Urban science and behavioral informatics with AI/machine learning

Application of AI for travel behavior
modelling in urban networks
Makoto Chikaraishi
Hiroshima University

Problem setting• Standard logit model: 𝑃• The conventional form of

exp 𝑉
Σ

exp 𝑉

:

– Linear approximation (rooted to the Taylor's
theorem)– Also known as a linear-in-parameter model• Problem at hand:– Is there any better way to determine thefunctional form?
• Obviously, taking into account the non-linearlity of 𝑉
would improve the goodness-of-fit.
• What is the cost of doing that?
2

Problem setting• Can we understand the non-linear
transformation oflogically?
Example: contribution of travel time to mode/route choice model
𝑉

Highly non-linear transformation

Linear approximation
Logarithm transformation
Travel time of mode j

3Problem setting• In what context we may NOT need to
explain the impacts of each factor logically?
Example: contribution of eye movement to the speed choice
Speed-up
Constant
Speed-down

Adapted from Wang et al. (2019)4

Contents
1. Logically explainable non-linear
transformation
2. Spline-based non-linear transformation
3. NN (Neural network)-based non-linear
transformation5

Advanced discrete choice models[based on Hato (2002)]
Normal to Gumbel

Multinomial logit (MNL)(Luce, 1959)

Error component logit (ECL); Mixed
logit (MXL); Kernel logit (KL);Heteroscedastic logit (HL)
(Boyd and Mellman, 1980; Cardell andDunbar, 1980; McFadden, 1989; Bhat,
1995; See Train (2009) for details)

Nested logit (NL)
(Ben-Akiva, 1973)
Generalization

Generalized extreme value (GEV)(McFadden, 1978)
GumbelClosed-form models

Heteroscedastic/mixed distributions

Generalization

Generalization

Special caseOpen-form models

Multinomial Probit (MNP)(Thurstone, 1927)

1

Weibull to GEV (not MEV)

to Weibull Multinomial weibit (MNW)

Paired combinational
Cross-nested logit
logit (PCL) (Chu, 1981) (CNL) (Vovsha, 1997)(Castillo, et al., 2008)

Generalized nested logit (GNL), recursive nested
logit extreme value model (RNEV), network-GEVGeneralized G function(Mattsson et al., 2014)

Generalization

Generalization(Wen & Koppelman, 2001; Daly, 2001; Bierlaire, 2002)

q-generalized logit(Nakayama, 2013,
Nakayama andChikaraishi, 2015)

2

Variancestabilization(Li, 2011)

3

Deep neuralnetwork6

Derived from McFaddenʼs G function or “choiceprobability generating functions” (Fosgerau et al., 2013)
Derived from the generalized G function

Castillo, E., Menendez, J.M., Jimenez, P., Rivas, A. (2008) Closed form
expressions for choice probabilities in the Weibull case. Transportation
Research Part B 42, 373-380.
Chikaraishi, M., Nakayama, S. (2016) Discrete choice models with qproduct random utilities, Transportation Research Part B

LOGICALLY EXPLAINABLE
NON-LINEAR
TRANSFORMATION

7

Nonlinearity of(2) Difference in systematic utilityuij  f Vij    ij
Linear systematic utility

εij:

Gumbel distribution

Logarithm systematic utility
Vij

-uij
Choice probability 1 exp  Vij 
  
pij 
 1 
exp
k    Vik 

Vij

-uij
Choice probability


pij 

Vij

1




1

V 
k

ik

8

Distribution/linearity: an example(1) Difference in distribution assumptionuij  g Vij ,  ij 
Gumbel distribution

Logit model

𝜀 isindependent from 𝑉 1 exp  Vij 
  
pij 
 1 
exp
k    Vik 

𝑢 : Random utility
𝑉 : Systematic utility (linear in parameters)
εij: Error term

Weibull distribution

𝜀 isdependent
on 𝑉

Weibit (or multiplicative) model


pij 

Vij

1




1

V 
k

ik

9

Distribution/linearity: an example
uij  g  f Vij ,  ij 
Randam utility distribution:

Gumbel  Weibull
Logit
model

Castillo et al. (2008)

Systematic utility function:

Weibitmodel

Linear  Logarithm(See Castillo et al. (2008) for elegant explanations)10

q-product random utility
q-product
random utility

q-generalized
GumbelDerivation of
q-product logit

Behavioral
implication• Preliminaries: q-generalization (Tsallis, 2009)– Generalized Boltzmann–Gibbs statistical mechanics– The core concept is the so-called Tsallis entropy, where
the “q-generalization” plays a central role
q-logarithm:

ln 𝑥 ≔

q-exponential: exp

𝑥

1
1

𝑞

𝑥 ≔ 1

1

lim ln 𝑥

ln 𝑥→

lim exp 𝑥

𝑞 𝑥

q-product: 𝑥 ⊗ 𝑦:𝑥𝑦Some properties:
ln exp 𝑥𝑥,

ln 𝑥 ⊗ 𝑦→lim 𝑥 ⊗ 𝑦

1

→

ln 𝑥

ln 𝑦

exp 𝑥
𝑥𝑦

11

8

q i =0.06

q i =0.2

4q i =0.4
q i =0.6
q i =0.8
q i =1.0-202lnq i (x)10Generalization of logit and weibit

0

2

4

6

8

10

x

q-logarithm:

ln 𝑥 ≔𝑥1
1

𝑞

lim ln 𝑥
→

ln 𝑥
12

q-generalized random term
• q-generalized reverse Gumbel distribution
𝑓 𝑥

𝑔

𝑦

exp

exp

exp

2

3

4

5

6

0.5
0

1

2

3

4

5

6

0

1

2

3
x

4

5

6

3

4

5

6

x
Solid line: i j =1; i =1/3; q i =1.0
Dotted line: i j =2; i =1/3; q i =1.0
Dashed line: i j =3; i =1/3; q i =1.0

0.0

0.5

f i j (x)
0.0

0.0

0.5

0.5

f i j (x)f i j (x)

0

2

1.0

Solid line: i j =1; i =1/3; q i =0.9
Dotted line: i j =2; i =1/3; q i =0.9
Dashed line: i j =3; i =1/3; q i =0.9

1.0

1.0

Solid line: i j =1; i =1/3; q i =0.7
Dotted line: i j =2; i =1/3; q i =0.7
Dashed line: i j =3; i =1/3; q i =0.7

1

(Weibull)

x
1.5

1.5

x

1.5

1

0.0

0.0

0.0

0.5

0.5

f i j (x)f i j (x)1.0
f i j (x)0Solid line: i j =1; i =1/3; q i =0.3
Dotted line: i j =2; i =1/3; q i =0.3
Dashed line: i j =3; i =1/3; q i =0.3

1.0

Solid line: i j =1; i =1/3; q i =0.1
Dotted line: i j =2; i =1/3; q i =0.1
Dashed line: i j =3; i =1/3; q i =0.1

1.0

Solid line: i j =1; i =1/3; q i =0.0
Dotted line: i j =2; i =1/3; q i =0.0
Dashed line: i j =3; i =1/3; q i =0.0

1.5

1.5

1.5

(reverse Gumbel)

0

1

2

3
x

4

5

6

0

1

2

3
x

4

5

6

13

Behavioral implication• Risk attitude

ln 𝜇 is an isoelastic utility function (also known as a
power utility function), which has been widely used ineconomics
This gives the parameter 𝑞 a clear behavioral meaning:𝒒𝒊 is equivalent to the Arrow–Pratt measure of
relative risk aversion when imposing Gumbel
distribution on its error component

𝑅 𝜇

𝜇

𝜕 ln

𝜇

⁄𝜕 𝜇

𝜕ln

𝜇

⁄𝜕 𝜇

𝑞

14

Behavioral implication• Example
Route 1: travel cost μ1 = 0.1x

OriginDestination

Choice probability:

Route 2: travel cost μ2 = 0.1(x + 5)

exp( ln q (  j ))

Pj 



3

Route 3: travel cost μ3 = 0.1(x + 10)

80%

80%

80%

60%

60%

60%

40%

Share

100%

Share

100%

Share

100%

40%
20%

20%
Route 1

Route 2

Route 2

10

(a) 𝑞

20

x

30

40

0.0 (logit)

50

40%

Route 3

Route 10%0

exp( ln q (  j ' ))20%
Route 1

Route 30%j '1

Route 2

Route 3

0%
0

10

20

(b) 𝑞

x

30

0.5

40

50

0

10

(c) 𝑞

20

x

30

40

50

1.0 (weibit)

15

Other choice models with nonlinearity transformation

Swait, J., 2001. A non-compensatory choice model incorporating attribute
cutoffs. Transportation Research Part B: Methodological 35, 903-928.
16

Li, B. (2011) The multinomial logit model revisited: A semiparametric approach in discrete choice analysis.
Transportation Research Part B 45, 461-473.

SPLINE-BASED NONLINEAR TRANSFORMATION

17

Choice probability
Li (2011) shows that we can derive a number of discrete choice models
under different error tem distributions:
𝑃

exp

𝜃𝛼

𝛼 exp 𝜃𝑧 exp 𝜃𝑧 𝑑𝑧

∈
𝛃𝐱 𝐢𝐣
∈

∈

∈

𝛃𝐱 𝐢𝐣

The above equation indicates the choice of error term distribution would
be equal to the choice of non-linear transformation of systematic utility.
(as we already confirmed)
18

Semi-parametric discrete choice models
Semi-parametric approach (such as P-splines approach) can beused as an approximation of any base distribution.
𝐻 𝑉

exp 𝑆 𝛃𝐱 𝐢𝐣Σ ∈ 𝐻 𝑉Σ ∈ exp 𝑆 𝛃𝐱 𝐢𝐣

𝑆 𝛃𝐱𝐢𝐣 : Sensitivity function

19

Emergence of Deep Learning• Limitations of linear-in-parameter model– No consideration of non-linearity– No consideration of interactions among variables• Possible solutions (Goodfellow et al., 2016)
1. Theory-driven (e.g., assuming noncompensatory, using BPR function, etc.)
2. Use Kernel, Splines, etc.
3. Learn the function from the data (e.g., deeplearning)  produce more accurate results inmany cases20Sifringer, B., Lurkin, V., Alahi, A., 2018. Enhancing Discrete
Choice Models with Neural Networks. 18th Swiss Transport
Research Conference, Monte Verità, May 16–18.

DISCRETE CHOICE WITH
NEURAL NETWORK

21

Background and objective• RUM model vs neural network– Advantage of RUM model• Interpretability of the results.

– Advantage of neural network• Better goodness-of-fit• Objective– Bringing the predictive strength of Neural
Networks, a powerful machine learning-based
technique, to the field of Discrete Choice Models(DCM) without compromising interpretability ofthese choice models.

RUM: random utility maximization

22

RUM model and neural network (NN)
Discrete choice model as a Random Utility Maximization (RUM) model

Utility function:

Choice probability:(negative) log-likelihood:
A discrete choice model from the perspective of neural network

Softmax activation function:
Cross-entropy:
The conventional MNL can be seen as a neural network model with asimple network structure.
23

Discrete Choice Model with NN
Utility function with non-linear component:

Non-linear
component(via NN)

Linear-inparameterscomponent:: A vector of parameters (1×d): A set of explanatory variables (I×d)

whereis the ensemble of input features, and, 𝜓 isthe function defined by multiple neural network layers
and their corresponding activation functions.

24Discrete Choice Model with NN
Non-linear
component

Linear-in-parameterscomponent

Hold a monotonicity
Constraint.
25

Empirical analysis• Dataset– Swissmetro dataset (Bierlaire et al., 2001)– A stated preference data on mode choice– 10700 entries from 1190 individuals• Linear-in-parameters component:

26

Empirical analysis•

Non-linear component:
1.
2.
3.
4.
5.
6.
7.
8.

Travel purpose: Discrete value between 1 to 9 (Business, leisure,travel,... )First class: 0 for no or 1 for yes if passenger is a first classtraveler in public transport
Ticket: Discrete value between 0 to 10 for the ticket type (Oneway, half-day, ...)
Who: Discrete value between 0 to 3 for who pays the travel (self,
employer, ...)
Male: Travelerʼs gender, 0 for female and 1 for male
Income: Discrete value between 0 to 4 concerning the travelerʼs
income per year
Origin: Discrete value defining the canton in which the travel
beginsDest: Discrete value defining the canton in which the travel ends

27

Multinomial Logit as Benchmark

28

Hybrid model (1)Note: Statistical properties of the parameters are obtained through Biogeme (Bierlaire, 2009)29

Simplified hybrid model (2)All remaining variables are used here

30

Conclusions & future works
Conclusions:• Combining the advantage of linear-in-parameters RUM model andthe advantage of neural network where highly non-linear impactsof explanatory variables
Future works:• The selection of hyper
parameters (it wouldchange the results)• Possibility of using themodel for long-term
demand forecasting(cross-validation may not
be enough)• Possibility of using
different NN components(e.g., convolutional NN,
recurrent NN, etc.)

Comparison of key parameters31References
•

•
•
•
•

•
•
•

Ben-Akiva, M. (1973) Structure of passenger travel demand models. Ph.D. Thesis,
Massachusetts Institute of Technology. Dept. of Civil and Environmental
Engineering (http://hdl.handle.net/1721.1/14790 ).
Bhat, C.R. (1995) A heteroscedastic extreme value model of intercity travel mode
choice. Transportation Research Part B 29, 471-483.
Bierlaire, M. (2002) The Network GEV model. Proceedings of the 2nd Swiss
Transportation Research Conference, Ascona, Switzerland.
Cardell, N.S., Dunbar, F.C. (1980) Measuring the societal impacts of automobile
downsizing. Transportation Research Part A: General 14, 423-434.
Castillo, E., Menendez, J.M., Jimenez, P., Rivas, A. (2008) Closed form expressions
for choice probabilities in the Weibull case. Transportation Research Part B 42,
373-380.
Chikaraishi, M., Nakayama, S. (2016) Discrete choice models with q-productrandom utilities, Transportation Research Part B.
Daly, A. (2001) Recursive nested EV model. ITS Working Paper 559, Institute forTransport Studies, University of Leeds.
Daly, A., Bierlaire, M. (2006) A general and operational representation ofGeneralised Extreme Value models. Transportation Research Part B:
Methodological 40, 285-305.

32

References
•
•

•
•

•
•
•
•

•

Fosgerau, M., McFadden, D., Bierlaire, M. (2013) Choice probability generating
functions. Journal of Choice Modelling 8, 1-18.
Fosgerau, M., Frejinger, E., Karlstrom, A., 2013. A link based network route choicemodel with unrestricted choice set. Transportation Research Part B:
Methodological 56, 70-80.
Hato, E. (2002) Behaviors in network, Infrastructure Planning Review, 19-1, 1327 (in Japanese).
Koppelman, F.S., Wen, C.-H. (2000) The paired combinatorial logit model:
properties, estimation and application. Transportation Research Part B:
Methodological 34, 75-89.
Li, B. (2011) The multinomial logit model revisited: A semi-parametric approachin discrete choice analysis. Transportation Research Part B 45, 461-473.
Luce, R. (1959) Individual Choice Beahviour. John Wiley, New York.
Mai, T., Fosgerau, M., Frejinger, E., 2015. A nested recursive logit model for route
choice analysis. Transportation Research Part B: Methodological 75, 100-112.
Mai, T., 2016. A method of integrating correlation structures for a generalized
recursive route choice model. Transportation Research Part B: Methodological 93,
Part A, 146-161.
Mattsson, L.-G., Weibull, J.W., Lindberg, P.O. (2014) Extreme values, invarianceand choice probabilities. Transportation Research Part B: Methodological 59, 8195.
33

References
•

•
•

•

•
•
•

•

McFadden, D., (1978) Modelling the choice of residential location, in: Karlqvist, A.,
Lundqvist, L., Snickars, F., Weibull, J. (Eds.), Spatial Interaction Theory and
Residential Location. North-Holland, Amsterdam.
McFadden, D. (1989) A Method of Simulated Moments for Estimation of Discrete
Response Models Without Numerical Integration. Econometrica 57, 995-1026.
Nakayama, S. (2013) q-generalized logit route choice and network equilibrium
model. Proceedings of the 20th International Symposium on Transportation andTraffic Theory (Poster Session).
Nakayama, S., Chikaraishi, M., 2015. Unified closed-form expression of logit andweibit and its extension to a transportation network equilibrium assignment.
Transportation Research Part B 81, 672-685.
Thurstone, L.L. (1927) A law of comparative judgment. Psychological Review 34,
273-286.
Train, K. (2009) Discrete Choice Methods with Simulation, 2nd Edition ed.
Cambridge University Press.
Vovsha, P. (1997) Cross-nested logit model: an application to mode choice in the
Tel-Aviv metropolitan area. Transportation Research Board, Presented at the 76th
Annual Meeting, Washington DC.
Wen, C.-H., Koppelman, F.S. (2001) The generalized nested logit model.
Transportation Research Part B 35, 627-641.
34