19th Behavioral Modelling Summer School
12‐15 September, 2020

Application of Machine Learningin Travel Behavior Analysis
Muhammad Awais Shafique, PhD
Assistant Professor
University of Central Punjab, Pakistan

What is Machine Learning?
“Field of study that gives computers the ability to learn
without being explicitly programmed.”
Arthur Samuel (1959)“A computer program is said to learn from experience E withrespect to some task T and some performance measure P, if
its performance on T, as measured by P, improves withexperience E.”
Tom Mitchell (1998)2

Types of Learning Algorithms• Supervised learning• Teach the computer and then let it use the learning to do thetask (labelled data).

• Unsupervised learning• Let the computer determine structure and patterns in thedata (unlabeled data).

• Semi‐supervised learning• Let the computer determine patterns and then teach thecomputer to do the intended task (unlabeled and labelleddata).

• Reinforcement learning• Let the computer determine the ideal behavior in anenvironment to maximize its performance.
3

Classification Algorithms• Support Vector Machines (SVM)• Decision Tree (DT)• Random Forest (RF)4Support Vector Machines• Among the best “off‐the‐shelf” supervised learningalgorithms.
• Off‐the‐shelf: A method that can be applied directly to data
without requiring a great deal of time‐consuming data
preprocessing or careful tuning of the learning procedure.
• SVM is a two‐class classifier which forms a separating
hyperplane.
• When a set of training data containing class labels is
supplied to SVM, it outputs an optimal hyperplane whichthen classifies new examples.

5Support Vector Machines
70605040302010

Margin00200

400600800

100012006Support Vector Machines• Map data into a feature space where they are linearly
separable.

From Presentations by Nello Cristianini and Jason Weston respectively 7

Support Vector Machines

Support Vector Machines Explained by Tristan Fletcher 8

Decision Trees• Decision trees repeatedly split the dataset in order to arrive
at a final outcome.
• The split is made into branch‐like segments and thesesegments progressively form an inverted tree, which
originates from the starting node called the root.
• The root is the only node in the tree which does not have anincoming segment.
• The tree terminates at the decision nodes, also known asleaves or terminals.
• All the other nodes present within the tree are calledinternals or test nodes.
9

Decision Trees• The variables or features associated with the data are used
to make each split.
• At each node, the variables are tested to determine themost suitable variable to make the split.
• This testing is repeated on reaching the next node andprogressively forms a tree.
• Each terminal node corresponds to a target class.

10

Example11

Random Forest• A bootstrap sample of the dataset is used to independently
grow individual trees, and the majority vote is taken toconclude the final prediction.
• In addition to using a randomly selected bootstrap sampleof the data for growing each tree in the forest, randomness
was introduced in the splitting of nodes.
• In standard trees, each node in the classification orregression tree is split using the best split among allvariables.
• In random forests, each node is split using the best among a
subset of the variables randomly selected at that node.
12

Random Forest• Suppose 𝑛 number of trees are grown. Each tree isgenerated by randomly selecting nearly 63% of the given
training data.
• The sample data is therefore different for each tree. The
remaining 37% data, known as out of bag (OOB) data, isused to estimate the error rate.
• The trees are fully grown without any requirement ofpruning, which is one of the advantages of random forest.

13

Random Forest• At each node a subset of variables or features is selected
and the most suitable feature among them is used for thesplit.
• The size of subset is a variable which is generally taken as𝑘 where 𝑘 is the total number of features.
• Once the forest is grown by using the labelled training
dataset, the test data is introduced for the prediction.
• The individual predictions by the trees are aggregated toconclude the final prediction result (i.e. majority vote forclassification and average for regression).

14

15

Discussion Paper• Zhao, X., Yan, X., Yu, A., & Van Hentenryck, P. (2020).
Prediction and behavioral analysis of travel mode choice: A
comparison of machine learning and logit models. Travel
behaviour and society, 20, 22‐35.

16Introduction• For travel‐behavior analysis, we mostly use discrete choicemodelling.
• Particularly the use of models belonging to logit family isquite popular such as Multinomial Logit Model, Nested Logit
Model and Mixed Logit Model.
• With tremendous increase in popularity of machinelearning, it is finding its way into travel‐behavior research aswell.

17Model Development• Logit models are based on calculating utilities (deterministicand error term) for available alternatives, and selecting thealternative which gives maximum utility.
• They have a defined layered structure.
• In contrast, machine learning methods learn from the data
to form a target function that maps the input variables tothe target (alternative selected).
• The structure is quite flexible, and may follow layeredstructure, tree‐based structure, rule‐based structure etc.

18Model Evaluation• Goal: Reduce overall prediction error (bias + variance +irreducible error)• Bias: Error due to incorrect assumptions• Variance: Error due to model sensitivity• Irreducible error: Error due to data noise• Bias‐variance tradeoff → underfitting vs overfitting

19

Data Collection• Stated‐preference survey conduced in University ofMichigan.
• 8,141 observations collected from 1,163 individuals.
• Respondents were asked to estimate trip attributes.
• Questions were then asked about a new public transitsystem.
• Variables included trip attributes, socio‐demographic
aspects, residential preferences, and selected mode choices.

20

Data Collection21Models Examined• Multinomial Logit Model• Mixed Logit Model• Naïve Bayes (NB)
• Classification and Regression Trees (CART)• Random Forest (RF)• Boosting Trees (BOOST)• Bagging Trees (BAG)• Support Vector Machines (SVM)• Neural Networks (NN)22

Result Comparison

23

Result Comparison

24Model Interpretation

25

Model Interpretation
RF has relative flat tails
before 10 min and after
25 min, showing people
tend to become
insensitive to very shortor very long transit
times

26

Model Interpretation

Travelers are moresensitive to wait
times less than 5 min27

Model Interpretation

The choice probabilityof PT decreases moresignificantly from 0 to
1 rideshare comparedto from 1 to 2rideshares.

28Model Interpretation

29

Model Interpretation• Consequently, different coefficients for a variable indifferent data intervals were specified for the logit models(MNL and mixed logit).
• The model fit improved and the coefficient estimates largely
agreed with the nonlinearies revealed by the RF model.
• Thus, ML Models can automatically learn the nonlinearities.
• Can assist in improving Logit Models.

30

Model Interpretation31Model Interpretation
Penalty ofTransfer
Rideshare stop
Wait time

MNL
5.5 min
4.2 min
1.5 min

Mixed Logit
13.4 min
8.9 min
3.4 minNN
2.6 min
1.0 min0.2 minRF
3.1 min
2.1 min
0.7 minAccording to literature• Penalty effects of a transfer > 5 min of in‐vehicle travel time
• Value of wait time is slightly larger than that of in‐vehicle travel time32Model Interpretation• The behavioral outputs of the logit models appear to be
more reasonable than those of RF.
• Another possibility: the results of RF are closer to groundtruth.
• In‐vehicle time vs total time
• Proposed mobility‐on‐demand system is described as an
app‐based system that provides accurate real‐timeinformation. This may lead to smaller penalties.

33

Conclusion• Random Forest significantly outperforms the logit models in
prediction both at the individual level and the aggregate
level.
• Mixed logit model underperforms the MNL.
• Machine‐learning and logit models largely agree on variable
importance and the direction of influence that eachindependent variable has on the choice outcome.
• Partial dependence plots show that machine‐learningmodels can readily capture nonlinear associations.
• RF’s behavioral results are somewhat inconsistent with theexisting literature.
34

awais.shafique@ucp.edu.pk
facebook.com/m.awaisshafiqueJp.linkedin.com/in/awaisshafique/www.researchgate.net/profile/Muhammad_Shafique19
35