THE USEFULNESS OF THE SEQUENCE ALIGNMENT METHODS IN 2 VALIDATING RULE-BASED ACTIVITY-BASED MODELS 3 4

1 2 The aim of this paper is to achieve a better understanding of rule-based activity-based 3 models, by proposing a new level of validation on the process model level in the 4 ALBATROSS model. To that effect, the work activity process model, which includes six 5 different decision steps, is investigated. Each decision step is evaluated during the prediction 6 of individuals’ schedules. The comportment of execution in the process model contains 7 activation dependency. This branches the execution and evaluation of each agent under 8 examination. And yields a sequence of decisions for each agent, where the Sequence 9 Alignment Method (SAM) is employed to evaluate how similar/dissimilar predicted with 10 observed decision sequences are. SAM utterly fits for assessing the analysis of decision 11 sequences on this level. The original CHAID decision trees at each decision step utilized in 12 ALBATROSS are compared with other well known induction methods chosen to appraise 13 the purpose of the analyses. Additionally, the performance of the models is compared at three 14 existing validation levels: the classifier or decision step level using confusion matrix 15 statistics. The work activity trips Origin-Destination (OD) matrix level and time of day work 16 activity start time level, using the correlation coefficient. The results of validation on the 17 proposed process model level show conformance to those already existing, with additional 18 information to help in better understanding the process model’s behaviour. 19 20 INTRODUCTION 21 22 In the past few decades, many studies have been conducted in order to try to understand the 23 nature of travel demand. Travel demand is derived from the human needs to participate in 24 activities that are distributed in time and space. Models that simulate travel demand using an 25 activity-based approach have been gaining growing attention in recent times due to their 26 strong behavioral foundation and insightful theoretical demand. Recognizing that travel is a 27 demand derived from individuals' needs to perform activities, researchers in travel demand 28 modeling have become increasingly interested in analyzing and predicting individuals' 29 decisions about activity participation. Activity-scheduling models share the objective to 30 predict the sequence of decisions that leads to an observed activity pattern of 31 households/individuals. Activity-based models aim at predicting on a daily basis and for 32 individuals which activities are conducted, by whom, for how long, at what time, the 33 location, and which transport mode is used when traveling is involved (1). The data 34 requirements for activity-based models are in general demanding compared to conventional 35 travel demand models. This is obvious specially that this type of micro-simulation models 36 should be able to predict the travel behaviour in detail including how the activities are 37 selected and scheduled. And so the validation of behavioural models becomes a difficult task. 38 Rule-based activity-based models are no exception as the validation process can be 39 performed on several levels hence, validating the model on an additional or new level may 40 incur extra knowledge to further calibrate and improve its performance. An existing and fully 41 operational rule based activity based model is the ALBATROSS model (1); it is a 42 computational process model, where schedules are predicted using CHAID based induction 43 tree method. 44 The validation of the ALBATROSS model is performed on many different levels, in 45 their original work (1) considered model performance on three levels: (i) the choice facet or 46 TRB 2012 Annual Meeting Paper revised from original submittal. Sammour, Bellemans, Vanhoof, Janssens and Wets 3 the decision tree induction level, by measuring the predictive accuracy of each decision rule 1 in the scheduling process. (ii) At the activity pattern level, sequence alignment methods are 2 used to assess the correspondence between the observed and predicted activity sequences (2). 3 (iii) At the trip matrix level, using correlation coefficients calculated to measure the degree of 4 correspondence between the observed and the predicted Origin-Destination matrices. 5 Decision trees derived from survey data may become large, complex and difficult to 6 interpret. In several experimental and analytical studies using the ALBATORSS model, 7 examples as in (4) and (5) performed validation on three levels, choice facet, activity pattern 8 and trip matrix levels. 9 The objective of this study is to investigate and assess the performance and predictive 10 behaviour of activity-based models on the decision process level of rule based activity based 11 models. The process model level is a core component of the scheduler engine in 12 ALBATROSS, which may reveal extra information on the model. And consequently assess 13 in more understanding the effect of using a specific induction method and in return improve 14 model performance. By further analyzing the process model the sequence alignment method 15 (SAM) was selected to measure how similar predicted to observed decision sequences are. 16 To this end, this work attempts to prove that evaluating activity-based models on this new 17 level expose information helps in additional understanding of the model. 18 The remaining part of this paper is organized as follows, in the next section the 19 ALBATROSS model and the FEATHERS framework used to implement the model for 20 Flanders are described, followed by a discussion of the diary data used for training the 21 model. The analyses and the process model are further discussed explaining the induction 22 methods and elaborating on the usefulness of adapting SAM in process models. Then 23 experiments design and discussion of results are discussed, followed by the conclusion and 24 future works. 25 26 THE FEATHERS / ALBATROSS SYSTEM 27 28 ALBATROSS is a fully operational rule based activity-based model that incorporates 29 household-level decision making (1) (4). In ALBATROSS, rules are used to predict activity30 travel choices of individuals and households. The decision rules are formalized from the 31 training of decision trees by using a CHAID decision tree induction method on surveyed 32 activity-travel diary data. In ALBATROSS, to generate a schedule for each person for each 33 day a sequential decision process is assumed, in which the rules are derived from 26 decision 34 trees, and the activity scheduling process model consists of four components or sub models 35 (14). The first component is responsible for generating primary work activities and their start 36 time, duration of each work episode if more than one episode is predicted, and their location, 37 and finally the transport mode for the work trip. The second component is used to generate 38 secondary fixed activities, usually work-related such as bring/get, business or other 39 mandatory activities. In addition it decides which type of activities performed, the number of 40 episodes for each activity, and their start time and duration. The third component is similar to 41 the second component, except it determines the flexible activities part of the schedule. The 42 fourth and last component is in charge of predicting the transport mode of secondary fixed 43 and flexible activities, as the transport mode of the primary work is already decided by the 44 first component. It is important to note that in ALBATROSS the activity travel behavior of 45 the two heads only is captured. A full account of the Albatross model system is given in (1). 46 TRB 2012 Annual Meeting Paper revised from original submittal. Sammour, Bellemans, Vanhoof, Janssens and Wets 4 The analysis performed in this work is performed on the first component dealing with work 1 activity scheduling excluding the transport mode decision step. Figure 1 depicts the work 2 activity decision process model used in ALBATROSS. Each numbered rectangle refers to a 3 decision tree model derived from activity diary data. The index j used in the figure refers to 4 the number of work episodes, if more than one work activity episode is predicted. 5 The first decision step evaluates whether the individual’s schedule contains a work 6 activity, if so, the duration of the work activity is predicted next. Followed by the number of 7 work activity episodes, subsequently the ratio between work episodes and the break time 8 duration is decided. And finally the work activity start time is predicted. Decision steps 1 and 9 3 are discrete choice decisions, whereas, decision steps 2, 4, 5 and 6 are continuous choice 10 decisions. It is noteworthy that if decision step 1 infers no work episode for the individual 11 under consideration then decision steps 2-6 will not be executed. Similarly, if decision step 3 12 evaluates to not including a second work episode, then decision steps 4 and 5 will not be 13 evaluated. This implies that there is an activation dependency in the execution of this process 14 model. 15 The analysis in this work is developed within the FEATHERS (Forecasting Evolutionary 16 Activity-Travel of Households and their Environmental RepercussionS) framework (6). The 17 FEATHERS framework is developed to facilitate the development of modular activity-based 18 models for transportation demand in Flanders (Belgium). The scheduling engine that is 19 currently implemented in the FEATHERS framework is based on the scheduling model that 20 is present in the ALBATROSS system (1). The framework is fully operational at the level of 21 Flanders. The scheduling is based on CHAID decision trees (8), trained based on the 22 Onderzoek VerplaatsingsGedrag Vlaanderen (OVG) travel survey data. The modular design 23 of FEATHERS allows for ease of adaptation of classification methods other than CHAID 24 decision trees, such as Bayesian networks (9), simple classifiers (5), and association rules 25 (10). Taking the