Trajeglish: Traffic Modeling as Next-Token Prediction
Authors: Jonah Philion
Xue Bin Peng
Sanja Fidler
Venue: ICLR 2024 poster (2024)
Keywords: self-driving
traffic modeling
autonomous vehicles
simulation
motion prediction
transformer
Primary area: applications to robotics, autonomy, planning
TL;DR: An LLM-like architecture for modeling motion of vehicles, pedestrians, and cyclists for AV simulation

Abstract: A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. In pursuit of this functionality, we apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Using a simple data-driven tokenization scheme, we discretize trajectories to centimeter-level resolution using a small vocabulary. We then model the multi-agent sequence of discrete motion tokens with a GPT-like encoder-decoder that is autoregressive in time and takes into account intra-timestep interaction between agents. Scenarios sampled from our model exhibit state-of-the-art realism; our model tops the Waymo Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%. We ablate our modeling choices in full autonomy and partial autonomy settings, and show that the representations learned by our model can quickly be adapted to improve performance on nuScenes. We additionally evaluate the scalability of our model with respect to parameter count and dataset size, and use density estimates from our model to quantify the saliency of context length and intra-timestep interaction for the traffic modeling task.

Preprint
TRAJEGLISH: TRAFFIC MODELING AS NEXT-TOKEN PREDICTION
Jonah Philion Xue Bin Peng Sanja F.d NVIDIA, University of Toronto, Vector Institute, 'Simon Fraser University (iphilion, japeng, sfidler) @nvidia. com
ABSTRACT
A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. In pursuit of this function- ality, we apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Using simple data-driven tokenization scheme, we discretize trajectories to centimeter-level resolution using a small vocabulary. We then model the multi-agent sequence of discrete motion tokens with a GPT-like mecoder-decoder that is autoregressive in time and takes into account antra-timestep interaction between agents. Scenarios sampled from our model exhibit state-of-the-ar realism; our model tops the Waymo Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%. We ablate our modeling choices in full au- tonomy and partial autonomy settings, and show that the representations learned by our model can quickly be adapted to improve performance on nuScenes. We additionally evaluate the scalability of our model with respect to parameter count and dataset size, and use density estimates from our model to quantify the saliency of context length and intra-timestep interaction for the traffic modeling task.
INTRODUCTION
In the short term, self-driving vehicles will be deployed on roadways that are largely populated by human drivers. For these early self-driving vehicles share the road safely, is imperative that they become fluent in the ways people interpret and respond to motion. A failure on the part of self-driving vehicle to predict the intentions of people can lead to overconfident or overly cautious planning. A failure on the part of self-driving vehicle to communicate to people its own intentions can endanger other road users by surprising them with uncommon maneuvers.
In this work, we propose an autoregressive model of the motion of road users that can be used to simulate how humans might react if self-driving system were to choose a given sequence of actions. At test time, as visualized in Fig. [ the model functions policy, outputting a categorical distribution over the set of possible states an agent might move to at each timestep. Iteratively sampling actions from the model results in diverse, length. We call our approach Trajeglish ("tra-JEG-lish") due to the fact that we model multi-agent trajectories as a sequence of discrete tokens, similar to the representation used in language modeling. and to make an analogy between how road users use vehicle motion to communicate and how people use verbal languages, like English, to communicate.
consiseentt multi-agent rollouts of arbitrary
A selection of samples from our model is visualized in Fig. 2 model is prompted with only the initial position and heading of the agents, in contrast to prior work that generally requires at least one second of historical motion to begin sampling. Our model gener- ates diverse outcomes for each scenario, while maintaining the consisensy of the trajectories. We encourage readers to consult our project page for videos of scenarios sampled from our model in full control and partial control settings, as well as longer rollouts of length 20 seconds.
When generating these samples, the
Our main contributions are:
A simple data-driven method for tokenizing trajectory data call "k-disks" that enables us to tokenize the Waymo Open Dataset (WOMD) (Ettinger et al., (00T]) at an expected discretization error of cm using small vocabulary size of 384.

---

Preprint
Inputs and outputs At given timestep, our model predicts a distribution over a fixed set of V states defined relative an agent's current location and heading, and condi- tions on map information, actions from all previ- ous timesteps (green), and any actions that have already been chosen by other agents within the current timestep (blue). We model motion of all agents relevant to driving scenarios, includ- ing vehicles, pedestrians, and cyclists.
Figure 1:
transformer-bas -ased architecture for modeling sequences of motion tokens that conditions on map information and one or more initial states per agent. Our model outputs a distribu- tion over actions for agents one at a time which we show is ideal for interactive applications.
State-of-the-Be quantitative and qualitative results when sampling rollouts given real-world initializations both when the traffic model controls all agents in the scene as well as when the model must interact with agents outside its control.
We additionally evaluate the scalability of our model with respect to parameter count and dataset size, visualize the representations learned by our model, and use density estimates from model to quantify the extent to which intra- ttra-timestep dependence exists between agents, well as to measure the relative importance of long context lengths for traffic modeling (see Sec.
1.1 RELATED WORK
Our work builds heavily on recent work in imitative traffic modeling. The full family of generative models have been applied to this problem, including VAEs (Suo al. 2021 Rempe et al., 2021). GANs (Igl et al. 2022), and diffusion models (Zhong et al., 2022 Jiang et al.) (023). While these approaches primarily focus on modeling the multi-agent joint distribution over future trajectories, our focus in this work is additionally building reactivity into the generative model, for which the factorization provided by autoregression is well-suited. For the structure of our encoder-decoder, we draw inspiration from Scene Transformer (Ngiam et al., 2021) which also uses global coordinate frame to encode multi-agent interaction, but does not tokenize data and instead trains their model with a masked regression strategy. A limitation of regression that it's unclear if the Gaussian or Laplace mixture distribution is flexible enough to represent the distribution over the next state, whereas with tokenization, we know that all scenarios in WOMD are within the scope our model, the only challenge is learning the correct logits. A comparison can also be made to the behavior cloning baselines used Symphony (Igl et al., 2022) and "Imitation Not Enough" (Lu et al., 2023) which also predict a categorical distribution over future states, except our models are trained directly on pre-tokenized trajectories as input, and through the use of the transformer decoder, each embedding receives supervision for predicting the next token well as all future tokens for all agents in the scene. In terms of tackling the problem of modeling complicated continuous distribu- tions by tokenizing and applying autoregression,
work is most similar Trajectory Transformer Jaanerr et al. 2021] which applies a fixed-gr grid tokenization strategy to model state state-action sequences for RI. Finally, work parallels MotionLM (Seff et al.) (202)) which is concurrent work that also uses discrete sequence modeling for motion prediction, but targets 1- and 2-agent online interaction prediction inistead of N-agent offline closed-loop simulation.
IMITATIVE TRAFFIC MODELING
In this section, show that the requirement that traffic models must interact with all agents each timestep of simulation, independent the method used to control each of the agents, imposes certain structural constraints how the multi-agent future trajectory distribution is factored by imitative traffic models. Similar motivation is provided to justify the conditions for submissions to the WOMD sim agents benchmark to be considered valid loop policies (Montali al. 2023).
We are given an initial scene with N agents, where a scene consists map information, the di- mensions and object class for each of the N agents, and the location and heading for each of the
2

---

Preprint
Figure 2: Trajeglish Visualizations of samples from our model. Rollouts within each row given the same single-timestep initialization, outlined in black. Future trajectories become lighter for timesteps farther into the future. Note that while some tracks overlap in the figure, they overlap when time taken into account; there are no collisions in these rollouts. Vidcos are available on our project page
agents for some number of timesteps the past. For convenience, we denote information about the scene provided at initialization by We denote the state of vehicle i at future timestep by sj = (x), yi. h() where (x, y) is the center of the agent's bounding box and h is the heading. For scenario of length T timesteps, the distribution of interest for traffic modeling is given
p(s)..... 8 8 8 8)
c).
We refer to samples from this distribution as rollouts. In traffic modeling. our goal to sample rollouts under the restriction that at each timestep, a black-box autonomous vehicle (AV) system chooses a state for a subset of the agents, We refer to the agents controlled by the traffic model as "non-player characters" or NPCs. This interaction model imposes the following factorization of the joint likelihood expressed in E
p(s 1 .... 8 N 22 ... ..... 8 .... 87 N
c)
I...No II p(s No. cc. s p(s, No+ 81....1 S
155T
NPCs
where 81...t-1 =\t {8], s7 (s!...., s/² is the set of states for agents through N time and arbitrarily assigned the agents out of the traffic model's control to have indices N.....Ng. The factorization in Eq, 2 shows that seek model from which sampled in previous timesteps as well as any states already sampled at the current timestep.
o is the set of all states for all agents prior to timestep 1, 8, s....No
sample agent's next state conditional on all states
We note that, although the real-world system that generated the driving data involves independent actors, it may still be important to model the influence of actions chosen by other agents at the same timestep, point we expand in Appendix A. While intra-timestep interaction between agents is weak in general, explicitly modeling this interaction provides a window into understanding cases when it is important consider for the purposes of traffic modeling.
METHOD
In this section, introduce Trajeglish, autoregressive generative model of dynamic driving scenarios. Trajeglish consists of two components. The first component is a strategy for discretizing, or "tokenizing" driving scenarios such that model exactly the conditional distributions required by the factorization the joint likelihood Eq. 2 The second component an autoregressive transformer- baseed architecture for modeling the distribution of tokenized scenarios.
Important features Trajeglish include that it preserves the dynamic factorization of the full like- lihood for dynamic test-time interaction, it accounts for intra-timestep coupling across agents, and

---

Preprint
[=0
[=1
t=2
Figure 3: Tokenization We iteratively find the token with minimum corner distance to the next state. An example trajectory is shown green. The raw representation the tokenized trajectory is shown as boxes with blue outlines. States that have yet be tokenized are light green. Token templates are optimized to minimize the error between the tokenized trajectories and the raw trajectories.
M
0.00
-0.25
-0.25
(m)
X (m]
0.25
0.00
0.00
0.25
0.25
3.0
(m)
[m)
Figure 4: Raw motion token representation We plot the raw representation of action sets extracted with k-disks for [V] E (128, 256, 384, 512}. Agents sample one of these actions at each timestep.
it enables both efficient sampling of scenarios well as density estimates. While sampling is the primary objective for traffic modeling, we show in Sec. (..3 the density estimates from Trajeglish useful for understanding the importance of longer context lengths and intra-tine intra-timestep dependence. We introduce our tokenization strategy in Sec. 3.1 and our autoregressive model Sec. 3.2
3.1 TOKENIZATION
The goal of tokenization is to model the support of continuous distribution as a set of [V| discrete options. Given æ R. p(æ), tokenizer is function that maps samples from the continuous distribution to one of the discrete options f R" + V. renderer is a function that maps the discrete options back to raw input r V -> R". A high-quality tokenizer-renderer pair is one such that r(f()) s æ. The continuous distributions that we seek to tokenize for the case of traffic modeling given by Eq. [ We note that these distributions are over single-agent states consisting of only a position and heading. Given the low dimensionality of the input data, we propose simple approach for tokenizing trajectories based fixed of state-to-state transitions.
Method Let Su be the state of agent with length and width the current timestep. Let be the state at the next timestep that we seek to tokenize. We define V = {s;} to be a set of template actions, each of which represents change in position and heading the coordinate frame of the most recent state. We use the notation 0; E N to indicate the index representation of token template
M
MM
Taken Freueeccy
0.04
Taken Frequency
0.04
Tikan Requency
0.04
Vehicle vol
Takan Requency
0.04
0.03
0.03
0.03
Pediestrian
0.00
0.02
0.02
0.02
Pedestrion train
0.02
0.01
0.01
Cyclist train
0.01
100
100 200
100
200
300
100
200 30o 400 soo Sorted Token
Sorted Taken
Sorted
Sorted
Figure 5: Token frequency We plot the frequency that each token appears in the validation and training sets. Note that sort the tokens by their frequency for each class individually for the ID. Increasing the vocabulary size increases the resolution but also results in a longer tail. The distribution of actions on the training set and validation set match closely.
4

---

Preprint
Vehicle
Pedestrian
Cyclit
0.25
0.25
0.25
0.20
6. 22 (m)
5:d% 2g km)
0.15
0.15
0.15
lemeens
Ic disks
0.10
0.05
0.05
0.05
0.00
0.0
1.5
7.5
9.0
3.0
Tejeectoy
Teajectory
Teajetory
Figure 6: K-means k-disks We plot the average discretization error for multiple template sets sampled from k-means and k-disks with [V] = 384. Alg. consistently samples better template sets than k-means.
8; and & to represent the raw representation of the tokenized state Our tokenizer f and renderer defined by
f(80, 8) a.- arg min di, w (8;, localllo. 8))
(3)
r r a.) = å global(so, 8:)
(4)
where d1, 81) is the average of the L2 distances between the ordered corners of the bounding boxes defined by so and $1, "local" converts to the local frame of su, and "global" converts st- to the global frame out of the local frame of 8o- We use (:.- throughout the rest of the paper to refer to this mean corner distance metric. Importantly, in order to tokenize full trajectory. this process of converting states 8 to their tokenized counterpart á is done iteratively along the trajectory, using tokenized states as the base state 80 in the next tokenization step. We visualize the procedure for tokenizing a trajectory in Fig. 3 Tokens generated with our approach have three convenient properties for the purposes of traffic modeling: they are invariant across coordinate frames, invariant under temporal shift, and they supply efficient access to measure of similarity between tokens, namely the distance between the raw representations. We discuss how to exploit the third property for data augmentation in Sec. A.2
Optimizing template sets We propose easily parallelizable approach for finding template sets with low discretization error. We collect large number of state transitions observed in data, sample one of them, filter transitions that are within meters, and repeat [V] times. Pseudocode for this algorithm is included in Alg. [ We call this method for sampling candidate templates "k-disks" given its similarity to k-means++, the standard algorithm for seeding the anchors k-means (Arthur) & Vassilvitskii 2007), as well as the Poisson disk sampling algorithm (Cook) 1986). We visualize the template sets found using k-disks with minimum discretization error in Fig. 4 We verify in Fig. Sitha the tokenized action distribution is similar on WOMD train and validation despite the fact that the templates optimized the training set. We show in Fig. 6 that the discretization error induced by templates sampled with k-disks is in general much better than that of k-means, across agent types. A comprehensive evaluation of k-disks in comparison to baselines is in Sec. A.3
3.2 MODELING
The second component of our method is an architecture for learning a distribution over the sequences of tokens output by the first. Our model follows an decoder structure very similar to those used for LLMs (Vaswani et al., 2017, Radford al. [019 Raffel et al.) 2019). A diagram of the model is shown in Fig. 7 Two important properties of our encoder are that it not equivariant to choice of global coordinate frame and it is not permutation equivariant to agent order. For the first property, randomizing the choice of coordinate frame during training is straightforward, and sharing a global coordinate frame enables shared processing and representation learning across agents. For the second property, permutation equivariance is not actually desirable in case since the agent order encodes the order in which agents select actions within a timestep; the ability of our model to predict actions should improve when the already-chosen actions of other agents are provided.
Encoder Our model takes input two modalities that encode the initial scene. The first is the ini- tial state of the agents in the scene which includes the length, width, initial position, initial heading, and object class. We apply single-layer MLP encode these values per-agent to embedding of size C. We then add a positional embedding that encodes the agent's 's order as well as agent identity across the action sequence. The second modality is the map. We use the WOMD representation of
5

---

Preprint
N-
full-attention encoder
causal-attention decoder
encoder
encoder
embed
00 0
1
0202
X-1
[=1
t=0 agent information
map objects
future agent states states
Figure 7: Trajeglish modeling We train encoder-decoder transformer that predicts the action token of an agent conditional on context such as previously selected action tokens, map information, and initial agent states. The diagram visualizes the forward pass of the network during training in which initial agent states and map objects are passed into the network, and the model is trained with standard LLM-like next-token prediction on the sequence of multi-agent action tokens, shown in the top right. The bolded components transformers.
map a collection of "map objects". where map object is variable-length polyline representing a lane, a sidewalk, or a crosswalk, for example. We apply a VectorNet encoder to encode the map to sequence of embeddings for most M map objects (Gao et al.] 2020). Note that although the model is not permutation equivariant to the agents, it is permutation invariant to the ordering of the
map objects. Similar to Wayformer (Nayakanti et al. 2022), we then apply layer of latent query attention that outputs a final encoding of the scene initiaizzatin.
Decoder Given the set of multi-agent future trajectories, we tokenize the trajectories and flatten using the same order used to apply positional embeddings to the t agent encoder to get sequence ayaf...ak. We then prepend a start token and pop the last token, and use an embedding table to encode the result. For timesteps for which agent's state wasn't observed in the data, set the embedding to zeros. We pass the full sequence through transformer with causal mask during a linear layer to decode distribution over the V template states and train to maximize the probability of the next token with cross-entropy loss. We tie the token embedding matrix the weight of the final linear layer, which we observed results in small improvements (Press & Wolf 2017). We leverage flash attention (Dao et al. 2022) which we find greatly speeds up training time, as documented in Sec. A.8
training. Finally,
We highlight that although the model is trained to predict the next token, it is incorrect to say that a given embedding for the motion token of a given agent only receives supervision signal for the task of predicting the next token. Since the embeddings for later tokens attend to the embeddings of earlier tokens, embedding at a given timestep receives signal for the task of predicting all future tokens across all agents.
EXPERIMENTS
We the Waymo Open Motion Dataset (WOMD) evaluate Trajeglish in full and partial control environments. We report results for rollouts produced by Trajeglish on the official WOMD Sim Agents Benchmark in Sec.
4.1 We then ablate design choices in simplified full and partial Finally, we analyze the representations learned by our model and the
control settings in Sec. 4.2
density estimates provides in Sec. The hyperparameters for each of the models that we train can be found in Sec. A.4
4.1 WOMD SIM AGENTS BENCHMARK
We test the sampling performance of our model using the WOMD Sim Agents Benchmark and re- port results in Tab. Submissions this benchmark are required to submit 32 rollouts of length

---

Preprint
Table 1: WOMD Sim Agents Test
Realism meta metric
Kinematic metrics f
Interactive metrics f
Map-based metrics
Method
minADE (m)
Constant Velocity
0.2380
0,0465
0.3372
0,3680
7.924
Wayformer (Identical)
0.4250
0.3120
0.4482
0.5620
2.498
MTR++
0,4697
0.3597
0,4929
0.6028
1.682
Wayformer (Diverse)
0,4720
0.3613
0.4935
0.6077
1.694
Joint-Multipahh+
0,4888
0,4073
0,4991
0.6018
2.052
MTR E
0,4911
0.4180
0,4905
0.6073
1.656
MVTA
0,5091
0,4175
0.5186
0.6374
1.870
MVTE*
0.5168
0.4202
0.5289
0.6486
1.677
Trajeglish
0.5339
0,4019
0,5811
0.6667
1.872
marginal w/o map
10.0
7.5
ADE (m)
marginal
no intra
ADE
5.0
5.0
trajeglish
trajeglish (last)
trajoglish
w/ reg.
0.0
0.0
trajeglish
1.0
2.0
3.0
4.0
6.0
1.0
2.0
3.0
4.0
w/ reg. (last)
Rollout Time
Rollout Time
Figure 8: Partial control ADE Left shows ADE for the vehicles selected for evaluation under partial control, but rollouts where agents are fully autonomous. Right shows the ADE for same vehicles but with all other agents on replay. When agents controlled by Trajeglish go first in the permutation order, they behave similarly to the intra model. When they go last, they utilize the intra-timestep information to produce interaction more similar to recorded logs, achieving lower ADE.
seconds at 10hz per scenario, each of which contains to 128 agents. We bold multiple submis- sions if they are within 1% each other, as in Montali et al. (2023). Trajeglish
top submission along the leaderboard meta metric, outperforming several well-established motion prediction models including Wayformer, MultiPath+++, and MTR (Shi et al. 2022; 2023). while being the first submis- sion to use discrete sequence modeling. Most of the improvement is due to the fact that Trajeglish models interaction between agents significantly better than prior work, increasing the state-of-the art along interaction metrics by 9.9%. full description of how sample from the model for this benchmark with comparisons on the WOMD validation set is included in Appendix
A.5 A.5
4.2 ABLATION
To simplify our ablation study, we test models this section on the scenarios they train on, of at most 24 agents and 6.4 seconds in length. We compare performance across 5 variants of our model. Both "trajeglish" and "trajeglish w/ reg." refer to our model, the latter using the noisy tokenization strategy discussed in See. A.2 The "no intra" model is important baseline designed to mimic the behavior of behavior cloning baselines used in Symphony (Igl et al.] (022) and "Imitation Is keep the same architecture but adjust the masking strategy in the decoder to not attend to actions already chosen for the current timestep. The "marginal" baseline designed mimic the behavior of models such as Wayformer (Nayakanti et al. 2022) and MultiPath++ (Varadarajan et al. 2021) that are trained to model the distribution over single-agent trajectories instead of multi agett ceene-cnssiseent trajectories. For this baseline, we keep the same architecture but apply a mask to the decoder that enforces that the model can only attend to previous actions chosen by the current agent. Our final baseline is the same as marginal baseline but without a map encoder. We use this baseline to understand the extent to which the models rely on the map for traffic modeling.
Not Enough" (Lu et al. 2023). For this baseline,
We report results in Fig. 8 a partial controllability setting in which a single agent in each scenario is chosen to be controlled by the traffic model and all other agents set replay. The single-agent ADE (average distance error) for the controlled-agee -ggent is similar in full autonomy rollouts for all models other than the model that does not condition the map, expected. How-
Partial control

---

Preprint
0.0s Initialization Length
0.3s initialization Length
0.6s Initiali zation Length
0.032
wee map
0.010
0.010
0.030
trajeglish
0.008
0.008
0.008
trajeglish
w/ reg.
0.006
0.006
0.006
GT
1.0
2.0
1.0
1.0
0.275
0.275
0.275
manginal web map
0.250
0.250
0.250
marginal
0.225
0.225
0.225
trajeglish
trajeglish w/ reg.
0.200
0.200
0.200
0.175
0.175
0.175
GT
1.0
2.0
3.0
1.0
2.0
6.0
Rollout
(6)
Figure 9: Full Autonomy Collision Rate Vchicle collision rate shown on top and pedestrian collision rate is shown on bottom. From left to right, we seed the scene with an increasing number of initial actions from the recorded data. Trajeglish models the log data statistics significantly better than baselines when seeded with only initial timestep, well with longer initialization.
Rolaive Nequive Lap-tteiihodd
1.04
Vehicle
Relatvve Nogaove Log Lielhrod
1.04
Rellfive Leg-Litnilltnd
1.04
Cyclist
1.03
1.03
1.03
seconds
0.2 seconds
1.02
1.02
0.02
0.3 seconds
1.01
1.01
1.02
seconds
1.00
1.00
0
10
15
20
5
10
15
10
15
20
Agent Permul ation
Agent Permitation Index
Agent Permutation Indeex
Figure 10: Intra-Timester Conditioning We plot the negative log- likelihood (NLL) when vary how many agents chaose an action before given agent within given timestep. As expected, when the context length increases, intra-timestep interaction becomes much less important take into account.
ever, in rollouts where all other agents are placed on replay, the replay trajectories leak information about the trajectory that the controlled-agem took the data, and result, the no-intra and tra- jeglish rollouts have lower ADE. Additionally, the Trajeglish rollouts in which the controlled agent placed first do condition on intra-timenn titra-timestep information and therefore behave identically the no-intra baseline, whereas rollouts where the controlled-agen is placed last in the order provide the model with more information about the replay trajectories and result decreased ADE.
Full control We evaluate the collision rate of models under full control in Fig. 9 function of initial context, object category, and rollout duration. The value of modeling intra-timestep interac- tion is most obvious when only a single timestep is used seed generation, although intra-timestep modeling significantly improves the collision rate in all cases for vehicles, For interaction between pedestrians, Trajeglish is able capture the grouping behavior effectively. We observe that nois- ing the tokens during training improves rollout performance slightly in the full control setting. We expect these rates to improve quickly given more training data, as suggested by Fig.
4.3 ANALYSIS
understand the extent to which our model leverages intra- timestep dependence, in [0] we evaluate the negative log likelihood under our model of predict- ing an agent's next action depending on the agent's order the selected permutation, as a function of the amount of historical context the model is provided. In all cases, the agent gains predictive power from conditioning on the actions selected by other agents within the same timestep, but the log likelihood levels out more historical context is provided. Intra-timestep dependence is sig- nificantly less important when provided over timesteps of history, as is the setting used for most motion prediction benchmarks.
Intra-Timeser Dependence
8

---

Preprint
Validation Loss (Vehicles)
1.4
15.4M parameters
scratch
0.58
35 6.. parameters
Validation Loss
LoRA
fine-tuning
0.56
0.54
0.52
107
10a
109
10000 20000 Training Steps
30000
40000
Dataset Size (Tokens)
Figure 11: Scaling Behavior Our preliminary study on parameter and dataset scaling suggests that, com- pared LLMs (Kaplan et al.] (2020), Trajeglish is severely data-constrained on WOMD; models with 35M parameters just start to be significantly better than models with ISM parameters for datasets the size WOMD. more rigorous study of how hyperparameters of the training strategy affect sam- pling performance is reserved for future work.
Figure 12: nuScemes transfer We test the ability of model to transfer the maps and scenario ini- tializations the nuScenes dataset. The difference between maps and hchaviors found in the nuScenes datasct such that LoRA does not provide enough expressiveness fine-tune the model to peak perfor- mance. The fine-tuned models both outperform and train faster than the model that is trained exclusively on nuScenes.
(m))
0.0
0.0
0.5
2.0
2.5
3.0
Figure 13: Token Embedding Visualization We run PCA on the model embeddings at initialization and at convergence, and plot the (x. y) location of cach of the token templates using the top 3 principal component values determine the huc, lightness, and saturation of the point. The model learns that tokens that correspond actions close together euclidean space represent semantically similar actions. Note that the heading of each action not visualized, which also affects action similarity, especially at low speeds. Additionally, the top principal components include only 35% of the variance, explaining why some colors repeat.
Representation Transferability We measure the generalization of our model the nuScenes dataset (Caesar et al. 2019). As recorded in Sec. A.8 nuScenes is 3 orders of magnitude smaller than WOMD. Additionally, nuScenes includes scenes from Singapore where the lane convention opposite that of North America where WOMD is collected. Nevertheless, we show in Fig. 12 that our model can be fine-tuned to a validation NLL far lower than a model trained from scratch on only the nuScenes dataset. At the same time, we find that LoRA (Hu al. 202T) does not provide enough expressiveness to achieve the same NLL as fine tuning the full model. While bounding boxes have a fairly canonical definition, note that there multiple arbitrary choices in the definition of map objects that may inhibit transfer of traffic models to different datasets.
Token Embeddings We visualize the embeddings that the model learns in Fig. 13 Through the task of predicting the next token, the model learns similarity matrix across tokens that reflects the Euclidean distance between the raw actions that the tokens represent.
Preliminary Scaling Law We perform a preliminary study of how our model scales with increased parameter count and dataset size in Fig. IT] We find that performance between model of 15.4M parameters and 35.6 parameters is equivalent up 0.5B tokens, suggesting that a huge amount of performance gain is expected if the dataset size can be expanded beyond the IB tokens in WOMD. We reserve more extensive studies of model scaling for future work.
5 CONCLUSION
In this work, introduce discrete autoregressive model of the interaction between road users. By improving the realism of self-driving simulators, we hope to enhance the safety of self-driving systems they are increasingly deployed into the real world.

---

Preprint
REFERENCES
David Arthur and Sergei Vassilvitskii. K-means++: The advantages of careful seeding. In Proceed- ings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '07, pp. 1027-1035, USA, 2007. Society for Industrial and Applied Mathematics. ISBN 9780898716245.
Holger Caesar. Varun Bankiti, Alex H. Lang. Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: multimodal dataset for autonomous driving. CoRR, Abs/1903.11007,, 2019.
11C27
Robert L. Cook. Stochastic sampling in computer graphics. ACM Trans, Graph., 5(1):51-72, jan 1986. ISSN 0730-0301. doi: 10.1145/7529.89277 URL https::/ http:///ccooorggoo./o 1145/ 7529. 8927
Tri Dao, Daniel Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness, 2022.
Scott Ettinger, Shuyang Cheng, Benjamin Caine, Chenxi Liu, Hang Zhao, Sabeek Pradhan, Yuning Chai, Ben Sapp, Charles R. Qi, Yin Zhou, Zoey Yang. Aurélien Chouard, Pei Sun, Jiquan Ngiam, Vijay Vasudevan, Alexander McCauley, Jonathon Shlens, and Dragomir Anguelov. Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset. In Proceedings of the IFEF/CVF International Conference on Computer Vision (ICCV), pp. 9710- 9719, October 2021.
Jiyang Gao, Chen Sun, Hang Zhao, Yi Shen, Dragomir Anguelov, Congcong Li, and Cordelia Schmid. Vectornet: Encoding hd maps and agent dynamics from vectorized representation, 2020.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In International Conference on Learning Representations, 2020. URL https: //openreview. hwt//ffruutiteee wtta://oouuenorrrrooyrrror
Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shot- ton, and Gianluca Corrado. Gaia-1: generative world model for autonomous driving, 2023.
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li. Shean Wang. and Weizhu Chen. Lora: Low-rank adaptation of large language models. CoRR, abs/2106.09685,
2021. URL httLtttps///arrttt wwwp:iii.orr.aaooooo abs/ 2106. 09685
Maximilian Igl, Daewoo Kim, Alex Kuefler, Paul Mougin, Punit Shah, Kyriacos Shiarlis, Dragomir Anguelov, Mark Palatucei, Brandyn White, and Shimon Whiteson. Symphony: Learning realistic and diverse agents for autonomous driving simulation, 2022.
Michael Janner, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big sequence modeling problem. In Advances in Neural Information Processing Systems, 2021.
Chiyu Max Jiang, Andre Cormman, Cheolho Park, Ben Sapp. Yin Zhou, and Dragomir Anguelov. Motiondiffuser: Controllable multi-agent motion prediction using diffusion, 2023.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. CoRR, abs/2001.0836/ 2020. URL htths:/:: /ttp::////axvvv.a/ org/ abs 20000000000000000000.
Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam. CoRR, abs/1711.00 1.05101, 2017. URL httt:://aax:
05101
Yiren Lu, Justin Fu, George Tucker, Xinlei Pan, Bronstein, Rebecca Roelofs, Benjamin Sapp, Brandyn White, Aleksandra Faust, Shimon Whiteson, Dragomir Anguelov, and Sergey Levine. Imitation is not enough: Robustifying imitation with reinforcement learning for challenging driv- ing scenarios, 2023.
Nico Montali, John Lambert, Paul Mougin, Alex Kuefler, Nick Rhinehart, Michelle Li, Cole Gulino, Tristan Emrich. Zoey Yang, Shimon Whiteson, Brandyn White, and Dragomir Anguelov. The waymo open sim agents challenge, 2023.
10

---

Preprint
Nigamaa Nayakanti, Rami AI-Rfou, Aurick Zhou, Kratarth Goel, Khaled S. Refaat, and Benjamin Sapp. Wayformer: Motion forecasting via simple and efficient attention networks, 2022.
Jiquan Ngiam, Benjamin Caine, Vijay Vasudevan, Zhengdong Zhang, Hao-Tien Lewis Chiang, Jef- frey Ling, Rebecca Roelofs, Alex Bewley, Chenxi Liu, Ashish Venugopal, David Weiss, Ben- jamin Sapp. Zhifeng Chen, and Jonathon Shlens. Scene transformer: A unified multi-task model for behavior prediction and planning. CoRR, abs/2106.08417, 2021. URL https:
/aaaai..rrg/ab////22/... 08417
Jonah Philion. Fastdraw: Addressing the long tail of lane detection by adapting a sequential predic- 04354
tion network. CoRR, (tlbs/9995.44330 2019. URL htttttp://aaiii.oo///ab/////////
Ofir Press and Lior Wolf. Using the output embedding improve language models, 2017.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. CoRR, abs/ 19100.06883,,....11111111111 2019.
10683,
Marc' Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojeiech Zaremba. Sequence level train- ing with recurrent neural networks. In Yoshua Bengio and Yann LeCun (eds.), 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016. URL.ttp: httpttttp//axxii..rttt
abs 1511. 1atpp//111.16222222
Davis Rempe, Jonah Philion, Leonidas J. Guibas, Sanja Fidler, and Or Litany. Generating useful accident-prone driving scenarios via Iearned traffic prior. CoRR, abs/2112.050777 2021. URL
btttss://arxtnn org/ abs 2112. 05077
Stephane Ross and Drew Bagnell. Efficient reductions for imitation learning. In Yee Whye Teh and Mike Titterington (eds.), Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9 of Proceedings of Machine Learning Research, pp. 661-668, Chia Laguna Resort, Sardinia, Italy, 13-15 May 2010. PMLR. URL https: //proceedings mlr
html
Ari Seff, Brian Cera, Dian Chen, Mason Ng, Aurick Zhou, Nigamaa Nayakanti, Khaled S. Re- faat, Rami Al-Rfou, and Benjamin Sapp. Motionlm: Multi-agent motion forecasting language modeling, 2023.
Shaoshuai Shi, Li Jiang, Dengxin Dai, and Bernt Schiele. Motion transformer with global intention localization and local movement refinement. Advances in Neural Informution Processing Systems, 2022.
Shaoshuai Shi, Li Jiang, Dengxin Dai, and Bernt Schiele. Mir+:: Multi-agent motion prediction with symmetric scene modeling and guided intention querying. arXiv preprint (tti::2306.1777700 2023.
Simon Suo, Sebastian Regalado, Sergio Casas, and Raquel Urtasun. Trafficsim: Learning to simulate realistic multi-agent behaviors, 2021.
Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew W. Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. CoRR, Mtss///09.03999 2016. URLhtttp://axxi org/ abs///b////// 1609.000000 0ttp//116.039999 03499
Balakrishnan Varadarajan, Ahmed Hefny, Avikalp Srivastava, Khaled S. Refaat, Nigamaa Nayakanti, Andre Cormman, Kan Chen, Bertrand Douillard, Chi-Pang Lam, Dragomir Anguelov, and Benjamin Sapp. Multipath++:: Efficient information fusion and trajectory aggregation for be- havior prediction. CoRR, abs/2111.14973, 2021. URL httpt: http://aaaxxvv.oo///////1...
14973
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. CoRR, abs/1706.03762. 2017. URL htlp:htt:://aatt www.lvoorrgggoooo wwwpo///////oooo////// 03762

---

Preprint
Ziyuan Zhong, Davis Rempe, Danfei Xu, Yuxiao Chen, Sushant Veer, Tong Che, Baishakhi Ray, and Marco Pavone. Guided conditional diffusion for controllable traffic simulation, 2022.
Yukun Zhu, Ryan Kiros, Richard S. Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explana- tions by watching movies and reading books. CoRR, 506.06244, 2015. URL http:
//axxiv.or//aaal /aax....ce/abb//550000722
12

---

Preprint
A APPENDIX
A.1
INTERACTION
There are variety of reasons that intra-timestep dependence may exist in driving log data. To list few, driving logs recorded discrete timesteps and any interaction in the real world between timesteps gives the appearance of coordinated behavior in log data. Additionally, information that generally recorded in log data, such eye contact dependence. Finally, the fact that log data exists in 10-20 second chunks can result in intra- sirra-timestep dependence there were events before the start of the log data that result in coordination during the recorded scenario. These factors are in general weak, but may give rise to behavior in rare cases that is not possible to model without taking into account coordinatation across agents within single timestep.
signals, may lead intra-timestep
A.2 REGULARIZATION
Trajeglish is trained with teacher forcing, meaning that it is trained the tokenized representation of ground-truth trajectories. However, at test time, the model ingests its own actions. Given that the model does model the ground-truth distribution perfectly, there is inevitable mismatch between the training and test distributions that can lead to compounding errors (Ross & Bagnell 2010 Ranzato al. 2016 Philion 2019). We combat this effect by noising the imput tokens fed as input to the model. More concretely, when tokenizing the input trajectories. instead of choosing the token with minimum corner distance to the ground-truth state as stated in Eq. we sample the token from the distribution
s/ s)/opPhop))
(5)
meaning we treat the the distance between the ground-truth raw state and the templates logits of a categorical distribution with temperature a and apply nucleus sampling (Holtzman et al. 2020) to generate sequences of motion tokens. When
and Ptop 1, the approach recovers the tokenization strategy defined in Eq. 3 Intuitively, two tokens are equidistant from the ground- truth under the average corner distance metrie, this approach will sample one of the two tokens with equal probability during training. Note that we retain the minimum-distame template index as the ground-truth target
when noising the input sequence.
While this method of regularization does make the model more robust to errors in its samples at test time, it also adds noise to the observation of the states of other agents which can make the model less responsive to the motion of other agents at test time. As a result, we find that this approach primarily improves performance for the setting where all agents are controlled by the traffic model.
A.3 TOKENIZATION ANALYSIS
We compare our approach for tokenization against two grid-based tokenizers (van den Oord et al., 2016 (Seff et al. 2023 Janner et al. 2021), and one campling-based tokenizer. The details of these methods are below.
Vehicle
Pedestrian
Cyclist
Milli Sop
0.05
Adla
128 train
(m))
0.04
68% (m2
0.04
(ml
0.04
256 vol
0.03
0.03
0.02
0.02
394 train
512 vol
0.01
0.01
0.02
0.00
0.00
0.00
4.5
7.5
0.0
3.0
6.0
Trajectory Duration
Trajctory Duration
Trajectory Duration
Figure 14: K-disk expected discretization error Average corner distance for each of the k-disk vocabularies of sizes 128, 256, 384, and 512.
13

---

Preprint
Vehicle
Pedestrian
Cyclist
0.25
0.25
0:25
M| H|M. Www.WtWWWW-hhhteee th
oo (m)
(x
0.15
0.15
0.15
(s. (i.ybgrid
k-diaks
0.05
0.05
0.05
0.0
1.5
3.0
4.5
6.0
7.5
9.0
0.0
155
3,0
4.5
6.0
7.5
3.0
0.0
1.5
3.0
4,5
6.0
7.5
9.0
Tajectary Duration (6)
Tia Taajectoyy Duration 66)
Trajectory Duration
Figure 15: Tokenization method comparison Average corner distance for trajectories tokenized with a vocabulary of 384 with template sets derived using different methods.
Vehicle
Cyclist
120 vll
0.030
256 train
0.025
384 vol
0.020
0.007
512 train
0.0
1.5
3.0
4.5
6.0
7.5
9.0
0.0
1.5
3,0
4.5
6.0
7.5
9.0
0.0
1.5
3.0
4.5
6.0
7.5
9.0
Duration
Trajectory
Figure 16: Semantic Tokenization Performance We plot the probability that the bounding box of an agent has non-zero overlap with another agent in the scene for each timestep. The collision rate for the raw data shown in black.
(1r, y. -rrd We independently discretize change longitudinal and lateral position and change in heading, and treat the template set as the product of these three sets. For vocabulary sizes of 128/256/384/5 12 respectively, we use 6/7/8/9 values for æ and y, and 4/6/7/8 values for h. These values spaced evenly between 3-5.3,3.5) for x, (-0.2 m, 0.2 m) for y. and (-0.1, 0.1) rad for h.
(I, -)ridd We independently discretize change in only the location. We choose the heading for each template based on the heading of the state-to-state transition found in the data with change in location closest to the template location. Compared to the (xr, y, ht)-grdd baseline, this approach assumes heading is deterministic given location in order to gain resolution in location. We 12/16/20/23 values for æ and with the same bounds as in the (I, y, h) --rid baseline.
k-means
We run k-means many times on a dataset of (x,, y, h) state-to-state transitions. The dis-
tance metric is the distance between the (æt, y) locations. We note that the main source of randomness across runs is how k-means is seeded, for which we use k-means++ Arthur & Vassilvitskil (2007). We ultimately select the template set with minimum expected discretization error as measured by the average corner distance.
k-disks As shown in Alg. we sample subsets of a dataset of state-to-state transitions that are at least from each other. For vocab sizes of 128/256/384/512, we use of 3 centimeters.
Intuitively, the issue with both grid-based methods is that they distribute templates evenly instead of focusing them in regions of the support where the most state transitions occur. The main issue with k-means is that the heading is not taken into account when optimizing the cluster centers.
We offer several comparisons between these methods. Fig. 14 we plot the expected corner distance between trajectories and tokenized trajectories as a function of trajectory length for the template sets found with k-disks. In Fig. 15 we compare the tokenization error as a function of trajectory length and find that grid-based tokenizers create large oscillations. To calibrate to metric more relevant to the traffic modeling task, we compare the collision rate between raw trajectories as a function of trajectory length for the raw scenarios and the tokenized scenarios using k-disk template sets size 128, 256, 384, and 512 in Fig. 16 We observe that a vocabulary size of 384 is sufficient to avoid creating extraneous collisions. Finally, Fig. 17 plots the full distribution of discretization
14

---

Preprint
Method: k-disks Pedestrian
Vehicle
Cyclist
0.6
p1000
0.
0.2
0.2
1 cm
06 cm
E- ct
Trajetory
Trjjetory
Trajectoyy
Method: k-means Pedestrian
Vehicle
Cyclist
0.5
0.6
0.6
0.2
0.2
0.21
[-7 m
r-5 cm
5 m
1-30 ct
Trejetoryy
Pejectory
Trajetory
Method: yx.rid
Vehicle
Pedestrian
Cyclist
0.6
0.6
0.6-
plet Upd
r=8
0.2
p10h.
0.2
0.2
507
1 on
5- cm
g-30
3.0
Trajectoyy
Method: (x. y. h)-grid
Vehicle
Pedestrian
Cyclist
0.6
0.6
0.6-
02
0.
0.2
1-8 mn
5:9 c
0+ ct
Trjjecory
Trajectrry
Figure 17: Discretization error distribution We plot the probability that the discretized trajectory greater than cm S S 10 em away from the true trajectory as a function of trajectory length. Lower is therefore better. Each row visualizes the error distribution for different method, each with a vocabulary size of 384. We keep the y-axis the same across all plots. We note that k-means discretizes more trajectories to within 2 em than k-disks, but does not improve past whereas k-disks able to tokenize nearly all trajectories in WOMD to within centimeters.
errors for each of the baselines and Tab. reports the expected discretization across vocabulary sizes for each of the methods.
Algorithm 1 Samples a candidate vocabulary of size N. The distance d(zn, x) measures the average corner distance between a box of length meter and width meter with state To vs. state æ.
1: procedure SAMPLEKDISKSLA N, e)
2:
S ++f
3: while len(S) N do
4:
Pn X
5:
X < fr E d(xo, > o
6:
S S {
return 5
15

---

Preprint
Table 2: Tokenization discretization error comparison
E [[d(s, 8)] (cm)
method
V 128
[V|
256
[V]
384
[V] 512
(x, y, h)-g
20.50
16.84
14.09
12.59
(x, -gri
9.35
8.71
5.93
4.74
k-means
14.49
8.17
6.13
5.65
k-disks 2.66
1.46
1.18
1.02
A.4 TRAINING HYPERPARAMETERS
We train two variants of our model. The variant we use for the WOMD benchmark is trained on scenarios with up 24 agents within 60.0 meters of the origin, up to 96 map objects with map points within 100.0 meters of the origin, 2 map encoder layers, 2 transformer encoder layers, 6 transformer decoder layers, hidden dimension of 512, trained to predict 32 future timesteps for all agents. We train with a batch size of 96, with tokenization temperature of 0.008, tokenization nucleus of 0.95, a top learning rate of 5e-4 with 500 step warmup and linear decay over 800k optimization steps with AdamW W optimizer (Loshchilov & Hutter 2017). We use the k-disks tokenizer with vocabulary size 384. During training, we choose a random 4-second subsequence of a WOMD scenario, a random agent state to define the coordinate frame, and random order in which the agents are fed to the model.
For the models analyze in all other sections, predict 64 timesteps, using only 700k optimization steps. Training on these longer scenarios enables evaluate longer rollouts without the complexity of extending rollouts in a fair way across models, which we do only for the WOMD Sim Agents Benchmark and document in Sec. A.5
the same setting from above, but train to
A.5 EXTENDED ROLLOUTS FOR WOMD SIM AGENTS BENCHMARK
In order to sample scenarios for evaluation on the WOMD sim agents benchmark, we require the ability to sample scenarios with an arbitrary number of agents arbitrarily far from each other for an arbitrary number of future timesteps. While it may become possible to train high-performing model on 128-agent scenarios on larger datasets, we found that training our model on 24-agent scenarios and then sampling from the model using "sliding window" (Hu al. 2023) both spatially and temporally achieved top performance.
In detail, at given timestep during sampling, determine the 24-agent subsets with the following approach. First, we compute the 24-agent subset associated with picking each of the agents in the scene to be the center agent. We choose the subset associated with the agent labeled as the self- driving car to be the first chosen subset. Among the agents not included in a subset yet, we find which agent has a 24-agent subset associated to with the maximum number of agents already included in a chosen subset, and select that agent's subset next. We continue until all agents are included in at least one of the subsets.
Importantly, to define the order for agents within the subset, we place any padding at the front, fol- lowed by all agents that will have already selected an action at the current timestep. followed by the remaining agents sorted by distance to the center agent. In keeping this order, we enable the agents to condition on the maximum amount of pre-generated information possible. Additionally, this or- dering guarantees that the self-driving is always the first to select an action at each timestep, in accordance with the guidelines for the WOMD sim agents challenge (Montali et al. 2023).
To sample an arbitrarily long scenario, we have the option to sample up to < T 32 steps before computing new 24-agent subsets. Computing new subsets every timestep ensures that the agents within subset always close to each other, but has the computational downside of requiring the transformer decoder key-value cache to be flushed at each timestep. For our submission, we compute the subsets timesteps i (10,34, 58).
16

---

Preprint
Table 3: 2023 WOMD sim agents validation
Realism Meta metric f
Kinematic metrics
Interactive metrics
Map-based metrics f
Method
T 1.25, Ptop 0.995
0.5176
0.3960
0.5520
0.6532
1.5, Ptop 1.0
0.5312
0.3963
0.5838
0.6607
7 = 1.5, Ptop = 1.0, w/ h-smooth
0.5352
0,4065
0.5841
0.6612
Table 4: WOMD sim agents validation updated metrics
Realism Meta metric f
Kinematic metrics f
Interactive metrics
Map-based metrics 1
Method
Trajeglish (r = 1.5)
0.6078
0.4019
0.7274
0.7682
MTR_E
0.6348
0.4180
0.7416
0.8400
MVTA
0.6361
0.4175
0.7543
0,8253
Trajeglish (r 1.0)
0.6437
0.4157
0.7816
0,8213
MVTE
0.6448
0.4202
0.7666
0.8387
Trajeglish (r 1.0, AA=32)
0.6451
0.4166
0.7845
0,8216
While the performance of our model under the WOMD sim agents metrics was largely unaffected by the choice of the hyperparameters above, we found that the metries were sensitive the temperature and nucleus that we use when sampling from the model. We use temperature of 1.5 and a nucleus of 1.0 achieve the results in Tab. [ Our intuition for why larger temperatures resulted in larger values for the sim agents metric is that the log likelihood penalizes lack of coverage much more strongly than lack of calibration, and higher temperature greatly improves the coverage.
Finally, we observed that, although the performance of our model sampling with temperature 1.5 was better than all prior work for interaction and map-based metrics as reported Tab. 3 the per- formance worse than prior work along kinematics metrics. To test if this discrepancy byproduct of discretization, we trained a "heading smoother" by tokenizing trajectories, then train- ing small autoregressive transformer to predict back the original heading given the tokenized tra- jectory. On tokenized ground- -tuuh trajectories, the heading smoother improves heading error from 0.58 degrees 0.33 degrees. Note that the autoregressive design of the smoother ensures that does not violate the closed-loop requirement for the Sim Agents Benchmark. The addition of this smoother did improve along kinematics metries slightly, as reported in Tab. 3 We reserve more rigorous study of how to best improve the kinematic realism of trajectories sampled from discrete sequence models for future work.
A.6 DECEMBER 28, 2023 UPDATED SIM AGENTS METRICS
On December 28, 2023, Waymo announced an adjustment the metries for the Sim Agents bench- mark to improve accuracy of vehicle and off-road collision checking (more details about this adjust- ment can be found here). Upon re-optimizing hyperparameters of Trajeglish for the new metrics, we found that the optimal sampling hyperparameters were = 1.0 and Peop 1.0, which is more intuitive than our previously chosen hyperparameter of r 1.5 given that the metrics are intended to measure the extent to which the distribution of sampled scenarios and recorded scenarios match. We then re-trained our model to condition on 32 agents at a time instead 24 which also improved results slightly. For the final leaderboard results before the announcement of the 2024 Sim Agents Challenge. Trajeglish did end up ahead of all models it had beaten under the previous metrics, al- though by much slimmer margins, shown in Tab.
17

---

Preprint
Vehiclee
Pedestrian
Cyclist
2.0
0.9
marginal
minADE (m))
minADE (mi)
minaO (m))
2.5
0.8
marginal
0.7
2.0
trajeglish
intra
1.0
traieglish
05
0.5
0.0
0.2
0.4
0.6
0.0
0.2
0.4
0.6
0.0
0.2
0.4
0.6
Vitiakzttion Langth
Intiiaiiaaiinn Length
Iritaliztionn Langth (4)
Figure 18: Full Autonomy minADE As seed the scene with a longer initialization, the no-intra model and our model converge to similar values, and all models improve. When initialized with only a single timestep, the performance gap between models that take into account intra-timestep interaction and models that do not is significant.
A.7 ADDITIONAL ABLATION RESULTS
find the sampled scenario with minimum corner distance to the ground-truth scenario and plot that distance as a function the number of timesteps that are pro- vided initialization. When the initialization is single timestep. the minADE of both models that take into account intra-timestep dependence improves. As more timesteps are provided, the effect diminishes, as expected. We visualize small number of rollouts in the full autonomy setting in Fig. 21 and videos of other rollouts can be found on our project page
Full Control In Fig. 18
Partial Control To quantitatively evaluate these rollouts, we measure the collision rate and vi- sualize the results in Fig. 19 Of course, we expect the collision rate to be high in these scenarios since most of the agents in the scene are on replay. For Trajeglish models, we find that when the autonomous agent the first in the permutation to choose action, they reproduce the performance of the model with no intra-timestep dependence. When the agent goes last however, the collision rate drops significantly. Modeling intra-timestep interaction is promising way enable more realistic simulation with some agents on replay, which may have practical benefits given that the computational burden of simulating agents with replay is minimal. trajectory for agents controlled by Trajeglish shifts between the full autonomy setting and the partial autonomy setting. The agent follows traffic flow and cedes the right of way when replay agents ignore the actions of the agent controlled by the traffic model.
Fig. 22 visualize how the
A.8 ADDITIONAL ANALYSIS
Data and Training Statistics We report a comparison between the number of tokens in WOMD and the number of tokens in datasets used to train GPT-1 and GPT-2 in Tab. [ Of course, text token and a motion token do not have exactly the same information content, but we still think the comparison is worth making it suggests that WOMD represents dataset size similar to BookCorpus Zhu et al. (2015) which was used to train GPT-1 and the scaling curves we compute for our model shown in Fig. support this comparison. We also report the number of tokens collected per hour of driving to estimate how many hours of driving would be necessary to reach given token count. In Tab. 6 we document the extent to which using mixed precision and flash attention improves memory use and speed. Using these tools, our model takes 2 days to train on 4 A100s.
Context Length Context length refers to the number of tokens that the model has to condition on when predicting the distribution over the next token. Intuitively, the model is given more context, the model should get strictly better at predicting the next token. We quantify this effect in Fig. 20 We find that the relative decrease in off steeply for our model for pedestrians and cyclists, which aligns with the standard intuition that find a significant decrease in cross entropy with up to 2 seconds of context for vehicles. which is double the standard context length used for vehicles on motion prediction benchmarks (Ettinger al. 2021 Caesar et al. 2019).
entropy from increasing the context length drops
these kinds of agents are more Markovian. In comparison,
18

---

Preprint
marginal
0.06
w/o mep
marginal
1.7
Vehicle
0.05
trajeglish
no intra
Beletive Negetvve Log-Likelhhod
1.6
Pedestrian
Cyclist
trajeglish [lest]
Context
plco
trajcglish
reg.
0.03
trajeglish w/ reg. (last)
1.4
0.02
1.3
1.2
1.1
0.00
1.0
2.0
3.0
4.0
5.0
6.0
Rollout Time
1.0
1.5
2.0
Context Langth (9)
Figure 19: Partial control collision rate We plot the collision rate function of rollout time when the traffic model controls only one agent while the rest are on replay. We expect this collision rate to be higher than the log collision rate since the play agents do not react the dynamic agents. We note the collision rate decreases significantly just
Figure 20: Context Length We plot the neg-
ative log-likelihood (NLL) when we vary the
context length at test-time relative to the NLL at full context. Matching with intuition, while pedestrians and cyclists are more Markovian short horizon, interaction occurs on a longer timescale for vehicles.
by placing the agent last the order, showing that the model learns to condition on the actions of other agents within single timestep effectively.
Table 5: Dataset comparison by tokens
Table 6: Training efficiency
tokens
rate (tok/hour)
muScenes
0.85M
memory
speed (steps/hour)
WOMD
1.5B
1.2M
no intra
14.7 MiB
8.9k
WOMD (moving)
0.88M
Trajeglish (mem-efficient)
7.2 MiB
11.1k
BookCorpus (GPT-I)
IB
Trajeglish (1+fash)+)
5.6 MiB
23.0k
WehnWe (GPT-2)
Figure 21: Full control rollouts Additional visualizations of full control samples from our model. The model captures the collective behavior of agents at an intersection and individual maneuvers such as U-turns.
19

---

Preprint
controlled
80000
setost
Figure 22: Partial control comparison We visualize the effect of controlling only one agent with Trajeglish and controlling the others with replay. The left scene in each pair is full control sample from Trajeglish. The right scene is generated by placing all green cars on fixed replay tracks and controlling the single blue car with Trajeglish. Our model reacts dynamically other agents in the scene at each timestep.
20


Reviews:
[{"review_id": "NqBIX5lL1W", "reviewer": "Reviewer_KYot", "summary": "This paper presents a language modeling inspired approach to data-driven traffic simulation. The key step involved is tokenizing future driving scenario data into a sequential, language-style format, for which this paper compares several tokenization schemes. Once tokenized, a simple transformer encoder-decoder architecture is proposed to encode the initial scene state and autoregressively decode the tokenized scene future. Training the model follows the standard next token prediction objective as in language modeling, with an optional noise term on the ground truth tokens to deal with distribution shifts caused by teacher forcing. A diverse set of experiments on the Waymo Open Motion Dataset (WOMD) provide several insights on how design choices for this new formulation impact simulation quality.", "questions": "1. Please see “Weaknesses” - these are the key points with the most influence on my rating. If addressed via a fair and direct comparison to existing work, I am inclined to improve my rating.\n2. Given the simple and scalable architecture, it would be interesting to analyze the importance of scale (in terms of #parameters in the encoder/decoder) towards the performance of the proposed model.\n3. The clarity of Figure/Table captions and their placement within the document could be improved, currently, they are often very far from the text referencing them.\n4. How are actors ordered in the decoder? Is this randomized for each scene during both training and inference?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The key strengths of this work lie in its conceptual and architectural simplicity in comparison to existing methods. The idea is well-motivated and the presentation is clear. Besides this, the paper provides a detailed experimental analysis on different aspects of the proposed design space.", "weaknesses": "1. The benchmarking in Table 1 follows a much simpler setting with fewer max agents (24 vs. 128) and a shorter time horizon (6 seconds vs. 8 seconds) than prior work on WOMD [1,2,3]. \n2. As a result of this simpler benchmark and missing comparisons to any prior architecture, this paper does not address the key question of whether the proposed method is competitive to the current state-of-the-art despite its simplicity. At a glance, it seems to be much worse, with a minADE >3m in comparison to the SoTA methods with minADE < 1m on the more challenging standard WOMD setting. \n3. The paper is not self-contained, with important details (e.g., related work and several figures referenced during discussions in the main paper) only available in the appendix\n\n[1] https://arxiv.org/abs/2209.13508\n\n[2] https://arxiv.org/abs/2306.17770\n\n[3] https://arxiv.org/abs/2309.16534", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8G2LC3gKzk", "reviewer": "Reviewer_qzEz", "summary": "“Trajeglish: Learning the Language of Driving Scenarios” proposes a model that can create scene-consistent rollouts for a subset of agents in a scene. In particular, the proposal consists of a tokenization algorithm, “k-disks”, for tokenization an agent’s motion, and a transformer-based model architecture that autoregressively and causally rolls out agents’ future trajectories. The authors provide competitive results on WOMD and transfer to nuScenes.", "questions": "* How does your approach compare to the recently published “MotionLM: Multi-Agent Motion Forecasting as Language Modeling” (https://arxiv.org/pdf/2309.16534.pdf)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Strong tokenizer k-disks outperforming kMeans baselines with low discretization errors and convincing ablation study\n* Autoregressive and casual rollouts\n* Experiments demonstrating the benefits of intra-timestep dependence of agents\n* Experiments demonstrating the transfer to nuScenes", "weaknesses": "* Missing WOMD baseline results from other models\n* Similar contributions as the recently published “MotionLM: Multi-Agent Motion Forecasting as Language Modeling” (https://arxiv.org/pdf/2309.16534.pdf)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1odIqVH99H", "reviewer": "Reviewer_mABk", "summary": "This paper proposes a method called Trajeglish to generate future trajectories for traffic participants in a scenario. In particular, they propose a method to tokenize trajectory data using a small vocabulary. Besides, they propose a transformer-based architecture for modeling the action tokens on map information as well as initial states of traffic participants. To evaluate Trajeglish, the authors compare it with a behavior cloning method and a baseline that only models single agent trajectories. The result shows that their method achieves superior performance.", "questions": "-What’s the motivation of using a small vocabulary compared with using the actual values as in most of existing work (as in Appendix B)?\n\n-The provided video is a bit confusing. How do you control other vehicles that are neither replay nor trajeglish? In some videos the legends only show these two types but there are vehicles of other colors.\n\n-Can you also show the variance for Figure 8?\n\n-Figure 9 why the collision rate decreases when the rollout becomes longer?\n\n-Trajgelish behavior under longer horizon rollout (e.g., 200 timesteps)?\n\n-For the experiments, 16 scenarios are sampled for every clip in WOMD? Or only 16 clips in total?\n\n-Can you also provide some qualitative visualization for nuscenes in Appendix?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "### 1.The idea of tokenization using a small vocabulary is moderately novel.\n\n### 2.The visualization and illustration are well made and help the readers to understand the paper.", "weaknesses": "## Major:\n\n### 1.motivation of using tokenization (compared with using the actual values as in most of existing work in Appendix B) is not very clear.\n\n### 2.the experimental results are not very impressive\n\n(1) Improvements in Table 1 seem quite small. Can you show standard deviations for the results? \n\n(2) only evaluate on open-loop simulation but not on close-loop simulation\n\n(3) the baseline details are not given (e.g., “The “marginal” baseline is an equally important baseline designed to mimic the behavior of models such as Wayformer (Nayakanti et al., 2022) and MultiPath++ (Varadarajan et al., 2021) that are trained to model the distribution over single agent trajectories instead of multi-agent scene-consistent trajectories.” However, it is unclear if this baseline really can achieve similar performance as Wayformer / MultiPath++ as the authors did not give further details) and it is hard for one to assess if they are really strong baselines.\n\n### 3.motivation of having a model that take order into account is not very convincing\n\nIn particular, the idea of having this order seems very unnatural. For example, in the real world the likelihood of equally capable drivers (whether human or AI) to have collisions should be equal?\n\n## Minor:\n\n### 1.missing some relevant work on multi-agent trajectory prediction:\n\nHivt: Hierarchical vector transformer for multiagent motion prediction, Z. Zhou, L. Ye, J. Wang, K. Wu, and K. Lu.\n\nLanguage-Guided Traffic Simulation via Scene-Level Diffusion, Z. Zhong, D. Rempe, Y. Chen, B. Ivanovic, Y. Cao, D. Xu, M. Pavone, B. Ray\n\n### 2.did not discuss the limitations of the current work", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)