DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learning
Authors: Ke Guo
Haochen Liu
XIAOJUN WU
Chen Lv
Venue: ICLR 2026 Poster (2026)
Keywords: traffic simulation
multi-agent imitation learning
generative adversarial imitation learning
Primary area: applications to robotics, autonomy, planning

Abstract: Realistic traffic simulation is critical for the development of autonomous driving systems and urban mobility planning, yet existing imitation learning approaches often fail to model realistic traffic behaviors. Behavior cloning suffers from covariate shift, while Generative Adversarial Imitation Learning (GAIL) is notoriously unstable in multi-agent settings. We identify a key source of this instability—irrelevant interaction misguidance—where a discriminator penalizes an ego vehicle’s realistic behavior due to unrealistic interactions among its neighbors. To address this, we propose Decomposed Multi-agent GAIL (DecompGAIL), which explicitly decomposes realism into ego–map and ego–neighbor components, filtering out misleading neighbor–neighbor and neighbor–map interactions. We further introduce a social PPO objective that augments ego rewards with distance-weighted neighborhood rewards, encouraging overall realism across agents. Integrated into a lightweight SMART-based backbone, DecompGAIL achieves state-of-the-art performance on the WOMD Sim Agents 2025 benchmark.

Published as a conference paper at ICLR 2026
DECOMPGAIL: LEARNING REALISTIC TRAFFIC BE- HAVIORS WITH DECOMPOSED MULTI-AGENT GENERA- TIVE ADVERSARIAL IMITATION LEARNING
Ke Guo', Haochen Liu', Xiaojun Wu', Chen Lv
1 School of Mechanical and Aerospace Engineering, Nanyang Technological University
2 Desay SV Automotive
[ke. guo, Lyuchen] ) @ntu. edu. sg, haochen0020e ntu. edu.sg, Xiaojun, Wu@desaysv.com
f Corresponding author
ABSTRACT
Realistic traffic simulation is critical for the development of autonomous driving systems and urban mobility planning, yet existing imitation learning approaches of- ten fail to model realistic traffic behaviors. Behavior cloning suffers from covariate shift, while Generative Adversarial Imitation Learning (GAIL) is notoriously unsta- ble in multi-agent settings. We identify a key source of this instablity--irelevann irrelevant interaction misguidance- where discriminator penalizes an ego vehicle's realistic behavior due to unrealistic interactions among its neighbors. To address this, we propose Decomposed Multi-agent GAIL (DecompGAlL) which explicitly decom- poses realism into ego-map and ego neigher components, filtering out misleading neighbor-neighhor and neighbor-ma map interactions. We further introduce social PPO objective that augments ego rewards with srrrace--eeighed neighborhood rewards, encouraging overall realism across agents. Integrated into lightweight SMART-based backbone, DecompGAll. achieves state-of-the-an performance on the WOMD Sim Agents 2025 benchmark.
INTRODUCTION
Realistic traffic simulation a cornerstone for intelligent transportation systems, autonomous driving. and urban mobility planning. High-fidelity simulators enable safe, controlled evaluation of driving policies, stress prress-testing of safety-critical scenarios, and principled analysis of interventions (Wilkie et al. 2015 Li al. 2017] Pan et al. 2025). Yet achieving realism remains challenging due to diversity and complexity of human driving behaviors. Prior rule-based models (Treiber et al. 2000 Zhang et al. 2021b), while interpretable and efficient, struggle capture the complexity and diversity of world driving. Behavior cloning (BC) (Michic et al. 1990) -s used in Bergamini et al. (021); ) Xu al. (2023); Philion et al. (2024) casts imitation as supervised learning from expert trajectories, but suffers from covariate shift (Ross et al.) [011]:: the state distribution induced
by learned policy drifts from that of the expert, compounding errors over time.
2008), which These limitations have motivated inverse reinforcement learning (IRL) (Ziebart et al. learns a reward function that explains expert behavior and then trains a policy via reinforcement
learning (RL). By aligning expert and learner trajectory distribution (Ghasemipour et al. 2019).
IRL theoretically addresses covariate shift. Among IRL approaches, GAIL (Ho & Ermon
2016
has gained popularity for traffic modeling (Bhattacharyya et al.] 2018, Behbahani et al. 2019
Bhattacharyya al., 2019 Zheng al., 2020; Wei al.[2202 Koeberle et al.) 2022] Xin al. 2024), as it bypasses explicit reward inference by training discriminator to provide a surrogate reward that encourages policies to match the expert's trajectory distribution. However, multi-agent settings GAIL often suffers from training instability. To mitigate this, prior work (Bhattacharyya al. 2018) introduces parameter sharing for the policy and discriminator, and curriculum learning strategies that gradually increase the number of vehicles (Song et al. 2018) or the rollout horizon (Behbahani et al.) 2019). While these techniques provide partial improvements, they do not target the root cause and therefore fail to fully resolve the instability.

---

Published as a conference paper at ICLR 2026
Decompose
Discriminator
Fake
Real
Realistic
Conflict
Low
130 400
High Reward
Real
Real
Behavior
Reward
(a) Decentrailized GAIL
(b) Decomposed GAIL
Figure I: Comparison with standard decentralized GAIL. (a) A standard decentralized discrimi- nator evaluating local observations can yield spuriously low rewards for ego-realistic behavior due unrealistic neighbor-neighbor interactions. (b) DecompGAlL, separately assesses ego-map and ego-neighbor realism, combining them to obtain a high reward for expert-like ego behavior even when neighbors misbehave.
To investigate this instability, consider the scenario in fig. [ An ego (red) vehicle, controlled by the learned policy, drives realistically. nearby (green) vehicle, also policy-controlles behaves unrealistically and collides with another (yellow) vehicle. A standard decentralized discriminator evaluating the ego's ss local observation will output a low realism score, because expert data rarely contains such neighbor-neighbor collisions. This yields contradiction: realistic ego behavior is penalized. We hypothesize that the discriminator overuses weakly relevant neighbor-neighbor and maighbor-map interactions, which leads training instability. We term this the irrelevant interaction misguidance problem, whose severity grows with the number of input neighbors (as confirmed fig. 3
To address irrelevant interaction misguidance, we propose Decomposed Multi-agent Generative Adversarial Imitation Learning (DecompGAIL.). For each agent, we explicitly disentangle realism into (i) ego-map realism and (ii) ego-neighbor realism by separately discriminating the ego with the static map context and with each neighbor. This input design prevents the discriminator from inferring neighbor-neighbor and neighbor-map interactions-signall that are only weakly correlated with the ego's action yet are influenced by learned policy-yielding rewards that are both more stable and more informative for policy updates. In practice, further introduce social PPO objective that augments ego rewards with distance-weighted neighborhood rewards, encouraging agents to improve their realism without degrading that of nearby agents.
This paper makes the following key contributions:
We identify irrelevant interaction misguidance, showing how standard decentralized GAIL discriminators induce instability by overemphasizing weakly relevant neighbor-neighbor interactions.
We propose DecompGAlL, using a decomposed discriminator that separately models ego sgene and ego- neigheeg realism while suppressing irrelevant higher-order interactions to yield stable and informative rewards.
On the WOMD Sim Agents 2025 benchmark, DecompGAIL improves training stability over standard decentralized GAIL and achieves state-of-the realism.
RELATED WORK
Realistic Traffic Modeling. Traffic modeling can be naturally framed as a multi-agent imitation learning (IL) task (Brown et al. 2020), approached via BC or IRI. BC learns action distribution
conditioned on expert states, parameterized deterministically (Bergamini et al. [02 Kamenev et al.)
2022), as a Gaussian (Yan et al., 2023 Guo et al. 22244] Zhou et al. 2024 Lin et a 2025). as
a categorical distribution (Philion et al. 2024 Wu et al. 2024 Hu et al. 2024 Zhao et al. 2024).
However, BC suffers from covariate shifi Guo et al. (202)). A direct remedy is data augmentation.
Offline approaches perturb demonstrations by injecting noise or dropping trajectory segments (Philion
al. 2024 Wu al.
2024
Hu et al.) 2024)
More recent online approaches, such CAT-K (Zhang
2

---

Published as a conference paper at ICLR 2026
al. 2025a) and uniMM (Lin et al. 2025 generate augmented trajectories during learning. CAT-K always selects one of the top-K most likely policy actions that moves the agent closest to the expert next state, while uniMM extends this idea to Gaussian mixture policies by choosing the anchor nearest to the expert future trajectory. Nevertheless, these approaches still suffer from distribution shift, since augmentation remains anchored expert data rather than policy rollouts.
IRL provides a theoretical solution by letting the agent interact with the environment during train- ing. Among IRL methods, GAIL (Ho & Ermon 2016) most widely used in driving behavior modeling (Kuefler et al. 2017). but performance degrades in multi-agent domains because of the nonstationary training environment. To stabilize training, prior works (Bhattacharyya et al. 2018 Behbahani et al., 2019; Bhattacharyya et al., [019 Zheng et al., 2020; Wei et al., [2021] Koeberle et al. 2022 Xin et al. 2024) often employ parameter sharing, where all agents use the same policy and discriminator network, thereby pooling experience across agents. Curriculum learning is also
applied, gradually increasing the number of vehicles (Song et al., 2018) Chen et al. 2022) or the rollout horizon (Behbahani et al. 2019). However, these methods still produce undesirable behaviors such as off-map driving, collisions, and abrupt braking (Bhattacharyya et al., 2219).. To mitigate this, some works add hand-crafted rewards alongside adversarial rewards (Bhattacharyya et al. 2019 Xin et al., 2024; Bhattacharyya et al.] 2022), or purely engineered objectives (Peng et al., 2024
Chen al. (025). Yet such hand-crafted rewards introduce prior bias and may fail to capture the
complexity of human driving behavior.
Multi-agent Generative Adversarial Imitation Learning. Beyond traffic modeling, multi-agent GAIL has been explored in other domains. However, most approaches (Song et al. 2018) Yu et al.
2019 Zhang allaa2222222 2021a) are evaluated in small-scale settings with few agents, where spurious cross- agent interactions are weak. Multi-agent GAIL (Song et al. (2018) employs centralized discriminator that produces a single shared reward, complicating credit assignment and hindering scalability. MA- 2020) combines decentralized actor-crite training with adversarial learning, improving stability in cooperative tasks but still struggling to disentangle individual contributions. BM3IL (Yang et al. 2020) enhances scalability via mean-field approximation that averages other agents' actions as discriminator input, which reduces variance but limits fine-grained interaction modeling. Other works add regularization (e.g.. Wasserstein objectives (Arjovsky al., (017)) to curb discriminator overfitting, at the cost of expressiveness. To our knowledge, prior work has not directly analyzed the root cause of instability in multi-agent GAIL.
DAAC (Jeon et al.
BACKGROUND
Markoy Game A Markov game (Littman 1994) is an extension of the Markov Decision Process (MDP) to multiple agents. Markov game with N agents is defined by the tuple (S, [Ai P, (ri) (=: 7), where S is the state space, A, is the action space of agent i, P S Sx Ai X .... x AN + A(S) is the transition function, r : S x Ai x x... x AN - R is the reward function for agent i, and y E [0,1) is the discount factor. Each agent is equipped with policy n. S + A(A,), which maps states to distributions over actions. For notational convenience, we use bold symbols to denote joint quantities; for example, denotes the joint policy, the joint action.
Multi-agent Generative Adversarial Imitation Learning. The objective of multi-agent IL is to learn a joint policy To that mimics the expert joint policy Tg, given a of expert demonstrations Dg = ........ } with trajectories т ($1, a1...., ST, ar). GAIL casts IL as two-player minimax game between a policy and discriminator: the discriminator D& is trained to distinguish expert trajectories from policy rollouts, while the policy is optimized to produce trajectories that fool the discriminator. Multi-agent GAII. extends this framework to multiple agents, using either a centralized discriminator (which outputs a shared reward for all agents using all agents' observations) or decentralized discriminators (one per agent, each evaluating its own observations and actions). In this work, we focus the decentralized setting due its alignment with human driving decision- making. The objective is:
N log D (s( + Ema
N Σ log (1 Do(s,ai))
LGAIL = min max E
Σ
(1)
9
3

---

Published as a conference paper at ICLR 2026
Policy Network
Next Motion
Temporal
Map-agent
Agent-agent
Token Probalitity
Motion Tokens
xL
Socail PPO
Map Encoder
Encoded Map Tokens
Map Tokens
Ego Reward
Decomposed Discriminator
weighted sum
Scene Realism
Map-agent
Ego
BCE Loss
Temporal
Cat
Interaction Realism
MLP
Cat
Neighbors
Figure 2: Overview of the DecompGAIL framework with three components: Map Encoder (gray) extracting map features; a Policy Network (red) predicting motion-toke -token distributions; and a Decom- posed Discriminator (green) separately assessing scene (ego-map) and interaction (ego-neighbor) realism for expert and policy trajectories. A weighted combination forms each agent's 's reward, which then augmented with neighborhood rewards to build the social reward used by PPO training.
Training alternates between updating the discriminator D3 with binary cross-entropy (BCE) loss to classify expert versus policy samples, and updating the joint policy æ via multi-agent RI using the surrogate reward r; log(1 (2(8, a;)).
4 APPROACH
The overall framework of DecompGAIL is shown in fig. We first pretrain the map encoder and policy network with BC to provide a strong initialization and reduce compute. We then fine-tune the policy with DecompGAll: decomposed discriminator separately evaluates scene eego-map) and interaction (ego-neighbor) realism, explicitly omitting neighbor-neighbor and neighbor-map terms, and is trained with a BCE objective using expert and policy samples. The ego reward the weighted sum of the scene score and all pairwise interaction scores. PPO then optimizes a social reward that augments the ego reward with weighted neighbor rewards. The following sections detail DecompGAlL 's implementation within widely used, scalable Transformer architecture.
4.1 POLICY PRETRAINING
Policy Network. We instantiate our framework within SMART Wu et al. 2024 a widely used learning-based traffic model adopted by prior work (Zhang et al.) 2025a by Ahmadi & Schofield 2025). SMART parameterizes the joint action as independent categorical distributions over shared vocabulary of motion tokens. At each time step, predicts every agent's next-step token distribution conditioned on encoded map tokens m and all agents' past motion tokens azt := (aa..... Qt-1):
Ta(a; 8t)
II no(a; m));m);
where all agents share single policy mg.
Specifically, SMART employs a factorized Transformer Ngiam et al. 2021) with multi-head self- attention (MHSA) and multi-head cross-attentin (MHCA) to encode map motion interactions. The map encoder first produces encoded map tokens m by attending over spatial neighboring map tokens with MHSA. Then, for each agent at time the motion encoder takes the current motion-token
4

---

Published as a conference paper at ICLR 2026
embedding et as the query and sequentially applies temporal, map agent, and agent agent attention layers:
temp; (IHSA(((e)) Kee_,,.REE, (PE',,. 0 <<t<t
(3a)
map; MHCA ((temp))) k(m², RPE)), v(m', RPE")) jEN,
(3b)
agent; MHSA g(map)), k(map/, RPE)), umaa// RPE))) EEN;
(3c)
where RPE denotes relative positional encoding (Cui et al. 2022), and Ns is the neighborhood set of map or agent tokens determined by distance threshold. After L stacked layers of temporal, map-agent, and agent-agent attentions, an MLP head outputs the next-token distribution for each
agent.
BC Pretraining. We initialize the policy network by BC to accelerate training, since online interaction in GAIL is more computationally expensive. BC maximizes the likelihood of the expert joint action at each time step, conditioned on the expert's past motions and map tokens:
LBC max E [log No ace, m)];
(4)
This pretraining step enables the model to imitate expert behavior on observed trajectories. However, during testing rollouts, small prediction errors can accumulate, leading agents into states not encoun- tered in the demonstrations (Ross et al. (011). This distributional shift often results in implausible
behaviors such collisions off-road driving (Tian & Goel 2025).
4.2 DECOMPOSED GAIL FINE-TUNING
To address the covariate shift issue, we apply GAIL to online fine tune the BC-pretrained policy. A widely adopted practice in prior works (Bhattacharyya et al. 2018 Behbahani et al. 2019 Bhattacharyya et al. 2019) is parameter-sharing GATI. (PS-GAIL), which employs single discriminator shared across all agents. At each time step, the discriminator evaluates every agent's local observation, and the resulting score is used as the reward for updating the shared policy To via single-agent RL. To implement this, one can replace the policy's action head with binary classification head; the discriminator score for each agent i at time is:
Parameter-sharing GAIL.
D Dolse, a:) = Do sn m) = MLP(agent),,
(5)
where a
{a'z j N₁} denotes the motion token histories of all i's neighbors. When computing the reward for each agent we treat that agent as the ego agent.
Despite simplicity, PS-GAIL is often unstable (Bhattacharyya al., 2018), degrading simulation quality (see fig. 3. To examine the source, we conceptually decompose the discriminator's signal into four terms:
Dila'za N.V. m) = d1 (1 m)
Se a
JEN: ego-ncighbor (interaction) realism
ego-map (scene) realism
(6)
da (a sa m
2 aig m))
neighhor mup neighhur-neighhbur
higher orderr
Here, di measures alignment of the ego trajectory with the map (scene realism); Φ2 aggregates ego-neighbor plausibility (interaction realism); Φ3 captures neighbor-map and neighbor-neighhor signals; and d4 models higher-order correlations among ego, neighbors, and the map. Notably, da is only weakly coupled to the ego's action, yet its interaction count grows roughly quadratically with neighborhood size, making the overall reward increasingly noisy [M]] increases and thus less informative for ego policy updates. We refer to this phenomenon as irrelevant interaction misguidance.
5

---

Published as a conference paper at ICLR 2026
Algorithm 1: Decomposed GAIL
Input: Markov game M, expert trajectories DE. policy Ty, decomposed discriminator Dor,du value functions Væ, batch size B, discount GAE X, PPO clip weights wijs Xij. Output: Learned policy TT.
while not converged do
Collect B trajectories with ne in M; foreach agent i and step do
Compute realism: S; = Φ1 (a m), = 626a'aa- ag)) ;.): j EN:
Update discriminator Den by minimizing
LD ETE log St + Σ Wis log 12)
+ ET
log(1 (5:) Σ wis log (1
Compute agent reward
r = -lo((11-5)))
Wij log(1 19-11p -1,),
JEN;
and social reward r ri + Aapry:
Estimate advantages A and targets G with GAE; Update no, Vi using PPO with clipped surrogate and value loss and BC loss;
Decomposed Discriminator. This irrelevant interaction misguidance arises because the PS- GAIL network in eq. (5) can implicitly represent all four terms: map and agent tokens (positions and features) fused into single ego feature before scoring, entangling signals. To avoid this, we replace the monolithic discriminator with a decomposed architecture that (i) computes scene realism from ego-map inputs and (ii) computes interaction realism from pairwise ego-neighbor inputs, thereby suppressing neighbor neighbor weighbor-neighbor and higher-order signals by design.
Concretely, scene realism uses the temporal and agent attention features as eqs. Bap and followed by an MLP:
3b
s = Φ1 (a m) = (mapi). Interaction realism is computed per ego neighbr pair via another MLP over concatenated temporal features:
= 02 a = MLe((e RPE, temp/]),
(8)
where [:] denotes concatenation. The discriminator loss
CD = log St
Σ Wiy log 122
+ E.ro log(1 S + Σ wiy log(1 12)) JEM.
(9)
JEN,
with distance-decayed weights w,y = a exp( (xi(-d(i,j)/B) where d is inter-agent distance and a, B hyperparameters, to emphasize nearby interactions that are more causally tied to the ego 's action. The per-agent reward is
r log(1 (gg(1-00 - -S?)
Σ JEN;
Wiy log(1 11)).
(10)
Social Policy Learning. A straightforward way to optimize a decentralized multi-agent policy independent PPO (IPPO) (de Witt et al.) 2020). which treats each agent as single-agent environment. However, optimizing purely individual objectives can degrade the overall realism of the population (Schwarting al., 2019). We therefore define social reward for agent i:
St ri rt r2
rf + Σ Adp!
(11)
J.N.

---

Published as a conference paper at ICLR 2026
PS-GAIL (5 neighborr)
0.6
GAIL neighbers)
PS-GAIL (all neighbors)
0.80
DecompGAIL (all neighbors)
0.5
Pollcy Policy Discttiinntoor
Realism MetaMatric
0.78
0.4
0.3
0.74
0.2
0.72
0.70
2000
4080
6000
8000 10000
12000
14000
2000
4000
6000
8000
10600
12000
14000
Global Step
Global
Figure 3: Training stability, In the left plot, solid lines denote the mean output realism scores, while shaded areas indicate the standard deviation. The realism meta-metric evaluated the 2% validation split. The DecompGAlL. maintains lower variance and better simulation realism performance than PS-GAIL.
where v.V. aggregates neighborhood rewards and Aij is distance-decayed weight (analogous to Wiy). Training then follows IPPO, but with individual rewards replaced by social rewards. Advantages are estimated with Generalized Advantage Estimation (GAE), and each agent's value function is given by an MLP applied to the policy network's final agentt attention features. To improve stability, combine the PPO loss and BC loss. The full procedure is summarized in algorithm [ More details about training be found in appendix A
5 EXPERIMENTS
This section addresses the following research questions:
Q1: Does DecompGAIL improve training stability and performance over the widely used PS-GAIL. method?
Q2: How does DecompGAIL compare with state-of-the-art baselines in terms of realism?
Q3: What is the contribution of each component to overall performance?
We first describe the experimental setup, including dataset, metrics, and baselines, before presenting detailed analyses for each question.
5.1 EXPERIMENTAL SETUPS
Dataset. We conduct experiments the Waymo Open Motion Dataset (WOMD) (Ettinger et al. 2021), a large-scale benchmark for traffic simulation containing 487k training, 44k validation, and 44k test scenes. Each scenario is recorded 10 Hz, with second of historical context and 8 seconds of future trajectories for up to 128 traffic participants (vehicles, cyclists, pedestrians), along with high-definition maps. Following CAT-K (Zhang al. 2025a), evaluate 2% of the validation split (880 of 44,097 scenarios).
Metrics. We adopt the official metrics from the WOMD Sim Agents 2025 Challenge (WOSAC) MMonall et al. 2024):
Kinematic: weighted likelihood of linear speed, linear acceleration, angular speed, and angular acceleration.
Interactive: weighted likelihood of nearest object distance, collision, and time-to-collision.
Map-based: weighted likelihood of road edge distance, oof-road, and traffic-light violation.
7

---

Published as a conference paper at ICLR 2026
Table Results on the WOSAC 2025 leaderboard test split. (Wamoo 2025)
Model
Metametric
Kinematic
Interactive
Map-hased
minADE
CeeT--iny-DeoonGC... (ours)
0.7864
0.4919
0.8152
0.9176
1.4299
SMART-RI [Pei 2025
0.7858
SMART-lim' KL.P PSim Altmudi & Scholield 2025
0.7857
0.4927
0.8129
0.9183
1.3252
TrajTok Zhang al. 207256
0.7852
SMAKT-iny-CLSP Zhanng et al. 2025n
0.7846
0,4931
0.8106
0.9177
1,3065
UnIMM Lin et al. 2025
0.7829
0.4914
0.8089
0,9161
1.2949
SMART
2024
0.7814
0,4854
0.8089
0.9153
1.3931
L.M22 (Wanp al 2025
0.7779
0.4846
0.8048
0.9109
1.2827
InfGen (Peng
0.7731
0.4493
0.8084
0.9127
1.4252
The realism mela-metric is a weighted combination of these measures and serves as the primary evaluation criterion. We also report minADE, which is widely used in motion prediction but not included in the realism meta-metric. All metrics are computed from 32 rollouts of 8 seconds at 10 Hz.
5.2 STARILITY (Q1)
To assess stability, we compare DecompGAlI. to naive PS-GAIL baseline obtained by replacing our decomposed discriminator with standard local discriminator while keeping all other components unchanged. We track the discriminator scores and the realism meta-metric during training under four settings: PS-GAIL with 10, and available nearest neighbors within 60 m, and DecompGAll. with all neighbors within the same range.
As shown in fig. PS-GAIL exhibits increasing variance and a lower mean score as neighbor count grows, as the model overfits to penalize unrealistic nrighbor-neighbo and mapp interactions. Because these interactions are only weakly correlated with the ego 's own behavior, the resulting reward signals are unstable and misleading, ultimately degrading simulation performance. In contrast, DecompGAlI. maintains low variance with a mean near the expected equilibrium (0.5), providing stable, informative rewards that support steady RL improvement. As result, its simulation performance improves steadily throughout training.
5.3 PERFORMANCE (Q2)
To address Q2, we compare DecompGAIL, with state-of-the baselines on the WOSAC leader- board.
Baselines.
We compare both tokenized models (Pei et
2025
Ahmadi & Schofield 2025 Zhang
et al.
2025b a Wu et al. 2024 [Png et al. 2025 Wang al. 2025) and continuous model (Lin et al.
2025
covering broad spectrum of learning paradigms:
Naive BC [Wu al. 2024 Zhang et al.
2025b
Peng et
2025):
BC with data augmentation (Zhang al.
2025a
Lin 2025
RL finetuning approaches (Wang al.
2025
Pei et al.
2025
Ahmadi Schofield 2025).
Specifically, SMART-RI (Pei al. 2025) applies RI-style reinforcement fine-tuning. SMART-tiny- RLFTSim (Ahmadi & Scholield 2025) directly optimizes the realism meta-metric as reward for RL fine-tuning the SMART-tiny model (Wu et al., 2024). TrajTok (Zhang et al., 20255) enhances SMART by introducing trajectory tokenizer with better coverage, symmetry, and robustness, along with spatial-aware label smoothing. SMART-tin-CLLFF (Zhang et al., 2025a) augments training via expert-guided online trajectory generation to mitigate covariate shift. UniMM (Lin) et al. 2025) employs a continuous Gaussian mixture model with expert-guided data augmentation. LLM2AD Wang et al. 2025) integrates GRPO post-training with rest-time search and clustering larger SMART backbone equipped with enhanced tokenization and positional encoding. InfGen (Peng al.) 2025) models the entire
sequence of tokens-inchun agent states, motion vectors, and traffic signals-and uses a Transformer to autoregressively simulate traffic.
Results. As reported in table ! DecompGAIL. achieves the best realism meta-me meta-metric and consis- tently outperforms all baselines interactive metrics, while remaining competitive kinematic and
8

---

Published as a conference paper at ICLR 2026
Table 2: Ablation study on WOSAC 2% validation split.
Model
Metametrie
Kinematic
Interactive + Map-based
Collision Likelihood
DecompGAlL
0.7836
0.5077
0.8204
0.8926
0.9667
scene realism
0.7801
0,5053
0.8248
0.8795
0.9794
interact realism
0.7772
0.5043
0.8132
0.8869
0.9573
interact realism
0.7819
0.5140
0.8153
0,8921
0.9635
neighborhood reward
0.7871
0,5146
0.8258
0.8930
0.9788
mean neighborhood reward
0,7882
0,5140
0.8282
0.8935
0.9812
DecompGAIL
0.7889
0.5148
0,8283
0,8948
0.9837
map-based scores. Its relatively higher minADE can be attributed to the fact that method priori- tizes matching feature distributions between expert and policy trajectories, rather than optimizing for distance-basse based similarity. Overall, these results demonstrate that DecompGAIL fine-tuning delivers substantial improvements the realism of traffic simulation.
5.4 ABLATION (Q3)
The ablation study in Table 2 evaluates the contribution of each component of DecompGAll:
DecompGAIL Fine-tuning. We begin by evaluating the model prior to the DecompGAIL fine- tuning stage. Omitting this stage ("w/o DecompGAlL") reduces the realism meta-metric from 0.7889 to 0.7836 and lowers the collision likelihood from 0.9837 to 0.9667, indicating that a purely BC-pretrained model suffers from covariate shift and produces less realistic simulations.
Realism Decomposition. Next, we examine the influence of the realism decomposition and the disthnee-weightee design. Removing scene realism causes a marked decline in map-based scores (0.8795 vs. 0.8948) while leaving interaction performance relatively strong. Conversely, excluding interaction realism leads to a sharp degradation in both the interactive metric (0.8132 vs. 0.8283) and the collision likelihood (0.9573 vs. 0.9837), underscoring the necessity of explicitly modeling ego-neighbor interactions for realistic multi-agent behavior. Replacing distance-weightedd aggregation with uniform averaging partially recovers performance but still lags behind the distance-decayed variant, highlighting the effectiveness of distance-aware weighting.
Social Reward. Finally, we evaluate the role of social rewards. Removing neighborhood rewards ("w/o neighborhood reward") slightly decreases both interactive and map-based scores, while re- placing distance-weightn weighted aggregation with uniform averaging ("mean neighborhood reward")) also degrades performance. Together, these results demonstrate that distance-decayed social rewards are more effective at promoting realism across all agents.
5.5 HYPERPARAMETER
In this section, we analyze the effect of the decay parameters a and |3 used for interaction weighting and social rewards on the realism meta-metric. We first conduct a sweep over decay scales a (2.5,5, 10, 20} and decay ranges 3 E {1. 2.5, 5, 10} for the interaction weights, with social rewards disabled. The results in the left subplot of fig. show two key observations: (1) performance is more sensitive to the decay range /3 than to the scale or; and (2) extremely small or large 3 overemphasizes very close or very distant neighbors, both of which degrade realism.
Next, fixing the interaction weights at their optimal values (a 10, 3 = 2.5), sweep decay scales E (0.5, 1, 5, 10) and rates 3 E {1, 2.5, 5, 10) for the social reward weighting. We find in right subplot of fig. that social reward parameters have weaker influence than interaction weights. However, very large a or 3 make the neighborhood weights nearly uniform, which increases noise in the social reward and slightly reduces realism.
5.6 QUALITATIVE RESULTS
We provide qualitative demonstrations of our fine-tuned model in fig. showcasing one rollout in two distinct scenarios. In both cases, the policy generates realistic multi-agent trajectories, maintaining

---

Published as a conference paper at ICLR 2026
2
8.7945
2.100
214
A1933
E2077
5.772
A18J0
Figure 4: Inftuence of decay parameters and 3 on interaction weighting and social rewards.
Sim
time
time
Sim 0.5
Reward
Figure 5: Qualitative results WOSAC. Each shows rollout of model in different scene. Transparent boxes denote ground-truth agents; solid boxes are agents generated by our model. Agent colors indicate the per-agent reward from decomposed discriminator. The red rectangle in the first row highlights a low-reward near-collision; the red rectangle in the second row highlights low-reward off-road tendency.
collision-free and on-road behavior throughout the full 8-second horizon. This highlights the ability of our model to sustain expert-like motion even complex interactive settings.
To further interpret model behavior, visualize the per-agent rewards estimated by decomposed discriminator, encoded as the color each agent at every timestep. As illustrated, the discriminator consistently assigns lower rewards agents that deviate from realistic behavior-for example, when a vehicle approaches a near-collision situation or begins drifting toward the road edge. Conversely, agents following smooth and expert-like motion patterns are assigned high rewards. Additional rollout examples, including more challenging interactive scenarios, are provided in the supplementary.
CONCLUSION
We introduced DecompGAlI,, decomposed adversarial imitation learning framework for realistic multi-agent traffic simulation. By explicitly separating scene and interaction realism in the discrim- inator, DecompGAIL suppresses weakly relevant neighbor-neighbor and neighbor-map signals that otherwise misguide reward learning. Coupled with a social PPO objective that augments ego rewards with iistance-decayer neighborhood rewards, DecompGAIL delivers stable and informative training signals. Empirically, DecompGAIL improves training stability over PS-GAIL and achieves state-of-the-an afrt realism the WOSAC 2025 benchmark.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported by Nanyang Technological University (NTU)-DESAY SV Research Program under Grant 2018-0980.
REFERENCES
Ehsan Ahmadi and Hunter Schofield. Riftsim: Multi-agent traffic simulation via reinforcement learning fine-tuning (technical report for waymo open sim agents challenge). Technical report.
Waymo, 2025, URL http:: http:::/sstoaaggg.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning (ICML), pp. 214-223, PMLR, 2017.
Feryal Behbahani, Kyriacos Shiarlis, Xi Chen, Vitaly Kurin, Sudhanshu Kasewa, Ciprian Stirbu, Joao Gomes, Supratik Paul, Frans A Oliehoek, Joao Messias, et al. Learning from demonstration in the wild. In International Conference on Robotics and Automation (ICRA), pp. 775-781, 2019.
Luca Bergamini, Yawei Ye, Oliver Scheel, Long Chen, Chih Hu, Luca Del Pero, Blažej Osiński, Hugo Grimmett, and Peter Ondruska. SimNet: Learning reactive self-driving simulations from real-world observations. In International Conference on Robotics and Automation (ICRA), pp. 5119-5125, 2021.
Raunak Bhattacharyya, Blake Wulfe, Derek Phillips, Alex Kuefler, Jeremy Morton, Ransalu Senanayake, and Mykel Kochenderfer. Modeling human driving behavior through generative adversarial imitation learning. IEEE Transactions on Intelligent Transportation Systems (TITS), 24 (3):2874-2887, 2022.
Raunak P Bhattacharyya, Derek J Phillips, Blake Wulfe, Jeremy Morton, Alex Kuefler, and Mykel J Kochenderfer. Multi-agent imitation learning for driving simulation. In International Conference Intelligent Robots and Systems (IROS), pp. 1534-1539, 2018.
Raunak P Bhattacharyya, Derek J Phillips, Changliu Liu, Jayesh K Gupta, Katherine Crigge-Campbell, and Mykel J Kochenderfer. Simulating emergent properties of human driving behavior using multi-agent reward augmented imitation learning. In International Conference on Robotics and Automation (ICRA), pp. 789-795, 2019.
Kyle Brown, Katherine Driggs-Camphell, and Mykel Kochenderfer. Modeling and prediction of human driver behavior: A survey. arXiv preprint arXiv:2006.08822 46, 2020.
Haonan Chen, Tianchen Ji, Shuijing Liu, and Katherine Driggs-Campbell. Combining model-based controllers and generative adversarial imitation learning for traffic simulation. In International Conference on Intelligent Transportation Systems (ITSC), pp. 1698-1704, 2022.
Keyu Chen, Wenchao Sun, Hao Cheng, and Sifa Zheng. Rift: Closed-loop rl fine-tuning for realistic and controllable traffic simulation. arXiv preprint 12Xiv::5005033344 2025.
Alexander Cui, Sergio Casas, Kelvin Wong, Simon Suo, and Raquel Urtasun. Gorela: Go relative for viewpoint-invariant motion forecasting, arXiv preprint arXiv:2211.02545. 2022.
Christian Schroeder de Witt. Tarun Gupta, Denys Makoviichuk, Viktor Makoviychuk, Philip HS Torr, Mingfei Sun, and Shimon Whiteson. Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint 1ttxii::0//1005333 2020.
Scott Ettinger, Shuyang Cheng, Benjamin Caine, Chenxi Liu, Hang Zhao, Sabeek Pradhan, Yuning Chai, Ben Sapp. Charles R Qi. Yin Zhou, et al. Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9710-9719, 2021.
Seyed Kamyar Seyed Ghasemipour, Richard S. Zemel, and Shixiang Gu. A divergence minimization perspective imitation learning methods. In Conference Robot Learning (CoRI.), 2019.
11

---

Published as a conference paper at ICLR 2026
Ke Guo, Wei Jing, Junbo Chen, and Jia Pan. CCIL: Context-conditioned imitation learning for urban driving. In Robotics: Science and Systems, 2023.
Ke Guo, Zhenwei Miao, Wei Jing, Weiwei Liu, Weizi Li, Dayang Hao, and Jia Pan. Lasil: Learner- aware supervised imitation learning for long-term microscopic traffic simulation. In /EEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 15386-15395, 2024.
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. Advances in Neural Information Processing Systems (NeurIPS), 29, 2016.
Yihan Hu, Siqi Chai, Zhening Yang, Jingyu Qian, Kun Li, Wenxin Shao, Haichao Zhang, Wei Xu, and Qiang Liu. Solving motion planning tasks with scalable generative model. In European Conference on Computer Vision (ECCV), pp. 386-404, 2024.
Wonseok Jeon, Paul Barde, Derek Nowrouzzezahrai, and Joelle Pineau. Scalable and sample-efficient multi-agent imitation learning. In Proceedings of the Workshop on Artificial Intelligence Safety, co-located with 34th AAAI Conference on Artificial Intelligence, SafeAl@ AAAI, 2020.
Alexey Kamenev, Lirui Wang, Ollin Boer Bohan, Ishwar Kulkarni, Bilal Kartal, Artem Molchanov, Stan Birchfield, David Nistér, and Nikolai Smolyanskiy. Predictionnet: Real-time joint probabilistic traffic prediction for planning, control, and simulation. In International Conference on Robotics and Automation (ICRA), pp. 8936-8942, 2022.
Yann Koeberle, Stefano Sabatini, Dzmitry Tsishkou, and Christophe Sabourin. Exploring the trade off between human driving imitation and safety for traffic simulation. In International Conference on Intelligent Transportation Systems (/TSC), pp. 779-786, 2022.
Alex Kuefler, Jeremy Morton, Tim Wheeler, and Mykel Kochenderfer. generative adversarial networks. In IEEE Intelligent Vehicles Symposium (/V), pp. 204-211 2017.
Imitating driver behavior with
Weizi Li, David Wolinski, and Ming Lin. City-scale traffic animation using statistical learning and metamodel-based optimization. ACM Transactions on Graphics (TOG), 36(6):1-12, 2017.
Longzhong Lin, Xuewu Lin, Kechun Xu, Haojian Lu, Lichao Huang, Rong Xiong, and Yue Wang. Revisit mixture models for multi-agent simulation: Experimental study within unified framework. arXiv preprint arXiv:2501.17011 2025.
Michael L Littman. Markov games as framework for multi-agent reinforcement learning. In Machine learning proceedings. pp. 157-163. Elsevier, 1994.
Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for gans do actually converge? In International conference on machine learning (ICML), pp. 3481-3490, 2018.
Donald Michie, Michael Bain, and Jean Hayes-Michie. Cognitive models from subcognitive skills. IEEE Control Engineering Series, 44:71-99, 1990.
Nico Montali, John Lambert, Paul Mougin, Alex Kuefler, Nicholas Rhinehart, Michelle Li, Cole Gulino, Tristan Emrich, Zoey Yang, Shimon Whiteson, et al. The waymo open sim agents challenge. Advances in Neural Information Processing Systems (NeurIPS), 36, 2024.
Jiquan Ngiam, Vijay Vasudevan, Benjamin Caine, Zhengdong Zhang, Hao-Tien Lewis Chiang, Jeffrey Ling, Rebecca Roelofs, Alex Bewley, Chenxi Liu, Ashish Venugopal, et al. Scene transformer: unified architecture for predicting future trajectories of multiple agents. In International Conference on Learning Representations (ICLR), 2021.
Jia Pan, Weizi Li, Wenxi Liu, Iftekharul Islam, Ke Guo, Yajue Yang, Shuai Zhang, Xuebo Ji. and Dawei Wang. Mixed crowd navigation: Perception, interaction, planning, and control. Annual Review of Control, Robotics, and Autonomous Systems, 2025.
Muleilan Pei, Shaoshuai Shi, and Shaojie Shen. Advancing aulti-agent traffic simulation via rl -sylle reinforcement fine-tuning. arXiv preprint arXiv:2509.23999, 2025.
12

---

Published as a conference paper at ICLR 2026
Zhenghao Peng, Wenjie Luo, Yiren Lu, Tianyi Shen, Cole Gulino, Ari Seff, and Justin Fu. Improving agent behaviors with rl fine-tuning for autonomous driving. In European Conference on Computer Vision (ECCV). pp. 165-181, 2024.
Zhenghao Peng. Yuxin Liu, and Bolei Zhou. Infgen: Scenario generation as next token group prediction. arXiv preprini arXiv:2506. 23316, 2025.
Jonah Philion, Xue Bin Peng, and Sanja Fidler. Trajeglish: Traffic modeling as next-token prediction. In International Conference on Learning Representations (ICLR), 2024.
Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 627-635, 2011.
Wilko Schwarting, Alyssa Pierson, Javier Alonso-Mora, Sertac Karaman, and Daniela Rus. Social behavior for autonomous vehicles. Proceedings of the National Academy of Sciences, 116(50); 11977224497, 2019.
Jiaming Song, Hongyu Ren, Dorsa Sadigh, and Stefano Ermon. Multi-agent generative adversarial imitation learning. Advances in Neural Information Processing Systems (NeurIPS), 31, 2018.
Ran Tian and Kratarth Goel. Direct post-training preference alignment for multi-agent motion generation models using implicit feedback from pre-training demonstrations. In International Conference on Learning Representations (ICLR), 2025.
Martin Treiber, Ansgar Hennecke, and Dirk Helbing. Congested traffic states in empirical observations and microscopic simulations. Physical Review E, 62(2): 2000.
Mingyi Wang, Jingke Wang, Tengju Ye, Junbo Chen, and Kaicheng Yu. Do llm modules generalize? study on motion generation for autonomous driving. arXiv preprint arXiv:2509.02754, 2025.
Waymo. Sim agents - 2025 waymo open dataset challenge leaderhoard. ggen/Chaallenges/2255//
com/
agents/ 2025, Accessed: 2025-09-24.
Hua Wei, Chacha Chen, Chang Liu, Guanjie Zheng, and Zhenhui Li. Learning to simulate on sparse trajectory data. In European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD, pp. 530-545, 2021.
David Wilkie, Jason Sewall, Weizi Li, and Ming C. Lin. Virtualized traffic at metropolitan scales. Frontiers in Robotics and AI, 2:11, 2015.
Wei Wu, Xiaoxin Feng, Ziyan Gao, and Yuheng Kan. Smart: scalable multi-agent real-time mo- tion generation via next-token prediction. Advances in Neural Information Processing Systems (NeurIPS), 37:1 144114444-11107,, 2024.
Haojie Xin, Xiaodong Zhang, Renzhi Tang, Songyang Yan, Qianrui Zhao. Chunze Yang, Wen Cui, and Zijiang Yang. Litsim: A conflict-aware policy for long-term interactive traffic simulation. In International Conference Intelligent Transportation Systems (ITSC), pp. 3359-3364, 2024.
Danfei Xu, Yuxiao Chen, Boris Ivanovic, and Marco Pavone. Bits: Bi-level imitation for traffic simulation. In International Conference Robotics and Automation (ICRA), pp. 2929-29366 2023.
Xintao Yan, Zhengxia Zou, Shuo Feng, Haojie Zhu, Haowei Sun, and Henry X Liu. Learning naturalistic driving environment with statistical realism. Nature Communications, 14(1):2037, 2023.
Fan Yang, Alina Vereshchaka, Changyou Chen, and Wen Dong. Bayesian multi-type mean field multi-agent imitation learning. Advances in Neural Information Processing Systems (NeurIPS), 33: 2020.
Lantao Yu, Jiaming Song, and Stefano Ermon. Multi-agent adversarial inverse reinforcement learning. In International Conference on Machine Learning (ICML). pp. 7194-7201. PMLR, 2019.
13

---

Published as a conference paper at ICLR 2026
Xin Zhang, Weixiao Huang, Yanhua Li, Renjie Liao, and Ziming Zhang. Imitation learning from inconcurrent agget interactions. Conference on Decision and Control (CDC), pp. 43-48, 2021a.
Zeqing Zhang, Ruihua Han, and Jia Pan. An efficient centralized planner for multiple automated guided vehicles at the crossroad of polynomial curves. IEEE Robotics and Automation Letters, 7 (1):398 405, 2021b.
Zhejun Zhang, Peter Karkus, Maximilian Igl, Wenhao Ding, Yuxiao Chen, Boris Ivanovic, and Marco Pavone. Closed-loop supervised fine-tuning of tokenized traffic models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5422-5432, 2025a.
Zhiyuan Zhang, Xiaosong Jia, Guanyu Chen, Qifeng Li, and Junchi Yan. Trajtok: Technical report for 2025 waymo open sim agents challenge. arXiv preprint arXiv:2506.26618, 2025b.
Jianbo Zhao, Jiaheng Zhuang, Qibin Zhou. Taiyu Ban, Ziyao Xu, Hangning Zhou, Junhe Wang. Guoan Wang, Zhiheng Li, and Bin Li. Kigras: Kinematic-drive generative model for realistic agent simulation. IEEE Robotics and Automation Letters, 2024.
Guanjie Zheng, Hanyang Liu, Kai Xu, and Zhenhui Li. Learning to simulate vehicle trajectories from demonstrations. In International Conference Data Engineering (ICDE), pp. 1822 1825, 2020.
Zikang Zhou, HU Haibo, Xinhong Chen, Jianping Wang, Nan Guan, Kui Wu, Yung- Hui Li, Yu-Kai Huang, and Chun Jason Xue. Behaviorgpt: Smart agent simulation for autonomous driving with next-patch prediction. Advances in Neural Information Processing Systems (NeurIPS), 37: 79597-79617, 2024.
Brian D Ziebart, Andrew L Maas, Andrew Bagnell, and Anind K Dey. Maximum entropy inverse reinforcement learning. AAAI Conference on Artificial Intelligence, volume pp. 1433-1438, 2008.
14

---

Published as a conference paper at ICLR 2026
A IMPLEMENTATION DETAILS
We pretrain the policy network with BC for 32 epochs following CAT-K Zhang et al. 2025a taking about 44 hours, and then fine-tune with DecompGAlL for 2 additional epochs, taking about 10 hours. During tine-tuning, the map encoder is frozen to reduce GPU memory usage and the policy network and discriminator network share the map encoder, enabling larger batch sizes with negligible performance loss. Both pretraining and fine-tuning are conducted on x H800 80GB GPUs with total batch size of 80. We use Adam with weight decay of 0.01 for optimization, with a policy learning rate of x 10 during pretraining and 5 10 during fine-tuning. and discriminator learning rate x 10 4. To stabilize adversarial training, we generate fresh rollouts at each training step using the current policy, then update the discriminator and policy once. Additional hyperparameters for DecompGAlL. fine-tuning are provided table 3
Table 3: Hyperparameters for DecompGAlL Fine-tuning.
Hyperparameter
Value
MLP depth (all)
MLP width (all)
128
Network dropout (all)
PPO discount
0.99
PPO clip
0.2
PPO epochs
PPO batch size
PPO rollout length
16
GAE
0.95
Value loss weight
1 10-3
a in Waj
10
in Wij
2.5
in Xaj
in Xij
2.5
B DISCRIMINATOR GRADIENT PENALTY
To evaluate whether common gradient- pradient-penalty techniques can mitigate the instability caused by irrele- vant interactions, we apply WGAN-GP (Arjovsky et al. (017) and R1/R2 regularization (Mescheder et al. 2018) (with penalty weight fixed to to both the naive PS-GAIL baseline and our DecompGAIL. framework. Results reported in table
We find that gradient penalties do modestly improve PS-GAIL, largely by constraining the discrimi- nator's expressiveness and thereby reducing the variance of its output scores. However, with gradient penalties, the PS-GAIL. discriminator can still be misled by irrelevant neighbor-neighhoo and neighbor-map interactions, so its overall performance remains significantly below that of DecompGAlL. When adding these penalties to DecompGAIL, we observe the opposite trend: gra- dient penalties reduce overall realism despite slightly improving the kinematic metrics. This is expected- gradient penalties enforce smoothness in the discriminator's 's mapping, but critical compo- nents interactive and map-based realism such as collision and off-road likelihood are inherently non-smooth and depend on sharp decision boundaries. Penalizing discriminator gradients therefore harms the model's 's ability to detect these events, degrading the interactive and map-based based metrics.
Table 4: Effect of discriminator gradient penalties evaluated the 2% validation split.
Model
Metametris
Kinematic
Interactive 1
Map-hased
Collision Likellhood Offrand Likelihood
PS- PS-GAAL
0,7674
0.4830
0.8012
0.88641
0.9191
0.9461
WGAN-GP
0.7723
0.5011
0.8111
0.8654
0.9538
0.9322
PS-GAII. RI
0.7776
0.5011
018198
0.8114
0,9704
0.370
PS-GAIL R2
0,7706
0,5037
0.809]
0.8736
0.9484
0.9258
DecompGAlL: WGAN-GP
0.7865
0.8255
0.8894
0.9796
0.9441
Decomp(iAlL- R1
0.7561
0.5157
0,8233
0.8928
0.9734
034493
DecompGAIL-
0.7840
0.5145
0.8212
0.8900
0.9743
0.9669
DecompGAIL
15

---

Published as a conference paper at ICLR 2026
C UNFREEZING MAP ENCODER
To evaluate the effect of updating the map encoder during DecompGAIL, training, we conducted an additional experiment in which the map encoder is fine-tuned rather than frozen (see fig. 66. We observe that freezing the map encoder yields slightly better training stability and marginally higher realism.
DecompGAil [unfreezing map ancoder) DecompGAIL (freezing map encoder)
0.54
0.792
0.52
Realism MetaMetric
Policy Disciminater
0.788
0.50
0.786
0.48
0.782
2000
4000
600D
6000 10000
12000
14000
0.780
0
2000
4000
6000
8000 10000
12000 14000
Global Step
Global Step
Figure 6: Effect of freezing vs. fine-tuning the map encoder. Solid lines show the mean discrimi- nator realism scores, with shaded areas indicating standard deviation. The realism meta-metric is evaluated on the 2% validation split. Freezing the map encoder results in lower score variance and improved simulation realism throughout training.
D ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. No human-subjects or animal experimentation was involved. All datasets used primarily the Waymo Open Motion Dataset (WOMD) were accessed and processed in compliance with their respective licenses and usage guidelines, with no attempt re-identification extraction of personally identifiable information. We took care assess and mitigate potential biases in data and evaluation, and found no evidence of discriminatory outcomes within the scope of our experiments. No experiments security concerns. We are committed to transparency and integrity throughout the research process, including clear reporting methods, metrics.
conducted that could raise privacy or
E THE USE OF LARGE LANGUAGE MODELS (LLMs)
We used large language model (LLM), specifically OpenAl's ChatGPT-5, writing assistant to polish and refine the presentation of our paper. The LLM was employed for grammar correction, clarity improvements, style adjustments, and consistency of terminology across sections. Importantly, all technical content, research ideas, model design, experiments, and analyses were conceived, implemented, and validated entirely by the authors. The LLM did not contribute to research ideation, algorithm development, experiment design, interpretation of results. Its role was limited to improving readability and presentation quality.
F REPRODUCIBILITY STATEMENT
We have taken multiple steps to ensure the reproducibility of our work. Our dataset and pre- trained backbone are the same those provided in the open-source CAT-K repository (https: catk/tree/ ma main) Detailed descriptions of the model architecture included in Section A and in the appendix. We also report the official
/ //github. com/ cwm/No and all hyperparameters performance of our method on the public WOSAC leaderboard, which enables transparent comparison against other approaches.
16

---

Published as a conference paper at ICLR 2026
G REALISM SCORE VISUALIZATION
In this section, we visualize the ego vehicle's scene realism and interaction realism scores produced by our decomposed discriminator, for both successful rollouts and two failure cases.
G.1 SUCCESSFUL CASES
We present two successful rollouts illustrating both straight-road and intersection driving and inter- section. As shown in fig. 7 the realism scores remain close to 0.5 throughout the rollout, and all agents drive without collisions or off-road behavior.
Figure 7: Straight driving case. The ego vehicle is shown with cyan edges; its color encodes the scene realism score. Surrounding vehicles are colored by their interaction realism with respect to the ego. The realism scores remain near 0.5, and all agents drive smoothly without collisions or off-road deviations.
fig. the simulated ego vehicle chooses turning maneuver while the ground-truth agent continues straight through the intersection, demonstrating the behavioral diversity captured by our learned traffic model. The slightly lower scene realism score in this case likely due to low kinematic likelihood.
Figure Intersection case. The ego vehicle is shown with cyan edges; its color encodes the scene realism score. Surrounding vehicles are colored by their interaction realism with respect to the ego. The simulated ego vehicle chooses a turning trajectory, whereas the ground-truth agent proceeds straight through the intersection. This illustrates the diversity of behaviors produced by learned model. The slight decrease in scene realism mainly reflects kinematic deviations rather than unrealistic maneuvers.
17

---

Published as a conference paper at ICLR 2026
G.2 FAILURE CASES
Figure D shows failure case in which the ego vehicle drifts outside the drivable area. As expected, discriminator assigns very low realism score those timesteps, correctly indicating the implausible off-road behavior.
Figure 9: Off-road failure case. The ego vehicle is shown with cyan edges; its color encodes the scene realism score. Surrounding vehicles are colored by their interaction realism with respect the ego. In the middle and right frames, the ego vehicle violates the road boundary (purple color line) and leaves the drivable surface. Our discriminator correspondingly outputs very low scene realism score, accurately detecting the off-road behavior.
Figure I0 presents failure case where the ego vehicle collides with neighboring agent. Here, our discriminator produces a very low interaction realism score, correctly flagging the implausible collision event.
Transe
Figure 10: Collision failure case. The ego vehicle is shown with cyan edges; its color encodes the scene realism score. Surrounding vehicles are colored by their interaction realism with respect the ego. In the left frame, the ego vehicle makes contact with another vehicle. The discriminator assigns a very low interaction realism score this agent pair, properly reflecting the implausibility of the collision.
18


Reviews:
[{"review_id": "R0rYHg8V88", "reviewer": "Reviewer_bk4B", "summary": "This paper addresses a key limitation in multi-agent Generative Adversarial Imitation Learning (GAIL) for traffic simulation, where existing discriminators penalize realistic agents simply because other nearby agents behave unrealistically. The authors call this phenomenon irrelevant interaction misguidance.\nTo solve this, they propose DecompGAIL, which decomposes the discriminator into two terms: ego–map realism (how the ego vehicle interacts with the static environment) and ego–neighbor realism (how it interacts with relevant nearby agents). Irrelevant neighbor–neighbor or distant-agent interactions are filtered out through distance-weighted aggregation. In addition, a social PPO reward encourages coherent scene-level realism by adding distance-weighted neighbor rewards to each agent’s objective. Experiments on the WOMD Sim Agents 2025 benchmark show that DecompGAIL improves realism metrics and training stability compared with prior multi-agent GAIL baselines.", "questions": "How sensitive is performance to the decay parameters $\\alpha$ and $\\beta$ for interaction weighting and social rewards?\n\nWhen freezing or fine-tuning the map encoder during discriminator training, how does stability or realism change?\n\nCould the social reward cause feedback loops where agents learn to exploit mutual rewards without improving realism?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Clearly identifies and mitigates the irrelevant interaction misguidance problem.\n\n- The decomposition is simple, computationally efficient, and compatible with existing frameworks.\n\n- Demonstrates improved realism and stability on public benchmarks with comprehensive ablations.\n\n- Offers a general recipe that can transfer to other multi-agent imitation settings.", "weaknesses": "- Comparisons omit Wasserstein or gradient-penalized discriminators, making it unclear whether decomposition alone drives the gains.\n\n- The social reward could unintentionally amplify correlated neighbor behaviors.\n\n- The related work section omits several closely related papers:\n\nA. Kuefler, J. Morton, T. Wheeler, and M. J. Kochenderfer, “Imitating Driver Behavior with Generative Adversarial Networks,” IEEE Intelligent Vehicles Symposium (IV), 2017, pp. 204–211.\n\nR. P. Bhattacharyya, B. Wulfe, D. J. Phillips, A. Kuefler, J. Morton, R. Senanayake, and M. J. Kochenderfer, “Modeling Human Driving Behavior through Generative Adversarial Imitation Learning,” CoRR, 2020.\n\nH. Chen, T. Ji, S. Liu, and K. Driggs-Campbell, “Combining Model-Based Controllers and Generative Adversarial Imitation Learning for Traffic Simulation,” IEEE ITSC 2022, pp. 1698–1704.\n\nK. Brown, K. Driggs-Campbell, and M. J. Kochenderfer, “Modeling and Prediction of Human Driver Behavior: A Survey,” arXiv:2006.08832, 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5I0ctljUFn", "reviewer": "Reviewer_sEmo", "summary": "This work proposes DecompGAIL, which aim to address the instability of the Generative Adversarial imitation Learning. The authors proposes to decomposes the realism to ego-map and ego-neighbor to avoid weakly relevant neighbor–neighbor interactions. Alghough it shows better training stability, the model is built upon a very strong pretrained backbone, and the results are saturated (only minimal gain) on the overall realism. The authors should conduct a more comprehensive study of why do we need Generative Imitlation Learning framework, and highlight how this GAIL-finetuning provided a different aspects, dimensions of traffic simulation", "questions": "- How does this work compared to supervised fine-tuning, what are the advantages of using GAIL, given that the gain is very minimal  compared to the original SMART 0.05 \n- The definition and handling of “ego” vs “neighbor” agents is unclear: during training, are all agents’ policies updated simultaneously, or is only the ego agent’s policy learned while neighbors remain fixed? How does this affect memory usage and training dynamics (especially when using a decomposed discriminator)?\n- Please address the weakness sections", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The paper is well-written and the proposed Decomposed GAIL method is more stable than prior GAIL baselines.\n- DecompGAIL achieves competitive results on the Sim Agent Challenge 2025, though the gain is very small.\n- The main potential of this work is the discriminator, which can provide a useful realism signal compared to prior metrics (see weakness section)", "weaknesses": "- Qualitative results are not interesting and are very similar to previous works\n- Overall, DecompGAIL’s advantage compared to prior works is unclear, given that the performance improvements are very small (± 0.01 realism score).\n- The pretrained backbone already attains a high discriminator score (~0.5) from Figure 3, which suggests the added GAIL module provides minimal gain.\n- I suggest the authors to start with a underperformed backbone w/ GAIL finetuning, and the main potential of this work is to  showcase the discriminator:whether it provides a uesful realism signal, compared to prior metrics such as Waymo Sim Agents Challenge\n- In Sec 5.4, the authors claimed that BC suffers from a covariate shift problem by mentioning higher collision likelihood and very minimal gain on realism. This may not hold true as the Sim Agent Challenge measures distributional realsim instead of rule satisfaction; see [1].\n\n\n[1] Wang M., Wang J., Ye T., Chen J., Yu K. (2025). Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving. CoRL.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dXCFpYwafK", "reviewer": "Reviewer_dJoy", "summary": "* This paper addresses the problem of training instability in multi-agent GAIL for realistic traffic simulation.\n* It identifies irrelevant interaction misguidance as a key issue, where the discriminator incorrectly penalizes a realistic ego agent because of unrealistic interactions among its neighbors.\n* The proposed solution DecompGAIL introduces a decomposed discriminator architecture that explicitly separates realism into (1) ego-map realism and (2) pairwise ego-neighbor realism.\n* This decomposed design structurally filters out the misleading neighbor-neighbor and neighbor-map interaction signals that cause instability.\n* The method is enhanced with a social PPO objective that augments the agent’s reward with a distance-weighted sum of its neighbors' rewards, promoting overall population realism.\n* The method achieves SOTA on WOSAC.", "questions": "* Are there failure cases or poor quality examples that could be included?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* DecompGAIL achieves state-of-the-art results on WOMD sim agents and demonstrates significantly improved training stability.\n* Irrelevant interaction misguidance is a novel problem that the authors discovered. GAIL has been notoriously difficult to train.\n* The authors show strong empirical validation that the proposed approach significantly improves training stability and in addition achieves strong results on WOSAC.\n* The ablation study validates that each proposed component contributes to the strong results.", "weaknesses": "* While the solution of decomposing the discriminator is sound it is engineered and strongly dependent on the input features that are used by the sim agent model. For example, it’s unclear how this approach could be applied to a simulator model that leverages images and sensor sim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)