JointDiff: Bridging Continuous and Discrete in Multi-Agent Trajectory Generation
Authors: Guillem Capellera
Luis Ferraz
Antonio Rubio Romano
Alexandre Alahi
Antonio Agudo
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-Agent
Diffusion
Controllable
Trajectory
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Generative models often treat continuous data and discrete events as separate processes, creating a gap in modeling complex systems where they interact synchronously. To bridge this gap, we introduce $\textbf{JointDiff}$, a novel diffusion framework designed to unify these two processes by simultaneously generating continuous spatio-temporal data and synchronous discrete events. We demonstrate its efficacy in the sports domain by simultaneously modeling multi-agent trajectories and key possession events. This joint modeling is validated with non-controllable generation and two novel controllable generation scenarios: $\textit{weak-possessor-guidance}$, which offers flexible semantic control over game dynamics through a simple list of intended ball possessors, and $\textit{text-guidance}$, which enables fine-grained, language-driven generation. To enable the conditioning with these guidance signals, we introduce $\textbf{CrossGuid}$, an effective conditioning operation for multi-agent domains. We also share a new unified sports benchmark enhanced with textual descriptions for soccer and football datasets. JointDiff achieves state-of-the-art performance, demonstrating that joint modeling is crucial for building realistic and controllable generative models for interactive systems. [Project](https://guillem-cf.github.io/JointDiff/)

Published as a conference paper at ICLR 2026
JOINTDIFF: BRIDGING CONTINUOUS AND DISCRETE IN MULTI-AGENT TRAJECTORY GENERATION
Guillem Capellera 1,2,3
Luis Ferraz'
Antonio Rubio
Alexandre Alahi
Antonio Agudo
Kognia Sports Intelligence
Visual Intelligence for Transportation, EPFL 3 Institut de Robòtica i Informàtica Industrial, CSIC-UPC
ABSTRACT
Generative models often treat continuous data and discrete events separate processes, creating a gap in modeling complex systems where they interact syn- chronously. To bridge this gap, we introduce JointDiff, novel diffusion frame- work designed to unify these two processes by simultaneously generating con- tinuous spatio-temporal data and synchronous discrete events. We demonstrate its efficacy in the sports domain by simultaneously modeling multi-agent tra- jectories and key possession events. This joint modeling is validated with non- controllable generation and two novel controllable generation scenarios: weak- possessor- possessor-guidaacee, guidance, which offers flexible semantic control over game dynamics through simple list of intended ball possessors, and lext-guidance, which enables fine-gras grae-graddd, language-driven generation. To enable the conditioning with these guidance signals, we introduce CrossGuid, an effective conditioning operation for multi-agent domains. We also share a new unified sports benchmark enhanced with textual descriptions for soccer and football datasets. JointDiff achieves state- of-the-art performance, demonstrating that joint modeling crucial for building realistic and controllable generative models for interactive systems. Project
INTRODUCTION
Modeling the dynamies of multi-agent systems is fundamentally challenging when continuous mo- tion is tightly coupled with discrete, state-altering events. This interplay is critical in domains like autonomous driving and robotics, but finds a particularly rich and demanding testbed in team sports. Here, the continuous trajectories of players are synchronously intertwined with discrete events like passes and possessions. Generating realistic sports gameplay therefore requires a model that can jointly represent these two modalities. However, existing generative models often fall short by treat- ing these components in isolation. This can lead to physically implausible generations, such as unrealistic passes or flawed ball-possessor interactions (Lee et al. 20244 Capellera al.) 2025 While deterministic models have started to incorporate events (Kim et al. 2023 Capellera et al. 2024). comprehensive generative framework missing. This deficiency compounded by eval- uation protocols that rely on individual-level metrics like minimum ADE/FDE (Alahi et al. 2016). which were inherited from pedestrian forecasting and fail to capture scene-level coherence (Casas
et al. 2020 Girgis et al.] 2021 Weng et al. 2023 Capellera et al. 2025). crucial to team sports.
To address this gap, we turn the expressive power of diffusion models. While continuous diffu- sion (Ho et al.] (2000) has excelled at generating high-fidelity data like trajectories (Mao et al.) 2023
Jiang et al. [2023] Gu et al. 2022; Rempe al. 2023 Bae et al., 2024 [i al. [2023] Yang al, 2024 Capellera et al.) 2025). discrete diffusion (Hoogehoom et al. [ Austin et al. 2021) has
concurrently emerged as a potent, non-autoregressive alternative to large language models (LLMs) for structured sequence generation (Lou et al. 2023). Nascent work has begun to unify these modal- ities for static tasks such as layout design (Levi et al. 2023) and visual-language modeling (Li et al. 2025). Our key insight is to unify these two paradigms for the temporally evolving complex systems. We introduce Join a novel framework that, to the best of our knowledge, is the first to apply joint continuous-discrete diffusion to simultaneously generate spatio-temporal continuous data (tra- jectories) alongside its corresponding synchronous temporal discrete events (possession events).
* "This work performed during research stay EPFL., Laussanne (Switzerland). Contact: i ratname. 1 nast (kogni asports................... com, epf1 ch, upc.

---

Published as a conference paper at ICLR 2026
Continmous Srajectortes
JointDur
Diecrete
eveuts
2
Obervvtions
Cruddanee
Playrs U1 tevolved
Wrak-
"Flayer itatt
Text
paases to Player
T
Figure I: JointDiff. Our model jointly generates continuous trajectories and discrete events, with guidance provided through either possesoss information or natural language text. Stars refer to the initial timestep.
Beyond realism, a truly useful generative model also needs to be controllable. While diffusion- guidance methods (Dhariwal & Nichol, 2021; Ho & Salimans 2022) have been used in motion synthesis to satisfy pedestrian goals or constraints (Jiang et al.) 2023 Rempe et al.) 2023), semantic control through discrete events in multi-an aggent systems remains unexplored. Our joint framework directly enables such control using the classifier-fn freee guidance (CFG) (Ho & Salimans 2022). We introduce weak paakk--ossss--gguiieee
(WPG), a novel conditioning method that allows users to steer gameplay by simply providing an ordered list of intended ball possessors, without rigid timing con- straints. We further extend controllability to natural language via text text-guided generation, facilitated by new, curated benchmark of text descriptions for soccer and football datasets. Our approach is il- lustrated Fig. In summary, our principal contributions are: 1) A novel joint continuous- discrete diffusion framework that simultaneously generates multi-agent trajectories and synchronous discrete events, leading to more realistic and coherent scenes; 2) Enabling high-level semantic controllabil- ity in dynamic domains. We introduce two novel controllable tasks sesskk-pssssso-guuiiiiei guiddde and text-guidance) and a dedicated CrossGuid module that effectively injects conditioning signals into the structured multi-agent embedding: 3) A unified benchmark for multi-agent modeling in sports, enhanced with new text descriptions for soccer and football datasets. Our method achieves state-of- the-art results on scene-level metrics.
RELATED WORK
Trajectory Modeling. The evolution of multi-agent trajectory modeling has progressed from Re- current Neural Networks (RNNs) and Variational RNNs (VRNNs) (Zheng et al. 2016 Felsen et al.)
2018
Zhan et al.] 2019; Yeh et al., 2219) Li et al., (021), to generative models like Generative Adversarial Networks (GANs) (Gupta et al. 2018 Sadeghian et al., 2019 Fang et al. 2020) and Conditional Variational Autoencoders (CVALs) (Salzmann et al. 2020; Graber & Schwing 2020
Yuan et al. 2 Lee et al. 20222] Xu et al. 2022 Zheng et al. (024). In recent years, non-sampling
approaches built on transformers Vaswani et al.] (017). and in some cases enriched with visual data
(Saadatnejad et al. 2023 Gao al.] (0224] Wu al. (022)) have achieved notable progress in multi- modal future prediction by effectively modeling long-range spatio-tempore dependencies (Alcorn)
& Nguyen 2021 Girgis al. N Ngiam al. 2021 Lee al. (024)) Building on this progress, Denoising Diffusion Probabilistic Models (DDPMs) (Ho et al. 2020) have emerged as the state-of-
the-art for generating high-fidelity and diverse trajectories (Mao al., 2023 Diang al., 2023 Gu
et al. 2022; Rempe et al.) 2023 Bae et al., 2024 Li et al., 2023 Fu et al.) (025). This generative power also extends to tasks like trajectory completion, where the recent diffusion model U2Diff
(Capellera et al.] 2025) has surpassed prior methods based on Graph VRNNs (GVRNNs) (Omid-
shafiei et al. [022 Xu et al. 2023), GANs (Liu et al. 2019), and CVAES (Xu & Fu 2025). Notably,
U2Diff also competes against forecasting-specins architectures despite using Independent and
Identically Distributed (IID) sampling method and without using time window constraints.
Multi-agent Controllability. Diffusion models have recently been augmented with guided sam- pling to satisfy user- speriiied constraints or objectives. Existing pedestrian and autonomous driving methods (Rempe et al.) 2023] Jiang et al. 2023 Yang al., 2024), typically focus controlling individual- attributes like waypoints, speeds. or physics constraints. Similarly, human motion generation approaches (Karunratanakul al. 2023) and roboties planning methods (Mishra al.,
2

---

Published as a conference paper at ICLR 2026
2023 Fang al. 2024) often guide single agent. In contrast, work focuses controlling
a broader multi-agent system through high-level semantic directives. We adopt the CFG paradigm
(Ho& & Salimans| 2022), widely used in image and video (Rombach al. 2022 Ho et al. 2022),
to bias generation toward a user-specified sequence of possessors or a natural language description.
This allows for comprehensive control of the entire scene rather than individual agent behavior.
Joint Sariinoous-isccrfe Diffusion. Joint diffusion models for mixed continuous discrrte data an emerging research direction, with applications in static domains such as layout design (Levi et al. 2023), CAD sketches (Chereddy & Femiani 2025), and vision-language modeling (Li et al., 2025 where absorbing state diffusion (Austin et al. 2022 is commonly used for discrete variables. In contrast, dynamic domains have been underexplored. Prior work (Zeng et al.) (224)) applies the multinomial formulation (Hoogeboom et al. 2021 to temporal point processes, but it's restricted to single-instance future prediction and relies on sequential, non-simultaneous generation. We extend the multinomial formulation to general controllable dynamic domains, exemplified by the multi- agent completion task, and introduce a unified diffusion framework that simultaneously models continuous trajectories and discrete events. This formulation proves more consistent than absorbing state diffusion in temporally evolving domain, as it enables continuous refinement of discrete variables throughout the denoising process. Furthermore, we are first to incorporate high-level se- mantic controllability, such as WPG and text-guidance, for joint continuous-discret generation in dynamic domains, consistently outperforming the non-joint baseline.
DIFFUSION BACKGROUND
Diffusion models are class of generative models that learn to reverse a progressive noising process, operating in two stages: a forward diffusion process and a learnable reverse denoising process. The forward process is fixed Markov chain that gradually adds noise to a data sample Xo (Xn)-
Over S steps, the data is corrupted following a variance schedule, {B, E (0, 1)} until (Xs) resembles a simple and known noise distribution. This process is defined as: ((1ss Xo)
II. IX,, x.-1). on the initial data Xn in closed form (X, Xg). The reverse process is generative model that learns to denoise the data by iteratively reversing the forward steps. Starting with a sample from the known noise distribution, Xs, neural network, po, learns approximate the reverse transitions: (o(ss)) = pXx) IT Po (X,-1 X,).
A key property is that we can sample X at any arbitrary timestep conditioned
The objective is to train the model to generate new samples that match the original data distribution (()).. This is achieved by minimizing variational upper bound on the negative log- lieliielloood:
S
Lvb Eyl- log po (Xu X,)
> Ey (DLL(((X,-) X,, (X. X,))
X.)
(1)
where is a constant term defined by (X Xx) (Xx)) The true posterior q(X,-) X,, Xo) is tractable, which allows for direct optimization approach of the neural network approx- imating the reverse transition Pa(X. (X.-1 X.).
While the original DDPM framework (Ho et al. 2020) handled continuous data with Gaussian noise, subsequent work adapted it for discrete, categorical data (Hoogeboom et al. 2021 Austin et al.) 2021). Our work builds on multinomial diffusion, which corrupts discrete events toward uniform distribution (see Appendix for details on these frameworks and their loss terms).
4 METHOD
4.1 PROBLEM STATEMENT
We model dynamic scene composed of N agents (e.g., ball and players) over time horizon of timesteps. The state of the scene at any time is described by a combination of continuous and discrete variables as:
Continuous state: The agent's 's 2D spatial coordinates represented by tensor Y T x2 where yr,, E R² is the position of agent n at time t. The agents are indexed such that 0 refers to the ball, and (1, (1...., N 1} refers to the players.
3

---

Published as a conference paper at ICLR 2026
Discrete state: A categorical event at each timestep, such ball possession, is represented by a one-hot matrix E {0, 13 Each row e, is a one-hot vector where a value at index n indicates that agent is in possession of the ball. States e, where the ball is not possessed (e.g., during a pass or shot) are assigned to the ball's own category, index n = 0.
The complete scene described by a tuple X (Y,E), which jointly represents the spatio- temporal trajectories and discrete events. We define two generative objectives for our model: Com- pletion and Controllable Generation. The first objective to generate plausible and coherent completions of dynamic scene. Given set of partial observations X (Y EE defined by binary mask M that specifies which time steps and agents are observed, the goal is to learn a model capable of sampling from the conditional distribution p(X X The second objective extends this to controllable generation by introducing an external conditioning variable Ç (e.g., natural lan- guage text) to guide the generation process. The model must learn sample from the augmented conditional distribution p(X X g). This framework enables scene generation that is influenced not only by partial observations but also by additional data from different domains.
4.2 JOINT CONTINUOUS DISCRRET DIFFUSION
To model the joint data distribution (X)) g( Eo), we design diffusion model that simulta- neously handles both continuous trajectories and discrete events.
The forward process corrupts the initial data X, (Yo Eo) over S timesteps. We assume the noising processes for the two modalities are independent, which allows us to factorize the joint forward transition as:
Y E, Yo, Eo) = (Y,, Yo) (E, En).
(2)
This factorization enables the application of a continuous diffuuson process the trajectories and a discrete diffusion process to the events Eg. For simplicity, we assume both processes are governed by a shared variance schedule { [ The individual closed-form transitions are:
q(Y. Yo) MN(Y Yo, o, (1 -1--60 āā))I, g E Eg) = Cat(E,; G, + -11 - a.)/N).
(4)
where a, = 1 Bg and ã II 0g and Cat(:p) denotes a categorical distribution with proba- bilities p. E describes standard Gaussian diffusion process from DDPM (Ho al. 2020)) where the initial state Yo is progressively corrupted with Gaussian noise. Following Hoogeboom al. (202T), Eq. defines multinomial diffusion process, where the one-hot matrices Eg are grad- ually mixed with uniform distribution over N categories. As s + S, Y, converges to sample from an isotropic Gaussian, and E, to a sample from a uniform categorical distribution.
For the reverse process, we make the conditional independence assumption at s - 1, allowing the joint posterior to be factorized as:
(Y -- E. E., E'...""..9
(Y Y., .9) (E Y. E.. g)).
(5)
Note that the model learns the dependencies between the continuous and the discrete modalities because the reverse network pe is conditioned on the full state X, = (Y a. E.). The reverse process is parametrized with a single neural network with two prediction heads. The network takes the noisy state (Y, E.). the denoising step s, the partial observations = and optional guidance as input.
A regression head predicts the noise added to the trajectories, denoted as E,, s, x(",))
6ø YY,
To(Y. Y.,, 3, _ ,).. A classification head predicts the original event probabilities, E. = No
The continuous reverse transition is defined as Gaussian distribution:
Pa PY (Y,-1 Y. E., 0 . (Y -Y..1i11 p4 (Y,, E,, X (.xxx))). 09)
(6)
where the variance of is a non learnable hyperparameter, typically set to 1- B,, and the mean 14g is computed from the predicted noise using the standard DDPM parametrization as ue (Y.
Y. VI-a, ( (Y,
Va,
4

---

Published as a conference paper at ICLR 2026
The discrete reverse transition is derived by plugging the network's prediction En into the true posterior ((E,1 E,, Eg) for steps 8 2 2. while for the final step (s = 1) we directly use the categorical distribution with parameter Eg. Specifically, we have for s = 11 and 2 2, respectively:
pe ( Y E1, x 9) Cat Eo) and pe (E Y Es, X .g) qE..1 1 Es Eo).
The posterior is categorical distribution ((E,-1 E,,, Ea) Cat(E,,11 1;, 0 post EE, En)) whose probabilities are defined as:
8 post Egg Eg)
0
166 0, and θ = la,E, + (1 (0.0-aa)NN [ [a--1E0 + (1--ā,-)/NN, ==
Training Objective. Our model is trained end-to-end by minimizing a joint objective derived from Eq. [ Since the forward process q acts independently on each modality, the true posterior also factorizes:
q (X,-11 X., Xo) s Y, Yo) (E..1 E, Eo).
The key property of the KL divergence is that it decomposes over factorized distributions. This allows the variational bound from Eq. [ to be separated into continuous and discrete terms:
Ca B log Y1. E,, X
Σ Dru(a(Y.-1 Y. Yo) Po(Y, 1 E. x ..)) ..))
+ B,
(u po BI, T .9) Σ Dru(a(E.-1 E., Bo) (p, E.,
x 99) 2.y + L.b
For the continuous part, we use the simplified objective common in DDPMs, reducing the objective CX to LY = E,,x,.e -olcc-co cY s, X 1)] where is the Gaussian noise injected at simple
step For the discrete modality, we retain the exact variational form LVP (see Appendix A for more details on how to compute each loss term). Our proposed resulting training objective is the weighted combination:
Cjoint
CY simple + X C
(8)
where À is balancing hyperparameter chosen so that both modalities contribute comparably during optimization. Instead of uniform sampling, we use the importance sampling method proposed by
Nichol & Dhariwal 1 to estimate the expectation over the timestep during training.
Joint Sampling. During inference, we generate samples by starting with pure noise and iteratively denoising it. To accelerate this process, we propose hybrid sampling procedure that uses different
strategies for each data type. For the continuous trajectories Y. we employ the deterministic De-
noising Diffusion Implicit Model (DDIM) sampler (Song et al. 2020) in Capellera et al. (2025)
It allows for larger jumps in the denoising process. The update rule to go from step 8 to 8 is:
Y Y = Va
(V1- - āo-5 V
06-6 VI- co co E, X .9).
For the discrete events, use the standard stochastic sampler (Hoogeboom al. 2021). At each step s, the network's classification head predict the original event distribution Eg To (Y,,. E,, s, X 9). We then sample E,-1 from the posterior q(E.11 E.).
Beta Schedule. To align the continuous and discrete sampling processes and improve model ac- curacy, we employ a hybrid schedule where the total number of discrete steps, denoted as 5d is reduced (S& < S). Following I et al. (2023), we align the discrete steps (s²) with the continuous ones 8 using sd [s. /S).. We empirically found that a good choice to match the DDIM skipping step Ç with the ratio S/S0t. See an empirical evaluation at Appendix D.3.1
Controllability. For controllable generation, we utilize the CFG (Ho & Salimans 2022) training approach. During training, we randomly drop the condition G with probability of 25%. which allows the model to learn to denoise both with and without the conditioning information. For non- controllable generation, the model is trained without any conditioning, During inference, we found that we can achieve effective guidance by using single forward pass with the conditional output. See empirical evaluation at Appendix D.3.3
5

---

Published as a conference paper at ICLR 2026
X
(pgiionaa)
Guro
Social Tempporal Dlock
Nash
MILA
CroeeGuad WPG
Placc Ti.If Inveived in pooserse
Temporral Manoba
[ 256
KV
GOLOCOD (7.M3
Purt
Ball Playes
"Rayer statt was and paorr
TS-Bamn
Player
X
2:4
206]
EB
CrossGrad
G
G
CroeeGrad
L.
Af'
Linear
DILLA
M
Join DDie
x Social
KV
M
Tranafonuser
Ball Player
Pr.N
Player
[M.
The overall pipeline of our JointDiff model, which takes as input the noisy states X,, observed states X mask M, and optionally (referred with dashed connections) the encoded guidance signal G. Stars (+²) refer to the initial timestep, t O. The model processes these inputs through two SemiorlTemporal Blocks and outputs the predicted Gaus- sian noise eg for trajectories and the event probability distribution πe. Right: Detailed view of Social-Temport Block featuring our proposed CrossGuid module. The module has two distinct implementations corresponding to different guidance modalities (WPG and Text). The red line (-) in Docial-Tempol Block indicates the data flow for non non-controllable generation, where CrossGuid bypassed. An extended diagram available in Fig.
Figure 2: Model Architecture. Left:
4.3 MODEL ARCHITECTURE
Our model, JointDiff, builds upon the U2Diff architecture (Capellera et al., 2025). which processes multi-agent trajectories using Social-Temporal Blocks. Each block comprises Temporal Mamba module (Gu & Dao, 2023) for modeling individual agent dynamics and Social Transformers en- coders (Vaswani et al. 2017) for capturing inter-agent interactions. We modify this foundation to create joint diffusion process capable of controllable generation.
As shown in Fig. 2 left, the model takes as input the noisy state X, E Tx formed by concate- nating the continuous trajectory coordinates Y, and discrete event indicators E, along the feature dimension, and the observed state X E IR xTXNx (constructed similarly), along with the binary mask M E R processes these through two Social-Temee Social-Temporal Blocks and produces two outputs using:
A regression head that predicts the Gaussian noise ep for the continuous trajectories.
A classification head that predicts the probability distribution for the original discrete events, yielding Eg.
To enable controllable generation, we introduce the CrossGuid module, which injects an external guidance signal 9 into the denoising network. This signal is first encoded into a conditioning tensor where L is the sequence length and d is the feature dimension, both specific to the guidance modality. During training, conditioning dropout (g 0) is performed by setting G zero tensor. Refer to Appendix B for architecture details.
G R signal
4.3.1 CROSSGUID FOR CONTROLLABLE GENERATION
The CrossGuid module is integrated within each Secial-Temporal Block, situated between the Temporal Mamba and the first Social Transformer. refines the intermediate representation H E T x256 (obtained after processing by the Temporal Mamba and Layer Normalization) using the conditioning tensor G. The operation defined as H' = CrossGuid G) = + AH, where the residual update AH is computed via Multi-Heal Attention (MHA) mechanism /Vaswani eet al. 2017). The implementation varies with the guidance modality, as detailed below and shown in Fig. 2 right, where H H²T E NxTx250 are the transposed tensor of H and H', respectively.

---

Published as a conference paper at ICLR 2026
Weak-Possessor--uuiee (WPG). This modality conditions generation on sequence of ball pos- sessors. The guidance signal Gweg is a sequence [7], R22, ,,, where each n, E {I...., - denotes player index. This sequence is encoded as one-hot matrix G R R
Key/Value (K, V). Each possessor index in G is mapped through a learnable agent em- bedding layer Nemb E TR x256, yielding K = V = Wemb(G) E TR. Lx266,
Query (Q), The query is derived solely from the ball's intermediate representation: Q = [[: RT22556 where index 0 corresponds to the ball.
Positional Encoding. ID sinusoidal positional encodings (Vaswani et al. 2017 are added to Q (along the temporal dimension 7') and to K, V (along the possession sequence dimen-
sion L.)
MHH and Update. The update is applied only to the ball's trajectory: H'|: 0] H .0] + MHA(Q, K, V).
Agent Embedding Addition. To facilitate social reasoning, the learnable agent embedding for each player n (1......N NN 1} is added to their respective representation: H' n] H[:.n] m) + (n).
Text-Guidance This modality conditions generation on a natural language prompt Grest. The text tokenized and encoded using a frozen, pre-trained T5-Base Encoder (Raffel et al. 2020), producing G E R 768 where is the number of tokens and = 768.
Key/Value (K, V). The text embeddings are projected to the model's dimension: K =
V LG) LX256
Query (Q). The query is formed from the representation of all agents. To distinguish
between agents, the learnable agent embedding is added before projection: Q/: n] = H
.n] + (n) E Tx26 for each agent
Positional Encoding, ID positional encodings are added to each agent's query Q n| (along time) and to K, V (along the text token sequence).
MHA and Update. The MHA operation is performed independently for each agent against the shared textual context. The update is applied to all agents: H[[:n] n] = H|: n1] + MHA(Q[, n], K, V) for n E (O,...,, 13.
5 EXPERIMENTS
Continuous trajectories (Y). We validate JointDiff on three public sports datasets: NBA, NFL and Bundesliga. The NBA dataset uses the widely adopted SportVU date[] with the splits from Mao let al. (2023) (32.5k training / 12k testing scenes). Each scene spans 6 seconds (T = 30 timesteps, Ips) with N 11 agents (the ball and 10 players). The NFL dataset comes from the Big Data Bow B following the splits of Xu & Fu (025) (10,762 training/ / 2,624 testing scenes). Each scene covers seconds (T = 50, 10 fps) with N 23 agents (the ball and players). Finally, the Bundesliga dataset is curated from the German soccer leagud) [Bassck et al. 2025), containing 2.093 training and 524 testing scenes from 7 matches. Scenes with fewer than N = 23 agents or out-of-play were removed. Each scene spans 6.4 seconds (T 40 timesteps, 6.25 fps) with N 23 agents, and the training set is augmented with 180 rotations, doubling its size.
(
Possessor event (E). To compare with methods that do not model events, extract possessor events from trajectories Y using a simple heuristic: a player possesses the ball if it is within 1.5 meters (see Appendix C. When multiple players are in range, the closest is chosen; if none, assign the ball as the possessor, acting as no possessor class (e.g., during pass or shot).
Guidance data (g). From the possessor events, we generate the soak--ossessor-pee guiddance signal (GwPa), a sequence of unique consecutive players filtered from the ground-truth events (e.g.. E [1, 1, 1, 0, O, 0, 0, 3, 3,3) yields GwrG [1,3]). We also create natural language descriptions (9eea) for NFL and Bundesliga using public metadata. NFL events are aligned with tracking data via

---

Published as a conference paper at ICLR 2026
Table 1: Completion Generation. The table reports results for our JointDiff and state-of-the-an baselines, solving the completion generation task. We report performance metrics for two distinct tasks: Future Generation (top) and Imputation Generation (bottom). Performance metrics computed over 20 generated modes, using min azg, with the exception of the uni-modal method, which are noted as having mode the IID column. The Gen column specifies whether a model generative.
NFL (yards)
Bundesliga (meeter)
NBA (meters)
Method
Gen
IID
SADEL
SADEL
SADEL
SPDEL
GroupNet
V
4,42 5.33
10.01 12.18
4.78 5.76
9.58 / 11.63
2.12 2.84
3.72/ 5.15
AutoBots 1unn
X
3.02 / 4.82
6.33 / 10.68
3.33 5.93
5.57 / 11.46
1.75
2.73/4.71
LED eVeRs
/
3.48 / 4.12
7.95/9.63
3.89 /4.58
8.06/9.74
2.77/
3,25/4.45
LED
/
1,63/3883
2.99/6.00 6,03
MART RECY24
X
2.55 4.22
5.99 10.31
2.50 1.50/4.1
5.06 9.00
1.52/2.46
2.77 4.77
MoFlow events
2.33 4.02
5.51
2.51 / 4.21
5.08 9.24
1.52/2.42
2.73 4.64
U2Diff evaRs
V
2 3.74
/
2.69 /4.21
5,46/ 9.44
1.48/2.12 2.12
2.68
4.14
JiintDiff (Ours)
V
2,36 3.40
5.53/ 8.40
2.47 / 3.06
5.02 8,29
1,39 / 2.01
2 3.95
TranSPORT
Teess Tmer
X
1.27
1.45
0.71
Sports-Traj xuRz
/
2.28 2.29
2.75 / 2.75
1.19/ 1.20
U2Diff everss
1.04 1.36
0.83
JointDiff (Ours)
0.84 1.03
0.91 / 1.18
Table 2: Controllable Generation. The table reports results for our JointDiff and variant without our joint framework (Ours w/o joint), solving the controllable future generation task. It is also included performance the non-controllable task (w/o 9) as well as two controllable tasks: WPG (w Gwpa) and text-guidance (w Grext). Performance metrics computed over 20 generated modes, using min aug for SADE and SFDE and max ovg for Acc.
Method
(yards) SFDE
Bundesliga (meters) SFDE
(meters)
SADE
SADE
Ours w/o joint
w/o G
2,42/3,57
5.67 8.722
.76
2.60 3.99
5,30 8.95
.67
1.46
24.4/4.19
.74/ 86/ /66
GwrG w Gieu
2.37/3.49
5.51
800.59
2.20/ 3,07
4.75/
.73/.50
1.29 / 1.91
2,27 /3.74
3.33
5.40 8.25
80 63
2.96
4.18/6.15
778/.5
Ours
w/o
3.36/3.40
.78/ .54
5.02/ 3.66
68 88/.3
1.39 /2001
/
.75 .44
w Gwir
2.29
57.9
844 66
2.13/2.85
6.22/6.16
.77
1.24 1.81
/
887.67
Fizst
2.19/3.09
5.04 /7.52
.86
2.08 /2.72
4.09
.80 /.5
possessor information, while Bundesliga follows Kim et al. (2025). This fine-grained conditioning provides more control than GWPG. Refer to Appendix [C22 for more details. The code to generate the datasets and the guidance data will be released jointly with unified dataloader to constitute an cssylu benchmark for future works.
Implementation. We use S 50 diffusion steps for continuous and St 10 for discrete data, with a shared quadratic noise scheduler from 30 = 10-4 to Bg 0.. (Capellera et al. 2025 Tashiro et al.) 2021]. The discrete loss coefficient is set to A = 0.1, which provided the best trade- off between trajectory and event accuracy (see Appendix D.3.2). Sampling employs DDIM with a skip interval
5 and an extra denoising step at = 1. yielding 11 steps: (50, 45...., .,. 5, 6,11,01 1). Training runs for 100 epochs (NBA/NFL) and 200 epochs (Bundesliga) with batch size 16; learning rate 10-3 is halved every 20 (NBA/NFL)) or 40 (Bundesliga) epochs. The model uses hidden size of 256 and 8 attention heads in all multi-head attention layers, while the Social Transformer employs 1024-dimensional feedforward layer. All models are trained on single RTX A6000.
5.1 COMPLETION GENERATION
We evaluate completion on two sub-tasks: future generation and imputation generation. For each scene, sample 20 modes and report both minimum (min) and average (aug) errors, where min reflects the best-case generation quality and avg measures distributional fidelity.
In future generation, models observe 10 frames and predict the future. We report scene-level errors as SADE and SFDE (Casas et al., 2020). Girgis et al.) [202]] Weng et al.) [023] Capellera et al. 2025). Prior work can be grouped by how multiple futures are produced. III models (GroupNeet
8

---

Published as a conference paper at ICLR 2026
Xu et al. 2022). LED (Mao et al., )023), U2Diff Caaelleraa et al. 2025 ours) draw modes independently from random noise. As image generation, this encourages coupling-fidelitt with the real data distribution and usually yields stronger atg performance. In contrast, non-IlD models (AutoBots (Girgis et al. 2021), LED (Mao et al. 2023), MART (Lee et al., 2024), MoFlow (Fu et al., 2025g) generate multiple correlated modes in single forward pass, which often improves min metrics empirically. As shown in Table toop. JointDiff achieves SOTA across datasets in avg, while, notably competing with min metries against non-IlD approaches. We also note whether model is generative or not. Refer to Appendix D.I for the implementation details of these baselines.
In the imputation generation setting, models are provided with the first frames and the final frame, and must predict the missing in-between trajectories. We evaluate against the IID models Sports- Traj (Xu & Fu, 2025). U2Diff, and the deterministic uni-modal TrRnSPORTmer (Capellera et al. 2024), reporting the SADE metric. As shown in Table 1 hottoom, our method achieves the SOTA this benchmark. The results show that diffusion- al.
models, such as U2Diff (Capellera) 2025 and Ours, consistently outperform the CVAE-based model Sports-Tres (Xu & Fu 2025 which suffers from mode collapse. Additional quantitative and qualitative results are provided in Appendix D.2 and D.5.1 respectively.
To assess perceptual quality, we conduct a human evaluation using pairwise comparisons the NBA future generation task. Fifteen participants judged fifteen random pairs drawn from JointDiff, ab- lated variant without joint modeling ("w/o joint"),
Wins
Ties
T.eesss
Proportion of Outromes
100%
n
43
21
50
23
26
44
12
21
U2Diff, MoFlow, and Ground Truth (GT) (interface shown in Appendix D.2.Ip. For fairness, each model generated 20 modes from the same past, and the sample with the lowest SADE was used. Results Fig. 3 show that JointDiff most preferred, out-
50%
80
24
65
33
0%
MoFFow
U2Diff
w/o joint
GT
performing MoFlow (80%). U2Diff (65%), and w/o
joint (53%). Removing ties increases the win rate over the ablated variant to 67%, indicating clear per- ceptual gains from joint modeling. JointDiff loses to GT in 44% of comparisons, with 24% ties, suggest- ing that many generated trajectories are difficult for users distinguish from real ones.
Figure 3: Human evaluation on NBA future generation. The histogram reports the pro- portions of wins, ties, and losses for JointD- iff against each baseline, with denoting the number of pairwise comparisons.
5.2 CONTROLLABLE GENERATION
We evaluate controllability by generating 20 future trajectories conditioned on ground -rrth guid- ance signals (9). For the trajectories (Yo), we report the (min avg) for SADE and SFDE. To assess the possessor prediction, we report accuracy (Ace) as the match between arg max (Eg) and the ground truth Eg, giving both maximum and average (max / avg) over the generated modes.
Table compares non-controlled and controlled tasks and evaluates our joint modeling approach against variant modeling only continuous trajectories ("Ours w/o joint"). Controlled tasks con- sistently outperform their non-controlled counterparts ("wlo G"), confirming the effectiveness of CrossGuid. Conditioning textual signals Gext further improves performance over GwrG. showing the benefit of fine-grained guidance. JointDiff surpasses Ours w/o joint overall across all datasets and metrics, validating the advantage of jointly modeling continuous trajectories and discrete possession events and improving denoising in both controlled and non-controlled settings. Figure 4 proviies qualitative comparisons using Bundesliga sample for the same past observations 8 conditioned on user-defined text prompts Grex showing improved controllable accuracy. See Appendix for addi- tional ablations (D.3), interpretability analysis (D.4) and qualitative examples (D..2)). Please refer to the video supplementary to see animated results.
5.3 CONSISTENCY ANALYSIS
We evaluate the consistency of the joint generation X, = (Y,, Eg). Using the same setup as in Ta- ble we generate 20 samples per scene and compute the Ace metric between the predicted discrete possessor Eg and the threshold-based heuristic possessor extracted from the predicted trajectories

---

Published as a conference paper at ICLR 2026
Ours w/o joint
Ours
Ours w/o joint
Ours
24
IN
(a) "Away Team has the possession. The ball starts left-center without a currier. Player 16 posseses the ball, moving it from left-center to down-side, then passes Player 21 who possesses the ball. Then Player 21 makes a pass to Player 19 into the box."
(b) "Away Team has the possession. The ball starts at left-center without carrier. Player 16 makes puss to Player 15 who receives. Player possesses the ball in left-center and pass up-side Player 20."
Figure 4: Controllable Generation. Comparison of JointDiff Ours w/o joint the text- guidance task giving the same past observations with different text prompts Grea- Legend: Ball, Home team, Away team,
O Past observations. See animated scenes in supplementary
Table Consistency Analysis. Consistency between predicted events and trajectories, reported as max / avg Acc 1 over 20 samples. We compare multinomial with the absorbing state framework.
Multinomial (Ours) Bundesliga
Absorbing Bundesliga
Method
NFL
NBA
NFL
NBA
Ours w/a G
.98 .66
.97 98800
.9/.922
.97 .80
.94 04/.70 7.70
.99/89
Ours Gwiss
96 / 884
.95 980
.99/
93 778
/1/
.90
Ours w Goost
.97 / 86
.9 / /81
.57 /.111
.9/_77
Yo. For reference, compare multinomial diffusion with the commonly used absorbing state formulation (Levi et al. 2023 Li et al. 2025). For the absorbing, we use the publicly available im- plementation from Li et al. 2025 and set the absorption rate to X 0.01 to match the magnitude of the discrete loss with the continuous one. Unlike the multinomial model, which enables refinement through all states at each denoising step, the absorbing mechanism freezes tokens once unmasked, preventing later correction (von Rütte et al.) (2025). As shown in Table 3 our method achieves high consistency, particularly for the best sample (mar)), and strong consistency
average (avg). The larger variance in NFL and Bundesliga aligns with their smaller training sets. Overall, this ablation shows that multinomial diffusion yields substantially more consistent predictions than the absorbing state approach in our dynamic domain. To assess the quality of the generations using absorbing state with respect to the ground truth, results for future generation are shown in Appendix D.3.5
CONCLUSIONS
We have introduced JointDiff, novel diffusion framework that unifies the generation of continuous trajectories and synchronous discrete events in multi-agent systems. Our model obtains state-of-the- art results on completion tasks and enables new forms of semantic control through WPG and text- guidance. We demonstrate that the joint formulation is key factor, as it significantly enhances the fidelity of the controllable generation. While this work assumes a dense event pattern, a promising future direction is to extend the framework sparse events, such time point processes. Finally, JointDiff provides a strong foundation for generating controllable, nov-dimensional data to steer high-dimensional generative models, such as for video synthesis in complex interactive domains.
Acknowledgmentt. This work has been supported by the project GRAVATAR PID2023-151 110---1111810---11 1840B-100 funded by CO0UU/AF1/10.13039/// 10001 1033 and by ERDF, UE and by the Gov. of Catalonia under 2023 DI 00058.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Silvio Savarese. Social Istm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 961-971, 2016.
Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion- time series imputation and fore- casting with structured state space models. arXiv preprini arXiv:2208.00339, 2022.
Michael A Alcorn and Anh Nguyen. baller2vec: A multi-entity transformer for multi-agent spa- tiotemporal modeling. arXiv preprint arXiv:2102.03291, 2021.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 14:11981-11 1-999,, 2021.
Inhwan Bae, Young-Jae Park, and Hae-Gon Jeon. Singulartrajectory: Universal trajectory predictor using diffusion model. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 17890-17901, 2024.
Manuel Bassek, Robert Rein, Hendrik Weber, and Daniel Memmert. An integrated dataset of spa- tiotemporal and event data in elite soccer. Scientific Data, 12(1):195, 2025.
Guillem Capellera, Luis Ferraz, Antonio Rubio, Antonio Agudo, and Francese Moreno-Noguer. Transportmer: A holistic approach to trajectory understanding in multi-agent sports. Proceed- ings of the Asian Conference on Computer Vision, pp. 1652-1670, 2024.
Guillem Capellera, Antonio Rubio, Luis Ferraz, and Antonio Agudo. Unified uncertainty-awant aware diffusion for multi-agent trajectory modeling. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22476-22486, 2025.
Sergio Casas, Cole Gulino, Simon Suo, Katie Luo, Renjie Liao, and Raquel Urtasun. Implicit latent variable model for scene-consistent motion forecasting. European Conference on Computer Vision, pp. 624-641. Springer, 2020.
Sathvik Reddy Chereddy and John Femiani. SketchDNN: Joint dontinuous-discetee diffusion for CAD sketch generation. In Forty-second International Conference on Machine Learning, 2025. URL httpthtpss//ppeeeeeeeee wttp://wwwwwieeeeeneeeeeeeeeeeee
net forum? 1d=
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-877 87-8444 2021.
Liangji Fang, Qinhong Jiang, Jianping Shi, and Bolei Zhou. Tpnet: Trajectory proposal network for motion prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6797-6806, 2020.
Xiaolin Fang, Caelan Reed Garrett, Clemens Eppner, Tomás Lozano-Pérez. Leslie Pack Kaelbling. and Dieter Fox. Dimsam: Diffusion models as samplers for task and motion planning under partial observability. In 2024 IEEE/RSJ International Conference on Intellientt Robots and Systems (IROS), pp. 1412-1419. IEEE, 2024.
Panna Felsen, Patrick Lucey, and Sujoy Ganguly. Where will they go? predicting fine-grained adversarial multi-agent motion using conditional variational autoencoders. In Proceedings of the European conference on computer vision (ECCV), pp. 732-747, 2018.
Yuxiang Fu, Qi Yan, Lele Wang, Ke Li, and Renjie Liao. Moflow: One-step flow matching for hu-
man trajectory forecasting via implicit maximum likelihood estimation based distillation. In Pro- ceedings of the Computer Vision and Pattern Recognition Conference, pp. 17282-17293, 2025.
Yang Gao, Po-Chien Luan, and Alexandre Alahi. Multi-transmotionn Pre-trained model for human motion prediction. In 8th Anmual Conference Robot Learning, 2024.
Roger Girgis, Florian Golemo, Felipe Codevilla, Martin Weiss, Jim Aldon D' D'Suuza, Samira Ebrahimi Kahou, Felix Heide. and Christopher Pal. Latent variable sequential set trans- formers for joint multi-agent motion prediction. arXiv preprint arXiv:2 104.00563, 2021.
11

---

Published as a conference paper at ICLR 2026
Colin Graber and Alexander G Schwing. Dynamic neural relational inference. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8513-8522, 2020.
Albert Gu and Tri Dao. Mamba: tinear-tmme sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752. 2023.
Tianpei Gu, Guangyi Chen, Junlong Li, Chunze Lin, Yongming Rao, Jie Zhou, and Jiwen Lu.
Stochastic trajectory prediction via motion indeterminacy diffusion. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17113-17122, 2022.
Agrim Gupta, Justin Johnson, Li Fei-Fei, Silvio Savarese, and Alexandre Alahi. Social gan: So-
cially acceptable trajectories with generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2255-2264, 2018.
Jonathan Ho and Tim Salimans. arXiv:2207.12598, 2022.
Classifier-free
diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising difflusio probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances neural information processing systems, 35:8633- 8646, 2022.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):
1735-1780, 1997.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances neural information processing systems, 1il:1244--246555 2021.
Chiyu Jiang, Andre Comman, Cheolho Park, Benjamin Sapp, Yin Zhou, Dragomir Anguelov, al. Motiondiffuser: Controllable multi-agent motion prediction using diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9644-9653, 2023.
Nitin Kamra, Hao Zhu, Dweep Kumarbhai Trivedi, Ming Zhang, and Yan Liu. Multi-agent trajectory prediction with fuzzy query attention. Advances in Neural Information Processing Systems, 33: 22530-22541, 2020.
Korrawe Karunratanakul, Konpat Preechakul, Supasorn Suwajanakorn, and Siyu Tang. Guided motion diffusion for controllable human motion synthesis. Proceedings of the IEEF/CV International Conference Computer Vision, pp. pp. 2151-2162, 2023.
Hyunsung Kim, Han-Jun Choi, Chang Jo Kim, Jinsung Yoon, and Sang-Ki Ko. Ball trajectory inference from multi-agen sports contexts using set transformer and hierarchical bi-lstm. arXiv preprint arXiv:2306. 08206, 2023.
Hyunsung Kim, Hoyoung Choi, Sangwoo Seo, Tom Boomstra, Jinsung Yoon, and Chanyoung Park. Elastic: Event-tracking data synchronization in soccer without annotated event locations. arXiv preprint arXiv:2508.0092:: 2025.
Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems. International conference on machine learning, pp. 2688- 2697. Pmlr, 2018.
Mihee Lee, Samuel S Sohn, Seonghyeon Moon, Sejong Yoon, Mubbasir Kapadia, and Vladimir Pavlovic. Muse-vae: Multi-scale vae for environment-awwre long term trajectory prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2221- 2230, 2022.
Seongju Lee, Junseok Lee, Yeonguk Yu, Taeri Kim, and Kyoobin Lee. Mart: Multiscale relational transformer networks for multi-agent trajectory prediction. In European Conference on Computer Vision, pp. 89-107. Springer, 2024.
12

---

Published as a conference paper at ICLR 2026
Elad Levi, Eli Brosh, Mykola Mykhailych, and Meir Perez. Dit: Conditioned layout generation diffusion layout transformer. In Proceedings of the IEEE/CVF
with joint International Conference Computer Vision, pp. 2106-2115, 2023.
Longyuan Li, Jian Yao, Li Wenliang. Tong He. Tianjun Xiao, Junchi Yan, David Wipf, and Zheng Zhang. Grin: Generative relation and intention network for multi-agent trajectory prediction. 18, 2021.
Advances in Neural Information Processing Systems,
Rongging Li, Changsheng Li, Dongchun Ren, Guangyi Chen, Ye Yuan, and Guoren Wang. Bediff: Bidirectional consistent diffusion for instantancous trajectory prediction. Advances in Neural Information Processing Systems, 36: 111404-0444414 2023.
Zijie Li, Henry Li, Yichun Shi, Amir Barati Farimani, Yuval Kluger, Linjie Yang, and Peng Wang. Dual diffusion for unified image generation and understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2779-2790, 2025.
Yukai Liu, Rose Yu, Stephan Zheng, Eric Zhan, and Yisong Yue. Naomi: Non-autoregressive multiresolution sequence imputation. Advances in neural information processing systems, 32, 2019.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. arXiv preprint arXiv:2310. 16834, 2023.
Weibo Mao, Chenxin Xu, Qi Zhu, Siheng Chen, and Yanfeng Wang. Leapfrog diffussio model for stochastic trajectory prediction. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 5517-5526, 2023.
Utkarsh Aashu Mishra, Shangjie Xue, Yongxin Chen, and Danfei Xu. Generative skill chaining: Long-horizon skill planning with diffusion models. In Conference on Robot Learning, pp. 2905- 2925. PMLR, 2023.
Jiquan Ngiam, Benjamin Caine, Vijay Vasudevan, Zhengdong Zhang, Hao-Tien Lewis Chiang, Jeffrey Ling, Rebecca Roelofs, Alex Bewley, Chenxi Liu, Ashish Venugopal, et al. Scene transformer: A unified architecture for predicting multiple agent trajectories. arXiv preprint arXiv:2106.08417, 2021.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference machine learning, pp. 3162-8171 PMLR, 2021.
Shayegan Omidshafiei, Daniel Hennes, Marta Garnelo, Zhe Wang. Adria Recasens, Eugene Tarassov, Yi Yang, Romuald Elie, Jerome T Connor, Paul Muller, et al. Multiagent off-screen behavior prediction in football. Scientific reports, 12(1):8638, 2022.
Mengshi Qi, Jie Qin, Yu Wu, and Yi Yang. Imitative non-autoregressive modeling for trajectory forecasting and imputation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattem Recognition, pp. 12736-12745, 2020.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, (1140):1-67) 2020.
Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, and Or Litany. Trace and pace: Controllable pedestrian animation via guided trajectory diffusion. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp- 13756-13766 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Saeed Saadatnejad, Yang Gao, Kaouther Messaoud, and Alexandre Alahi. traisotonansmotion:: Promptable human trajectory prediction. arXiv preprint arXiv:2312.161663 2023.
13

---

Published as a conference paper at ICLR 2026
Amir Sadeghian, Vineet Kosaraju, Ali Sadeghian, Noriaki Hirose, Hamid Rezatofighi, and Silvio Savarese. Sophie: An attentive gan for predicting paths compliant to social and physical con- straints. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1349-1358, 2019.
Tim Salzmann, Boris Ivanovic, Punarjay Chakravarty, and Marco Pavone. Trajectron++: Multi- agent generative trajectory forecasting with heterogeneous data for control. arXiv preprint arXiv:2001.030991 2, 2020.
Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint 1xXiv20000002002 2020.
Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffusion models for probabilistic time series imputation. Advances in Neural Information Pro- cessing Systems, 34:24804-244 24816, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- fion processing systems, 30, 2017.
Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, and Thomas Hof- mann. Generalized interpolating discrete diffusson. In Forty-second International Conference Machine Learning, 2025. URL https: httppp/eew//eeeeeee...... net. forum? dd=rvžv PDVVV sDPV9,
Erica Weng, Hana Hoshino, Deva Ramanan, and Kris Kitani. Joint metrics matter: A better stan- dard for trajectory forecasting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 20315 20315,20336, 2023.
Wei Wu, Xiaoxin Feng, Ziyan Gao, and Yuheng Kan. Smart: Scalable multi-agent real-time motion generation via next-token prediction. Advances in Neural Information Processing Systems, 37: 114048-1 100048-114071, 2024.
Chenxin Xu, Maosen Li, Zhenyang Ni, Ya Zhang, and Siheng Chen. Groupnet: Multiscale hyper- graph neural networks for trajectory prediction with relational reasoning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 6498 6507, 2022.
Yi Xu and Yun Fu. Deciphering movement: Unified trajectory generation model for multi-agent.
arXiv preprint 17X11:2405.17680, 2024.
Yi Xu and Yun Fu. Sports-traj: A unified trajectory generation model for multi-agent movement in sports. In The Thirteenth International Conference Learning Representations, 2025. URL
n wwtp://neeeeteeeeeeeeeeeeee
Yi Xu, Armin Bazarjani, Hyung-gu gunng Chi, Chiho Choi, and Yun Fu. Uncovering the missing pattern: Unified framework towards trajectory imputation and prediction. In Proceedings of the IEÉE/CVF Conference Computer Vision and Patten Recognition, pp. 9632 9643, 2023.
Brian Yang. Huangyuan Su, Nikolaos Gkanatsios, Tsung- Wei Ke, Ayush Jain, Jeff Schneider, and Katerina Fragkiadaki. Diffusion-es: Gradient-free planning with diffusion for autonomous and instruction-guided driving. In Proceedings of the IEEE/CVE conference on computer vision and paltern recognition, pp. 15342-15353, 2024.
Raymond A Yeh, Alexander Schwing, Jonathan Huang, and Kevin Murphy. Diverse generation for multi-agent sports games. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 4610-4619, 2019.
Ye Yuan, Xinshuo Weng, Yanglan Ou, and Kris M Kitani. Agentformer: Agent-aware transform- ers for socio-temporal multi-agent forecasting. In Proceedings of the IEEF/CVF International Conference Computer Vision, pp. 9813-9823, 2021.
Mai Zeng, Florence Regol, and Mark Coates. Interacting diffusion processes for event sequence forecasting. In International Conference Machine Learning, pp. 58407-58430. PMLR, 2024.
14

---

Published as a conference paper at ICLR 2026
Eric Zhan, Stephan Zheng, Yisong Yue, Long Sha, and Patrick Lucey. Generating multi-agent trajectories using programmatic weak supervision. In International Conference on Learning Rep- resentations, 2019.
Stephan Zheng, Yisong Yue, and Jennifer Hobbs. Generating long-term trajectories using deep hierarchical networks. Advances in Neural Information Processing Systems, 29, 2016.
Wenzhao Zheng, Ruiqi Song, Xianda Guo, Chenming Zhang, and Long Chen. Genad: Genera- tive end-to-end autonomous driving. In European Conference on Computer Vision, pp. 87-104. Springer, 2024.
15

---

Published as a conference paper at ICLR 2026
A DIFFUSION BACKGROUND
This section is intended to be an extension of the diffusion background provided in the main paper (Section 3). We rescue the notation from the main paper for clarity and detail the specifie processes for continuous and discrete data.
A.1 CONTINUOUS CASE
The data corrupted with Gaussian noise until it converges to standard isotropic Gaussian distri- bution. The forward process at any step can be stated in both recursive and closed form as:
(X, Xx,1) NXx:; -
(9)
gXx Xo) = (Xx,; vā,Xo, (1 ā.)I).
(10)
This closed-form expression allows to sample X, directly from Xo using the reparameterizzttion trick:
= V V,, where (N,
(11)
The reverse process generates data by starting with sample from the prior, X3 ~ (O,,),, and iteratively sampling from the reverse conditional distributions (X X.). The true posterior q(X,-1 X.) is intractable as it depends on the entire data distribution. Therefore, we approximate with neural network po(-) that outputs the parameters of a Gaussian:
pe (X,-1 X.) MN no (X.,, s). oi.).
(12)
The mean po commonly parameterized terms of predicted noise term co (X,,;8)) 8):
B.
M4 (X..
x,
X (X..3)
(13)
Vol,
VI a
For simplicity, the variance o? is typically set to non-learned constant, such as of 8.. With this parameterization, minimizing the variational bound L.v (Eq, [ equivalent to training the noise prediction network Eg with simplified objective:
simple Eg,xo.e |l|e colva,Xo + vI age, s)
(14)
A.2 DISCRETE CASE
Here, the data is categorical and typically represented by one-hot vectors of dimension N. The forward process corrupts the data until it converges to a uniform distribution across categories. This is defined multinomial diffusion process:
gX,, (X, X.) Cat(X,; a + (1 - ox,)
x
Cat(X,; -- Ba/N),
(15)
(16)
The posterior is categorical distribution q(x,-1 X X,) Cat (Cat(X,--a) -1 (X,-1i 0post (X,) whose probabilities are defined as:
Ppost X., Xo) = 0/ Σ Ō, and 0 = [a,X, + - a.)/N] o Tā,-1X, + (1- (1-0,,1)//
Here, denotes the element-wise product. Note that the result is normalized sum to
The reverse process learns to approximate the true posterior as in Eq. [ does so by learning to approximate the original clean data Xo from noisy version X,, Indeed. probability vector is predicted using neural network Ta (X,, s) X.. Subsequently, and 2 respectively:
parametrize when
(X X1) = (Xo: Xo) and Po (X X.)) = q(x,-1 X,, Xo)-
(17)
The DKL. expressions for 2 2 in Eq. can be computed
Dkr.(g(X.-1 X., Xo) X X.)) DRI. (Opnct (X,, Xo) Opost (X,, (x)x (18)
16

---

Published as a conference paper at ICLR 2026
X,
Optiioal))
Payon TI are inneral paoooase
gano
Nack
MILA
CrossOrud WPG
Linnar
MILA
Croseruud
text
JL, 206
RV
IL, 206
XV
Player
Playe
ROOO000 [7,
Player
[N, 264]
M
N
Plager 20.
(N 230)
N. T. 250)
X
It's Base Encoder
Temporaa Mamha
L23
H
Cromrriit
2 Sortal Pranafomnnt
Minb
[NN 266
Eojeetunaa
Projection,
[T, N
M,N, 206]
Linear
RDB
Linea
RDB
T
Projection,
Aach [128]
JointDiff
[2,N]
Figure 5: Model Architecture (extended version of Fig. 2). The light gray box represents the JointDiff architecture, and STB refers to the Docial-Temporal Block.
by enumerating the probabilities and using Σn, 6 poet (X,, Xu)n likelihood term can be computed as:
log Ppen (X, Ppee (x),
The log-
Pe (x X1) Σ Xo, log X0,0.
(19)
B ARCHITECTURE
We provide a detailed explanation of the general architecture, which can be shown in Fig
The model takes as input the noisy states X,, observed states X with their corresponding binary mask M, the denoising step and the optional guidance signal G. The noisy state X, is formed by concatenating the continuous trajectories Y, with the discrete possession events Eg. The observed states X similarly constructed using Y and E These two [T. NN tensors, along with the binary mask M, are then concatenated to create a single input tensor of dimension [T, N, 7]. This input tensor is first processed through layer (Projection,.) which embeds the data and combines it with a learnable agent embedding (Namb)) This results in embedding tensor J of dimension N,, 256). The denoising step is embedded through layer (Projection,) become an embed- ding Semb E IR 128, Both J and Semb are then processed by sequence of two residual denoising blocks (RDBs). Within each RDB, the Semb is added J after Linear embedding projection to dimension 256. Then the resulting tensor is processed by Social-Tempore Block (STB). As ex- plained in the main paper, for controllable generation, we introduce an operation called CrossGuid inside this block to guide the denoising process with the signal G. After the RDBs processing, the resulting embedding with the same dimension as J is processed through a layer (Projection defining output tensor embedding which is then projected in into two heads: regression head (Regression), based on a linear layer, to predict the Gaussian noise co added to the continuous tra- jectories; and classification head (Classification)) based on linear layer followed by softmax activation, predict the probability distribution of the original discrete events, yielding Eg mp.
Now we define each mentioned operation:
Projectionn:
The input tensor of shape [T, NN, 7] is combined with a learnable agent em- bedding Nim E [56, First, the input tensor is linearly projected to [T, N, 256] and concatenated with the agent embeddings (replicated T times along the temporal axis), yielding a [T, N, 512] tensor. Linear layer with ReLU activation re-projects this ten- sor to [T, N, 256), producing the embedding tensor J.
17

---

Published as a conference paper at ICLR 2026
Embeds the denoising timestep using Linear layer followed by SiLU activation, producing Semb 128
Projection,
RDB: Processes J conditioned on Semb, and optionally the guidance embedding G, as: RDB(J, Bembs G) = J + SJB(J Linear( G),
Social-Temporal Block (STB): Maps [T' 256] - [T" N, 256] via
STB(J,G) =
G)))
where TM denotes the Temporal Mamba, LN is Layer Normalization, CrossGuid is the proposed guidance module, and ST is the Social Transformer. For non-controllable tasks, CrossGuid reduces the identity.
Temporal Mamba (TM): As described in Capellera et al. (2025) TM applies bidirectional
Mamba modules (Gu Dao 2023) independently to each agent and sums results.
Social Transformer (ST): A Transformer encoder (Vaswani al. 2017) applied per timestep to model inter-agent correlations.
Projection Linear layer with ReLU maps [T, NN 256] N,
Regression: A Linear layer maps [T, N, 256] + [T, 2, producing the predicted noise co.
Classification:
Linear layer followed by a Softmax along the agent axis maps TT 256) + [T,N], N]. producing possessor probabilities ng.
C DATASETS
C.I POSSESSOR THRESHOLD
To robustly define the discrete possessor event data (E) from the continuous trajectories (Y), conducted data-driven analysis to determine an optimal geometric threshold. Our goal was to identify the distance which a player most likely be in possession of the ball.
Our methodology is based on the key observation that ball's trajectory is primarily linear when it is any player's possession (e.g., during a pass or a shot). Conversely, player taking possession or influencing the ball's path will induce a significant change in its direction. To quantify this, we analyzed the change the ball's direction by computing the angle between consecutive ball velocity vectors across our training datasets. For range of distance thresholds from to 3 meters, we calculated the average angle of change for the ball only during periods when was outside of that specific threshold distance from all players. The results are shown in Fig. 6
The optimal threshold was defined as the minimum distance that minimizes this average angle of change. This approach allowed to identify the point which the ball's trajectory becomes most linear, indicating the absence player control. Our analysis across all sports consistently iden- tified approximately 1.5 meters as the optimal threshold, supporting the use of single, unified geometric heuristic for defining possession events across different sports. This finding aligns with the observation that player influence the ball's trajectory is negligible beyond this distance.
C.2 TEXTUAL GUIDANCE DATASET GENERATION
The core methodology for caption generation is consistent across both Bundesliga and NFL datasets, ensuring a standardized approach data creation. The pipeline consists of the following stages:
Stage 1: Preprocessing and Standardization The process begins with spatio-temporal track- ing data, which is first segmented into fixed-length sequences. To ensure data consistency and privacy, all entities are anonymized. Players are systematically numbered from to 22 (with home players assigned 1-11 and away players 12-22) aligning their identifiers with their corresponding order in the final trajectory tensors.
Stage 2: Automated Feature Extraction For each standardized sequence, script programmati- cally analyzes the trajectory and events data to extract key semantic features. This au- tomated analysis includes identifying the ball possessor at each frame, detecting discrete
18

---

Published as a conference paper at ICLR 2026
NFL
Bundesliga
NBA
Mom ball angle (Rogress)
Moan ball onglee (deggoes)
Meer bal. (deggrees)
10
12
25
10
20-
15x
12
Threshold (metens)
Threshold (meters)
Threshold (meters)
Figure 6: Possession threshold determination. Average ball direction change versus distance threshold. The minimum at 1.5 meters indicates optimal threshold where ball motion is most linear without player influence, supporting our unified possession detection heuristic.
game events (e.g., passes, tackles), and mapping the ball's location meaningful zones on the field. These features are then compiled into structured, rule-based textual summary, which we term dense caption.
Stage 3: LLM-based Narrative Refinement In the final stage, the dense captions are transformed into fluent, numan-readabee narratives. We leverage a Large Language Model (LLM), prompting to act as an expert sports analyst. The LLM uses the structured informa- tion from the dense caption to generate a chronologically accurate and natural-sounding description that adheres to the common parlance of the respective sport.
While the pipeline is shared, its parameters and feature extraction rules are tailored to the unique characteristics of each sport. Refer Fig 13
examples of these two datasets.
C.2.1 NFL DATASET
Data Source: We process raw tracking data from the NFL Big Data Bowl, adopting the data splits and experimental settings from & Fu (2025).
Sequence Processing:
Trajectories are segmented into 50-frame sequences sampled at 10 fps (5.0 seconds in duration).
Feature Extraction:
Possession:
The player closest to the ball within 1.5-meter threshold is designated
the ball carrier.
Events: Key extracted events include ball foss.forward, and tackle.
Formation: Offensive team spatial configuration.
Short lextual description: A natural language description of the final outcome of the play, covering a time horizon of more than one split
Location: The ball's position is mapped to the corresponding yard line.
C.2.2
BUNDESLIGA DATASET
Data Source: We use the integrated trajectory and event dataset provided by Bassek al.
(2025).
Sequence Processing: Raw 25 fps tracking data synchronized with asynchronous event data, downsampled 6.25 fps, and segmented into 40-frame sequences (6.4 seconds in duration). Sequences with less than 23 agents and out-of-play are discarded.
Feature Extraction:
The player closest to the ball within a 1.5-meter threshold is identified as the possessor.
Possession:
Events: Key extracted events include pass, tack 1 e, and shot.
19

---

Published as a conference paper at ICLR 2026
Location: The ball's coordinates mapped predefined semantic grid that parti- tions the field into named zones (e.g,, down-corner, box).
D EXPERIMENTS
D.1 BASELINES
To construct the Table we implemented and evaluated several state-of-the-art architectures. Below we detail how results were obtained for each method:
GroupNet Xu et al., 2022): We used the official code and checkpoints for NBA. For [11, 23) match the
NFL and Bundesliga, we adapted the parameter hyper number of agents, and doubled both hidden.cc doiden.dimm and zdim.
AutoBots (Girgis al. 2021): :: We used the official repository with the same hyperparam-
eters as in Tett..t++. The AutoBotJoint variant was trained on all three datasets.
LED MMa et al., 2023): We used the official code and checkpoints for NBA. For NFL and Bundesliga, adapted the code to handle longer horizons and adjusted hyperparam- eters. Following the users' recommended procedure, we first pre-trained the denoiser on single timestep prediction task, then fine-tuned it for full temporal horizon prediction. Both trainings were performed using 100 epochs, batch size equal to 250, and a learning rate of 10-3
LED (Mao et al. 2023): Official code and checkpoints were used for NBA. We were unable train this stage for NFL and Bundesliga due GPU memory limitations.
MART (Lee et al. 2024), MoFlow (Fu et al. 2025): Official code and checkpoints were used for NBA. For NFL and Bundesliga, we trained using the same settings, changing only the prediction horizon.
Sports-Traj (Xu & Fu 2025): ) We used the official repository with the same hyperpa- rameters in their benchmark. We observed, consistent with the authors' checkpoints in another benchmark, that the 20 sampled modes were nearly identical.
TranSPORTmer (Capellera al. 2024), U2Diff (Capellera al., 2024): Official code and checkpoints were used for NBA. For NFL and Bundesliga, we trained with the same configuration. The authors provided the original codebase.
D.2 COMPARISONS
D.2.1
DUMAN-BASER METRICS
For each of the first 128 past observations in test set, we formed random pair from these selected samples. An example the interface shown in Fig
D.2.2 INDIVIDUAL-BASER METRICS
This section provides quantitative analysis using individual-based metrics ADE and FDE, reported as both minimum (min) and average (aug) values over 20 generated modes. Note that the average metrics are equivalent SADE and SFDE presented in Table of the main paper.
Individual metrics comparison is depicted in Table 4 Our approach remains competitive in terms of min metrics against non-lID baselines while, as stated the main paper, excels in the aug metrics. Crucially, our findings highlight key divergence: the widely used mán ADE/FDE metrics do not correlate with the human evaluation from the previous section. For example, on the NBA dataset, MoFlow achieves the state-of-the participants consistently preferred our approach. This suggests that SADE SFDE metries provide more reliable indication of perceptual quality than ADE FDE metrics Casas et al. (2020).
in min ADE / FDE, yet our human study showed that
D.2.3 COMPLETION GENERATION
We benchmark our approach on general trajectory completion using the datasets and experimental setup from Xu & Ful (2025), which includes the Basketball-U, Football-U, and Soccer-U datasets.
20

---

Published as a conference paper at ICLR 2026
Video
Iikal und sulan Mo mediu strer
Oroen inton
Leare
SErCIIE peaccion
1
c
D
Figure 7: Human Evaluation Interface.
Table 4: Future Generation. We report ADE and FDE metrics (min / avg) computed over 20
generated modes.
NFL (yands)
Bundesliga (meters)
NBA (meters)
Method
ID
ADE
FDE
ADE
FDE
ADE
FDE
GroupNet
1.70/5.35
1.89
3,23
0.94/2.84
1.22
AutoBots
1.82 / /482
3.23/ 10.68
2.07 / 5,93
2.94/ 1146
1.19/2.73
1.55 /4.71
LED ID eVeRz
/
1.65 1.65/4.12 /4.12
2.08 9966
2.06 / /4.57
3.17
0,92 /2.30
1.18 /4.4
eveRzs
0.81/3.83
1.10/6.03
MART neevvs
1.07 4,26
1.96/ 10.31
1.41 /4.16
2.48/
0.72/2.46
0.90/4.75
MoFlow evenzs
X
1.03 /4.02
L87/
1.47 /4,21
2.74/9.24
0.71 /2.42
0.86 /4.64
everes
1.40
2.67/ /.002 9.02
/4.21
/9.44
/2112
1.11
JoinDDiff (Ours)
1.31 /3.40
1.46 / 3.66
2.56 8.29
0.80
1.09 /3.95
In this task, a pre-defined mask using different strategies is applied to scene, and the model must complete the missing observations. We note that while our NFL dataset uses the same data splits as Football-U, it employs a different masking strategy. For comparison, we reuse the evaluation table from Xu & Ful (2025), also reported in Capellera et al.] (2025), which includes wide range of baselines. These baselines span statistical methods: Mean, Median, Linear Fit; vanilla models:
LSTM (Hochreiter & Schmidhuber 1997). Transformer (Vaswani et al. 2017)); and advanced
methods: MAT (Zhan et all][ Naomi (Liu al. 2019). INAM (Qi et al. 2020 SSSD (Alcaraz
& Strodthoff] 2022). GC-VRNN (Xu et al. 2023). Sports-Traj (Xu Fu 2024). and U2Diff (Capellera et al. 2025).
We use the ADE metric as defined in Xu & Fu (2025), which rename BADE to reflect its dependence batch size and distinguish it from the standard individual-level ADE or the scene-level SADE. The BADE metric defined as:
Results are reported in Table 5
ΣB
N m=
T 11
fl. Vi.nll2 (1 mfn))
BADE
(20)
ET mi,.)
where vi, is the ground truth 2D spatial location of agent n at timestep L. in scene b, oi., is its estimation, and m is value from the binary mask M where a value of O indicates location to be predicted. their setting, Xu & Fu (2025) use 128. We also report the standard SADE metric. The table presents the minimum (man) values across 20 generated modes. While the minimum mode for BADE selected across the entire batch of scenes, the minimum mode for SADE is selected for each scene independently, making it independent of batch size and mode ordering.
21

---

Published as a conference paper at ICLR 2026
Our approach notably outperforms previous baselines, showing significant performance improve- ment in SADE of 13% in Basketball-U, 19% Football-U, and 12% in Soccer-Ul U. We observe that the provided checkpoints for the Sports-Traj model in this benchmark produced modes that were nearly-identical, with minimal differences between them.
To advance the field, in the main paper advocate for the of the widely-adopted NBA dataset as an alternative to Basketball-U. We have also curated a new Bundesliga dataset for soccer. This dataset offers significant advattagss as its sequences are substantially longer than those in Soccer- U. providing richer temporal context for modeling. In our curation, Bundesliga sequences last 6.4 seconds (40 frames 6.25 fps), whereas the Soccer-U sequences, with 50 frames, last for two seconds or less.
Table 5: Completion Generation. We report the min for BADE (ADE in (uu & Fu 2025 and SADE over 20 generated modes.
Method
Basketball (Feet)
Football (Yards) BADE
Soccer-U (Pixels) BADE
BADE
SADE $
SADE
SADE
Mean
14.58
14.18
417.68
Median
14.56
14,23
418.06
Linear Fit
13.54
12.66
398.34
LSTT
7.10
7.20
186.93
Transformer
6.84
170.94
MAT
6.68
6.36
170.46
Naomi
6,52
6.77
145.20
INAM
5,80
134.86
SSSD
6.18
5.08
118.71
GC- VRNN
5,81
4,95
105.87
Sports-Traj Traj
94.59
100.48
U2Diff
4.65
3.13
2.42
2.35
53.93
51.14
JointDiff (Ours)
4.42
2.72
2.14
1.90
49.23
44,89
D.2.4 FUTURE GENERATION
We further evaluate our method on future trajectory prediction using the basketball dataset, referred
to as NBA NBA12/13 12/13 and protocol from Li et al. (2021). This setting enables comparison against tem- poral autoregressive baselines such NRI Kippf et al. 2018). dNRI (Graber & Schwing 2020).
and GRIN (Li et al., 202T); GAN-based models such as Social-GAN (Gupta et al. 2018): and transformer- based approach, FQA (Kamra al., 2020). Results are presented Table 6 which reports metrics equivalent our min SADE and SFDE over 100 samples. JointDiff achieves strong performance relative to these forecasting models while operating as a more general trajectory com- pletion framework, benefiting in particular from the non-autoregressive nature of diffusion along the temporal dimension.
Table 6: Future Generation on NBA 12/13 We report the mén for SADE and SFDE (ADE and FDE in (Li et al. 202I respectively) over 100 generated modes.
Method
12/18 (Feet)
SADE
SFDE J
NRI
2,10
4.56
dNRI
2.02
4.52
FQA
2.42
Social-Gan
1.88
3.64
GRIN
1.72
3.59
Diff (Ours)
1.36
2.52
D.3 ABLATIONS
This section evaluates our method through three ablation studies: first, we examine the effect of varying the number of denoising steps Sd in the discrete scheduler; second, we analyze the sen- sitivity of the hyperparameter in our proposed joint loss (Eq, [): and third, we investigate the contribution of additional components such extra Social Transformers and importance sampling.
22

---

Published as a conference paper at ICLR 2026
Table 7: Denoising Step in Future Generation. We report SADE and SFDE metrics (min / avg) and Acc (max/ (vg) over 20 generated modes.
NFL (yards) SFDE
Bundesliga (meters)
NBA (meters)
<
SADE
SADE 1
SEDE
Ace
SADE
STDE
Ace
5
2.45/ 3.59
3.70/8.80
.79/.51
2.63
/,25/8.92
(88.38
1.41 /2.05
2.56/ 4.02
.76/.42
10
3.43/3.46
5.61/8.37
.75/.47
2.74/4.23
5.41/9.19
.64/.35
1.40
2.55/3.90
.74/
5
3.40
5.43/8.40
.78/.54
3.66
5.02
/.39
1.39 /2.01
2.93/
.75/ .45
10
2.37/3.28
5.54 8.00
.73/.47
23.67
8.02/8.17
.63 /.36
1.38 1.95
2.52/3.82
.71 .40
5
2
5.76/ 8.43
.70/ .44
2.56/3.51
5.06
/.34
1.42 /2.06
2.58 2.58/ 4.06
.71 /.1
25
10
3.66/3.70
6.05 8.66
.37.30
2.45
5.00/7.63
.53/.28
1.40
3.86
.63 /.32
3.60
8.57 5.95 8.57
-55/.11
2.53/3.73
8.13
.54 /2
1.43
2.60/4.14
.66 6.5
10
2.65/ 3.62
6.07 / 8,50
.37..9
2.54 /3.73
/.09
.45
1.40 / 1.99
2,57
.54 1.27
D.3.1 DISCRETE DENOISING STEPS
We present an ablation study on the impact of the total discrete denoising steps (84) and the DDIM skipping parameter (O). The total number continuous denoising steps fixed to = 50 and we train approach with different discrete denoising steps S E {5. 10, 25, 50}. At inference generate the samples with two different skipping intervals Ç E (5. 10). This results in either 11 total steps (C 5) six total steps (C 10). The results in Table indiccae that configuration of Sd 10 and Ç = 5 is optimal for the reported SADE, SFDE, and Accuracy metrics.
D.3.2 LAMBDA IN JOINT LOSS
We conducted an ablation study on the weighting hyperparameter, X, in our joint loss function (Eq. R). Our goal to identify the largest value for that does not negatively impact the quality of the continuous trajectory generation. Table & shows the results for various À configurations: A E {0, 0.001, 0.01, 0.1, 1}. Note that the case where À is equivalent our ablated model, Ours w/o joint. We found that À = 0.1 represents the optimal trade-off, providing the most significant benefits from the joint modeling without overwhelming the primary trajectory generation task.
Table 8: Lambda Sensitivity in Future Generation. We report SADE and SFDE metrics (min / arg) and Acc (max/ ang) over 20 generated modes.
NFL (yurds) SPOE
Bundesliga (meters)
NBA (meters)
A
SADE
SADE:1
SHDE
Acc
SADI
SPIR
Acc
0
2.55/
5.67 8..2
.76 55
2.60
5,30 8.95
667 .44
1.46/2.13
2.64/4.19
774 .4
0.001
2.73/3887
6.27
.50
2.67 /4220
53/.22
2.16
2.66/4.19
.6/
0.01
2.56/3.60
5.95/8.87
68/
2
5.03
.6/
1.42/2.07
2
.33 .2
0.1
2340
5.53/8.00
.787 /5
2.47 /366
5.02/8.22
.68.39
1.39
/2.0
2.53/3.95
75
/3.7
3.95
.79/.56
2.71/
5.45
/69/.39
2.05
2.62/4.00
.76/44
D.3.3 GUIDANCE STRENGTH
As stated in the main paper, during inference we utilize a single forward pass corresponding to the conditioned output, which corresponds to using
CFG (Ho Salimans 2022). With our DDIM sampling, this choice yields optimal performance, while extreme values degrade results. This setting also avoids the extra cost of dual forward passes. In Table we report the same SADE / SFDE metrics as in Table [ with Gest for NFL and Bundesliga datasets. CFG training enables a unified model supporting both controllable (w = 0) and non non-controllable (w = -1) generation without notable degradation compared to our dedicated non-controllable model (Table 2 Ours w/o
g).
D.3.4 T5 ENCODER
We compared the performance using T5-Small, T5-Base, and La-Lerge encoders and observed only minor performance differences across the board. As shown in Table I0, T5-Base offers the best overall balance, achieving the lowest min / avg SADE and SFDE metrics on the NFL dataset, and competitive results on Bundesliga.
23

---

Published as a conference paper at ICLR 2026
Table Guidance Weight w. We report SADE and SFDE metries (min / avg) over 20 generated modes.
NFL (yards)
Bundesliga (meters)
SADE
SEDE
SADE
SEDE
-10
2.44 /.447
5.75/8.59
2.51/3.70
5.10 8.33
2.21 f3.11
5.09/7.58
2.12/2.84
4.19/6.06
0.0
2.19/3.09
5.04/7.52
2 2
4.09 5.68
0.5
2.19/3.10
5.04/7.56
2.12/2.73
4.18 5.66
1.0
2.21 /3115
5.10/768
2.21/2811
4,40/ 5.84
2.0
2.31 /3.31
8.04
2.49 3.07
5.05 6.41
3.0
2.46/3.53
8.59
/3.38
5.71/7.04
T5-Large performs slightly worse than T5-Base, which likely due the stronger dimension- ality compression required to map its 1024 dimensional embeddings down to our model's 256-dimensional hidden space. Overall, the encoder size has minimal impact on the model's perfor- mance, but the results suggest that excessive compression of very large embedding space can lead to slight degradation in trajectory prediction accuracy.
Table 10: T5 Encoder Size. We report SADE and SFDE metrics (min avg) for different encoder sizes (d), measured over 20 generated modes.
NFL (yards)
Bundesliga (meters)
T5 Encoder (d)
SADE
SADE
SFDE
Small (512)
2.22 33122
5.12/7.59
2.08 2.777
4,06 0 5.82
Base (768)
5.04/7.52 /.09
2.72
4.09 5.68
Lange (1024)
/.24
5.14/7.88
2.10/2.75
4.11/
D.3.5 MULTINOMIAL VS ABSORBING
In the main paper, we compare the consistency of our multinomial discrete diffusion model with the absorbing state formulation in Table 3 To provide more comprehensive assessment of gen- eration quality, we extend this analysis using same metrics as in Table [2 for controllable future generation. The results, presented in Table compare method when replacing the multinomial parameterization with an absorbingstate one (Ours w absorbing) against our original model. Across all tasks, the absorbing formulation exhibits consistently lower performance, which attribute to reduced consistency between the generated trajectories and discrete events.
Table 11: Controllable Generation with Absorbing. This table compares our JointDiff model (using multinomial diffusion) against an ablation using absorbing state diffusion for discrete events (Ours w absorbing). As in Table we report the same metrics for both non-controllable (w/o G) and controllable (w GwPG, mxx)) future generation tasks.
Method
NFL (yards) SEDE
Bundesliga (meters) SEDE
NBA (meters) SEDE
Ours absorbing
w/o G
5.78/ 2.607
.76/ .54
2.72/4.13
8.99
1.45 .64/.37
2.62 4.07 2.25 / 3.66
GwrG Guet
/.884
5.72/ 8.92
.3/ /65
2.36/3.44
4.43/7.04
775
1.28 1.89
.87 /6
2.41 3.48
5,46 8.29
.85 7,72
23.9/3..1
4.34/6.44
779/.57
Ours
w/o
2.36/33.0
/.440
.78
2.47 /3.66
5.02/8 29
.67
1.39
2.53 3,95
.75/ /55 .87
w Gwiys
3.29/3.26
5,29 7.94
844 65
2.13
4.22/6.16
1.7
1.24/ 1.81
2.20 3.53
Giest
2.19/3.09
5.04/7.52
.74
2.08 /2.2
4.09/
.80
D.3.6 ADDITIONAL ANALYSIS
We present the last ablation analysis which includes the relevance of two important components in our approach. The first one is the importance sampling from Nichol & Dhariwal (2021 1 and the second is the number of Social Transformers layers within residual denoising block. Table 12 shows the results of our approach without the importance sampling ("w/o IS") and with only one Social Transformer layer ("w ST"). Notice that the importance sampling is crucial when the size of
24

---

Published as a conference paper at ICLR 2026
the dataset is small, like the NFL and the Bundesliga. Using two Social Transformers also improve the results across the three datasets.
Table 12: Additional Ablation in Future Generation. We report SADE and SFDE metrics (min / aeg) and Acc (maz/ avg) over 20 generated modes.
Method
NFL (yards)
Bundesliga (meters)
NBA (meters)
SADE
Acc
S
SFDE
Acc
SADE J
SFDE J
Asc
w/o
2.74
09,899909
.75 350
2.78/ / 4,03
5.51/ 8.87
.65
1.44 /2.05
2.61 /4.00
f .44
w I x ST
2.46/3.55 3.54
/
778/.52
2.54/ / 3.69
5.18 8,332
.66 .38
1.44 /2.07
2.63 /4.05
.74 .43
Ours
2.36/3.40
5.53
.78/.54
2.47 / 3.66
5.02/8.29
.68/.39
1.39 /2.01
2.53/3.95
.75/
D.4 INTERPRETABILITY. ATTENTION ENTROPY
To understand how our joint modeling influences the learned correlations between agents, we ama- lyzed the attention entropy of our Social Transformer layers. Solving the future generation task in the NBA dataset, we generated and the ablated Ours w/o joint variant. We computed the entropy of the attention distribution for each agent at every layer and timestep. This entropy is defined as:
modes from batch of 128 using both our JointDifff (Ours) model
H(P) = H(P)=-> - P(En) loga P(x.);
(21)
where P is the attention distribution, and In is the n-th element in the set of agents (including the agent itself) over which the attention is computed. We then averaged this entropy across all attention heads, all four Social Transformer layers. all timesteps (T), all agents (N), and all 20 modes to obtain a single entropy value for each denoising step s. This per-step entropy provides measure of the uniformity of the attention patterns, with higher entropy indicating more uniform (less focused) attention and lower entropy indicating more specialized (highly focused) attention on specific agents.
The difference in this averaged entropy between the two models is depicted in Fig. Our analysis reveals that JointDiff maintains consistently lower attention entropy. The gap is most pronounced during the initial denoising steps, suggesting that agents in our model attend in a more focused man- ner from the beginning. This supports our hypothesis that providing the model with the possessor event allows it to immediately identify and prioritize the most salient interactions in the scene.
NFL
Bundesliga
NBA
Ours w/o joint
Ours w/o joint
3.4
Ours w/o joint
4.4
4.4-
Ours
Ours
Ours
Attention Eatropp
Arteation Eattrogs
Attcution Entrogge
3.2
4.2
4.2-
3.0
4.0
4.0-
2.8
3.8
3.8-
40
20
40
20
40
20
8
Figure 8: Entropy vs Denoising Step. We report the entropy of the inter-agent attention across the three datasets.
25

---

Published as a conference paper at ICLR 2026
GT
MoFlow
U2Diff
Ours
10
10
10
10
70
20
10
20
10
70
10
7
11:10
11100
110
Figure 9: Future Generation on NFL. Comparison of Ours vs. MoFlow and U2Diff baselines on generating future 40 timesteps conditioned on 10 past observed timesteps. Legend: team,
Ball, Home
Away team, O Past observations.
D.5 QUALITATIVE RESULTS
We provide further qualitative results in completion generation and controllable generation, while also showing failure cases. Please refer to the video supplementary to animated results.
D.5.1 COMPLETION GENERATION
This section provides qualitative comparisons of JifntDiff (Ours) against the state-of of-the-art genera- tive models MoFlow and U2Diff. We depict the mode with the best SADE metric over 20 generated modes in 9 for the NFL. in Fig 10 for the Bundesliga, and in Fig for the NBA.
D.5.2 CONTROLLABLE GENERATION
This section presents qualitative results for the controllable future generation task. For weak- possessor cossessor-guidance (Fig. 12), we compare a single generated mode from our full model (Ours) against the variant without joint training (Ours w/o joint). For text-guidance (Fig. [3], generate 20 modes for our method and qualitatively select the sample most aligned with the text description.
D.5.3 FAILURE CASES
A key limitation observed in results is the occasional inconsistency between the generated tra- jectories and the guxdagui This can be traced to the constrained size of our training datasets, 10k pairs for NFL and zs 2k (augmented to 4k) for Bundesliga. The limited data variety hinders
26

---

Published as a conference paper at ICLR 2026
GT
MoFlow
U2Diff
Ours
Figure 10: Future Generation on BundesLiga. Comparison of Ours vs. MoFlow and U2Diff baselines on generating future 30 timesteps conditioned on 10 past observed timesteps. Legend: Ball, Home team, Away team, O Past observations.
the model's ability robustly encode the wide range of possibilities described in natural language. Refer to 14 to see some failure examples from the same scenarios depicted in Fig.
E LIMITATIONS AND FUTURE WORK
Our model's architecture requires events to share the same centio-temppral structure as the trajectory data, i.e., allow for simple concatenation the input. This limits its application that are naturally structured this way and cannot directly handle unstructured data, such as sparse temporal point processes. A key direction for future work is develop methods integrate these more complex event types.
streams
second limitation stems from the NFL dataset. The public event data does not identify the player responsible for each action. Consequently, we must rely on combination of heuristics and tracking data to assign an actor, process that may lead sub-optimal outcomes.
27

---

Published as a conference paper at ICLR 2026
GT
MoFlow
U2Diff
Ours
Figure 11: Future Generation on NBA. Comparison of Ours vs. MoFlow and U2Diff baselines on generating future 20 timesteps conditioned on 10 past observed timesteps. Legend: Ball, Home team,
Away team, O Past observations.
F STATEMENTS
F.1 LLM USAGE
LLMs were used in two ways: (1) to improve the grammar and readability the manuscript, and (2) post-process the generated text dataset by correcting grammar and ensuring consistency (as described in the paper). All aspects of the research design, modeling, experimentation, and analysis were carried out independently of any LLM assistance.
F.2 ETHICS
This research uses trajectory data representing human agents. All datasets employed are either publicly available or synthetically generated, and contain no personally identifiable information. The trajectories and textual descriptions anonymized and represent abstract positions rather than identifiable individuals (as described in the paper). The intended applications this work include sports analytics and multi-agent simulation, which we believe pose minimal ethical risk.
F.3 DATASET
We commit to releasing the dataset and the code necessary to reproduce it upon acceptance of this paper for publication.
28

---

Published as a conference paper at ICLR 2026
Ours w/o joint
Ours
Ours w/o joint
Ours
20
20
30
40
20
30
40
20
30
40
1111
THU
1:11
100
000
11:011
10:01
0000
01000
m m m
m
(a) Players "[4, 8, 1I" involved in the possession.
(b) Players "[4, 8, 2J involved possession.
Ours w/o joint
Ours
Ours w/o joint
Ours
26
(c) Players "[16, 21J" involved in the possession.
(d) Players "[16, 22J" involved in the possession.
Ours w/o joint
Ours
Ours w/o joint
Ours
(c) Players "[4, 1, 1, 4J" involved the possession.
(f) Players "[4, 1, 2, 5]" involved in the possession.
Figure 12: Controllable Generation with WPG. Comparison Ours vs. Ours w/o joint on the puudenc weak-possesor guidance task giving the same past observations with different possessor sequences Gwrg. Legend: Ball, Home team, Away team, O Past observations.
29

---

Published as a conference paper at ICLR 2026
0
20
30
40
0
20
30
40
11
12
12
OT
20
O0
att ZZ
20
30
40
"Home Team has possession in SHOTGUN for- mation. Player snaps the ball Player 8 yard L. 25. Player possesses the ball and throws forward pass Player The hall travels yard 15 30."
(b) "Home Team has possession in SHOTGUN for- mation. Player 4 snaps the ball Player at yard L
25. Player 8 makes hand-off pass Player and he with the hall."
13
(c) "Away Team has the possession. The starts left-center. Player 19 possesses the ball left-center and passes to Player 22. The ball moves from left- center to box, then to up-corner. Player 22 possesses the ball at up-corner and attempts pass, which is in- tercepted by Home Player 7. Home Team gains pos- session and Player attempts clearunce, as the ball moves to box."
possession. The hall starts at (d) "Away Team possesses the ball at left-center left-center. Player 19 and passes to Player 22. The ball moves from left- center to box, then to up-corner. Player 22 possesses the ball at up-corner and attempts a pass to Player 19 inside the box."
Figure 13: Controllable Generation with Text. Examples on text-guidance task giving the same past observations with different prompts Gext- Legend: Ball, Home team, Away team, O Past observations.
30

---

Published as a conference paper at ICLR 2026
20
30
40
50
28
17
28
20
30
50
(a) "Home Team has possession in SHOTGUN for- mation. Player snaps the ball to Player 8 yard L 25. Player possesses the ball and throws forward pass to Player 5. The ball travels from yard L 15 to L 30."
(b) "Away Team has the possession. The ball starts at left-center. Player 19 possesses the ball at left-center and passes to Player 22. The ball moves from left- center to box, then to up-corner. Player 22 possesses the ball at up-corner and attempts a pass to Player 19 inside the box."
Figure 14: Failure Cases Controllable Generation with Text. Legend:
Ball,
Home team,
Away team, O Past observations.
31


Reviews:
[{"review_id": "z7b0b8L4N0", "reviewer": "Reviewer_nJy6", "summary": "This paper introduces JointDiff, a diffusion framework for multi-agent trajectory generation in sports. The framework is designed to unify continuous and discrete diffusion processes to simultaneously generate continuous spatiotemporal trajectories and discrete events (e.g., possession of ball). The paper presents two controllable generation scenarios (weak-possessor-guidance and text-guidance). The paper reports strong performance on scene-level metrics.", "questions": "I would like the authors to consider the broader landscape of related work, and then clarify what they view as the core novelty of this paper.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper's core hypothesis—that joint modeling of continuous trajectories and discrete events is beneficial—is validated on the domains of interest, as evaluated by scene-level metrics (Table 1).  The paper also successfully demonstrates two forms of semantic control (WPG and text-guidance -- shown in Table 2).  The paper also contributes a new text-annotated benchmark for the NFL and BundesLiga datasets, which enables the text-guidance task.", "weaknesses": "The authors claim this is the \"first to introduce this joint modeling paradigm to the domain of time-evolving multi-agent systems\". This claim, while true in its most narrow interpretation, overlooks a substantial and directly relevant body of un-cited prior art in adjacent fields like robotics and autonomous driving, where joint discrete-continuous diffusion models are already an active area of research for dynamic planning and generation.  (The authors do cite MotionDiffuser.)\n\nI would strongly encourage the authors to engage with the broader literature in these adjacent fields:\nhttps://arxiv.org/abs/2509.21983\nhttps://sites.google.com/view/dimsam-tamp\nhttps://generative-skill-chaining.github.io/\nhttps://arxiv.org/abs/2509.20109\nhttps://arxiv.org/abs/2405.15677\nhttps://arxiv.org/abs/2402.11502\nMy goal in listing these papers is not to say that any of them are doing exactly what the authors are doing.  But in characterizing the specific novelty of this paper, it is important to situate it in this broader context.  \n\nThe paper's architecture appears potentially over-engineered, consisting of a complex stack of modules (Temporal Mamba, Social Transformers, and CrossGuid) built upon the U2Diff architecture. The authors list limitations such as the \"architecture requires events to share the same spatio-temporal structure as the trajectory data\", but to me this type of limitation is a by-product of the engineering design.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T0LT3BnBFT", "reviewer": "Reviewer_5jzA", "summary": "The paper proposes JointDiff, a diffusion-based generative model for multi-agent trajectories in sports. This method jointly models the continuous trajectories and discrete events in multi-agent systems, enhancing the generation quality. The authors also introduce two new controllable generation tasks: weak-possessor-guidance (WPG), which conditions on an ordered list of possessors, and \"text-guidance,\" which conditions on natural language descriptions. To facilitate this, they propose an architectural module, CrossGuid, a cross-attention mechanism that injects the guidance signal into the diffusion model's denoising network. The authors validate their model on multiple datasets and release a new sports benchmark with enhanced text annotations. JointDiff demonstrates state-of-the-art results on completion/imputation tasks and demonstrating effective control in their new tasks. A human study is also provided, which shows a perceptual preference for the joint model's generations.", "questions": "1. What is the main contribution beyond the two prediction heads design and the cross-attention gating? Why is this different from prior joint continuous–discrete diffusion work?\n\n2. How does the model ensure consistency during generation? Since it learns to predict $Y_0$ and $E_0$ from two separate heads, is it possible for it to generate a final $(Y_0, E_0)$ pair that is inconsistent? Does this happen in practice?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. JointDiff is the first work to introduce joint continuos&discrete diffusion n modeling to time-evolving multi-agent systems. The authors also proposed new guidance methods for controllable generation and corresponding architecture (CrossGuid). In addition, a new sport benchmark is introduced. \n\n2. The evaluation on sport datasets is comprehensive. Three public sports datasets, NBA, NFL, and BundesLiga are covered, along with two tasks, completion and controllable generation. Human study is also provided.", "weaknesses": "1. A major concern is about the novelty of the proposed method. As stated in the related work, a couple of prior work in other domains have introduced joint diffusion modeling of continuous and discrete data. Can the authors further illustrate how this work distinguish with them in addition to different task settings?\n\n2. The CrossGuid module seems like a standard cross-attention block engineered for the sport multi-agent tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Op4N0zIklm", "reviewer": "Reviewer_Q4Be", "summary": "The paper propose Jointdiff, a diffusion-based framework tailored for trajectory generation in ball games. Jointdiff includes social-temporal block that models continuous multi-agent trajectory and discrete events in a single denoising process. Controllable generation is enabled via Crossguid, which injects external guidance signal represented by . The guidance could be weak-possessor-guidance or text descriptions. Jointdiff is evaluated with three types of ball games-- NBA, NFL and Bundesliga. Results shows Jointdiff achieves SOTA with respect to future prediction and imputation.", "questions": "1. How exactly is text guidance trained? Please specify the text encoder (e.g., frozen vs. finetuned), how its embeddings are injected (e.g., CrossGuid placement, keys/values/queries), and whether you use classifier-free guidance during training (drop-condition rate, loss formulation).\n2. Do you align text to trajectories/events with any auxiliary objectives (e.g., contrastive losses) or just condition the denoiser? Any ablations on encoder choice and guidance strength?\n3. Please report the full compute profile: GPU model/count, batch size.\n4. How controllable is the generation process? How is the sampling efficiency?\n5. How to deal with uncertainties in future generation? e.g., their might be multiple possible future scenarios given the same historical trajectory?\n6. Can the model be used to review the middle of a match? For example, a tactical switch or a player substitution?\n7. In your opinion, is there a foundation model that can model the trajectories of all ball games?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Principled diffusion-based joint modeling of continuous and discrete modalities. \n2. Controllable scene generation enable interesting applications, for example, it could simulates the outcome of different tactics, to make better decisions.\n3. Evaluation across three types of sports\n4. Supplementary materials includes many details about text guidance dataset generation, which could be useful for constructing text dataset for other use cases.\n5. Model architecture graph is very clear.", "weaknesses": "1. The main paper presents a broadly applicable diffusion architecture but offers limited methodological novelty; the contribution reads more application-driven than algorithmic. Key elements that could strengthen the perceived innovation—e.g., human evaluation design, text-dataset construction, and other data curation details—are largely relegated to the supplementary material. As a result, the main paper feels incremental and under-substantiated. The authors should write more these components into the main text, clearly describe what is genuinely new versus adapted, and include ablations that isolate architectural contributions from application engineering.\n\n2. The aligned training method of text and trajectory is not clear.\n\n3. The related-work discussion is thin and misses several foundational lines (e.g., NRI, dNRI, GRIN)[1,2] and key domain papers (e.g., TacticAI)[3], which weakens the paper’s positioning and novelty claim; at minimum, the authors should (i) contextualize how JointDiff differs conceptually, and (ii) add targeted comparisons/ablations or a rationale for incompatibilities.\n\n[1Kipf, T., Fetaya, E., Wang, K. C., Welling, M., & Zemel, R. (2018, July). Neural relational inference for interacting systems. In International conference on machine learning (pp. 2688-2697). Pmlr.\n\n[2]Li, L., Yao, J., Wenliang, L., He, T., Xiao, T., Yan, J., ... & Zhang, Z. (2021). Grin: Generative relation and intention network for multi-agent trajectory prediction. Advances in Neural Information Processing Systems, 34, 27107-27118.\n\n[3]Wang, Z., Veličković, P., Hennes, D., Tomašev, N., Prince, L., Kaisers, M., ... & Tuyls, K. (2024). TacticAI: an AI assistant for football tactics. Nature communications, 15(1), 1906.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wWsWhwSs3I", "reviewer": "Reviewer_nw5t", "summary": "The paper describes generating multi-agent trajectories in a sports scene, ensuring that the spatio-temporal trajectories of players and the ball are plausible. Motivated by the success of the recent diffusion-based generative framework, the paper proposes combining a discrete representation (ball possession) with a continuous representation (spatial coordinates of individual trajectories). Additionally, the framework can provide additional guidance of user intent, composed of text guidance or weak processor guidance. The results are presented with soccer and football datasets.", "questions": "Minor comment\n- temporal-evolving -> temporally evolving (line 051)\n- Why do you call the works non-generative in line 092?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The work proposes an important yet challenging problem, which generates controllable multi-agent trajectories. Theoretically, the framework combines the continuous and discrete representations in a consistent way and can be extended to many other large-scale control systems.", "weaknesses": "- Figures are hard to interpret. All the texts are too small, and the terminology is not clearly defined. Things to be in a clear layout with large enough text whose sizes are comparable to the main paper.\n\n- The exposition in Section 4.3 is weaker compared to other parts of the paper. It is hard to see how different components are put together, and the limited clarity of the figure does not help in understanding the structure. I suggest adding a simplified version of Figure 4 in the main paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)