Title: LaM-SLidE: Latent Space Modeling of Spatial Dynamical Systems via Linked Entities
Venue: NeurIPS 2025
Accepted: yes
Keywords: diffusion models

flow-matching

latent space models

latent diffusion models

dynamical systems

simulation

generative modeling

molecular dynamics

unified representations
Abstract: Generative models are spearheading recent progress in deep learning, showcasing strong promise for trajectory sampling in dynamical systems as well. However, whereas latent space modeling paradigms have transformed image and video generation, similar approaches are more difficult for most dynamical systems. Such systems -- from chemical molecule structures to collective human behavior -- are described by interactions of entities, making them inherently linked to connectivity patterns, entity conservation, and the traceability of entities over time. Our approach, LaM-SLidE (Latent Space Modeling of Spatial Dynamical Systems via Linked Entities), bridges the gap between: (1) keeping the traceability of individual entities in a latent system representation, and (2) leveraging the efficiency and scalability of recent advances in image and video generation, where pre-trained encoder and decoder enable generative modeling directly in latent space. The core idea of LaM-SLidE is the introduction of identifier representations (IDs) that enable the retrieval of entity properties and entity composition from latent system representations, thus fostering traceability. Experimentally, across different domains, we show that LaM-SLidE performs favorably in terms of speed, accuracy, and generalizability. Code is available at https://github.com/ml-jku/LaM-SLidE.


Peer reviews:
comments: SUMMARY:
The goal of the paper is to bring the advantages of latent modeling from image generation models into dynamical systems models. One part of the method focuses on modeling the IDs of trajectories so that idintifiable reconstruction is possible after latent space.

STRENGTHS AND WEAKNESSES:
**Weaknesses**

W1: GeoTDM comparison. While GeoTDM is basically the most important baseline and all experiments are taken from there, the differences with this method are not discussed in any section.

W2: Missing baselines. At this link https://paperswithcode.com/sota/trajectory-prediction-on-ethucy the first top 10 methods have better performance than the method proposed by this paper for the ETH-UCY dataset. The task is from Table 2 from the submitted paper.

W3: the improvement over GeoTDM is marginal.

QUESTIONS:
**Question 1**
>In order to leverage a latent systems representation, we need to be able to trace individual entities of the system

why? can you elaborate further. why needs to differ from image generators? 
> my interpretation: because you are tracing elements, trying to train a generative model in latent space. 

**Question 2**
Since this work is in latent space for dynamical system, how does it compare with methods such as: "Latent ODEs for Irregularly-Sampled Time Series" (NeurIPS 2019)? and why that type of work do not require IDs?

**Question 3**
> aspirin consists of 21 atoms, needs to have at least 21 unique identifiers

why? elaborate

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 1
DATE: 2025-06-28

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes LaM-SLidE, a novel framework for modeling spatial dynamical systems using latent space generative modeling with traceable individual entities. The method combines encoder–approximator–decoder architecture with identifier-based latent representations, allowing traceability of individual particles even when modeling is performed in a compressed latent space. A core feature is the ability to train and generalize across systems with variable numbers of entities, enabling parameter sharing and representation unification. The authors evaluate the method on a range of tasks, including pedestrian dynamics, basketball plays, molecular dynamics, and N-body systems, showing competitive or superior performance against a broad set of baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper tackles an important and nontrivial challenge—how to apply latent generative models to physical systems with variable-sized, traceable entities. The proposed approach is both technically novel and practically impactful.\n2. One of the most compelling aspects is the ability to jointly model systems of varying particle counts in a shared latent space, a feature often lacking in prior GNN- or diffusion-based methods.\n3. Experiments are comprehensive and cover diverse domains.\n4. The proposed framework is modular and general enough to be applied across domains.\n\nWeaknesses:\n\n1. Some symbols are defined but immediately undermined, or unused in the actual modeling (e.g., $M^t$ and equations (1–2)), leading to unnecessary conceptual load.\n2. The description of the latent dynamics approximator (especially from line 180 onwards) is vague and scattered. Even after reading Appendix B.4 and C, it remains unclear how the approximator operates in detail.\n3. While the model appears to support different input lengths $T_o$, it is unclear how this is explicitly handled.\n4. The paper does not explain why the modeling is not done step-wise in time if the system is Markovian. Such an autoregressive approach might be simpler and more interpretable.\n5. The paper lacks discussion on how the model handles or violates permutation invariance among identical particles, a property crucial in many physical systems.\n\nQUESTIONS:\n1. Lines 104–106 introduce $M^t$, but immediately state it is time-independent. Can this symbol be omitted in favor of a single static matrix $M$? Similarly, equations (1) and (2) formalize the Markov property, but this property is not used anywhere in the model. Would the paper be clearer without these?\n\n2. The paper defines the stochastic interpolation path in Eq. (6), with boundary conditions $\\alpha_1 = \\sigma_0 = 0$, $\\alpha_0 = \\sigma_1 = 1$. Shouldn't this be the opposite? Please clarify.\n\n3. The description of the approximator starting at line 180 is difficult to follow. Even after consulting Appendix B.4 and C, the architectural details and training procedure are unclear. Could the authors provide a more complete and explicit description of the model, perhaps with a step-by-step example or pseudocode?\n\n4. How does the model adapt to variable-length conditioning ($T_o$)? The framework appears to handle different observation lengths, but it is not clear how the approximator handles this in training or inference. Is $T_o$ used as an explicit condition, or is the model trained over varying $T_o$ lengths? This point deserves clarification.\n\n5. Why not use a step-wise (Markov) prediction? Given that the dynamics are assumed to be Markovian (as implied by Equations 1 and 2), why is the model trained to predict the entire future trajectory at once, rather than iteratively predicting the next step? A step-wise approach may allow for simpler and more interpretable modeling.\n\n6. For systems of identical particles (e.g., molecules with symmetric atoms), how does LaM-SLidE handle permutation invariance? The current identifier mechanism seems to break symmetry. Could the authors discuss the implications and whether permutation equivariance could be incorporated into this framework?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel approach that replaces GNNs for modeling spatial dynamical systems, using modern latent space generative modeling techniques instead. The key innovation is an identifier (ID) system where each entity is assigned a unique ID with learnable embeddings. These IDs are concatenated with entity positions and features, then encoded via cross-attention to a fixed number of latent tokens. A flow-matching model operates in this latent space for temporal dynamics, and a decoder uses ID-based queries to retrieve entity-specific information. The method is evaluated across diverse domains including molecular dynamics (MD17, Tetrapeptides), human motion (pedestrian, basketball), and N-body systems, achieving state-of-the-art or competitive performance while requiring 10-100x fewer function evaluations than baseline methods.\n\nSTRENGTHS AND WEAKNESSES:\n======\n\nStrengths:\n\n*Eliminates graph construction overhead: No need to learn to cluster and build the graph, as in GNN; learns to encode directly into fixed-size latent space, avoiding k-NN computations and edge message passing.\n\n*Enables modern architectures: The fixed-size latent representation enables usage of SOTA scalable generative models (flow matching, diffusion models) that have driven recent advances in image/video generation.\n\n*Superior performance: Achieves better performance than GNN-based methods across all tested domains, notably beating equivariant GNNs on MD17 without using domain-specific symmetries.\n\n*Unified architecture: Single model can handle molecules of different sizes (9-21 atoms in MD17), demonstrating better generalization than molecule-specific training.\n\n=====\n\nWeakness\n\n*Hyperparameter sensitivity: As mentioned in the paper, the pool size for potential entities matters. But tweaking this hyperparameter requires trial-and-error or domain expertise. More importantly, is it necessary to retrain the model for different pool sizes? Can models learned with different pool sizes share common knowledge such that we don't have to learn from scratch?\n\n\n*Dynamic entity handling is unclear: How does the method handle changes in entity count during dynamics? What if there are new entities entering the system? How are entity mergers or departures handled?\n\n\n*Tracking dependency: The method relies on perfect temporal tracking being provided by the dataset. How sensitive is the approach to tracking errors? What happens with identity switches or occlusions?\n\n*Semantic analysis missing: The paper needs more discussion on the semantic meaning of assigned entities given N entities and L latent vectors:\nFor N<L (overparameterization), do multiple latents correspond to the same entity or capture different aspects?\nFor N≥L (compression), does this lead to semantically meaningful clustering in latent space?\nI appreciate the brief discussion and results in Appendix G.3 on number of latent vectors, G.4 on pool size effects , though more extensive ablations would indeed clarify this.\n\n*(minor)Incomplete ablations: What drives the performance gains - the ID system, the flow matching model, or simply the larger model capacity? More ablations comparing to fixed-size GNN baselines would strengthen claims.\n\n\nIn summary,\nthis paper presents an innovative approach to modeling spatial dynamical systems by replacing the dominant GNN paradigm with fixed-size latent representations. The core contribution - the ID-based encoding/decoding system - elegantly solves the problem of maintaining entity identity while leveraging modern generative architectures. The experimental results are impressive, showing consistent improvements across diverse domains. So I'm leaning towards the acceptance of the paper.\n\nHaving said that, \n several limitations temper my enthusiasm. The requirement to pre-specify maximum entity counts and retrain for different pool sizes significantly limits the \"foundation model\" potential. The dependence on perfect tracking and inability to handle dynamic entity counts restrict real-world applicability. The paper would benefit from deeper analysis of what the model learns (e.g., embedding structure) and more thorough ablations to isolate the sources of improvement.\n\nQUESTIONS:\nPlease see the weakness part above,\n\nLIMITATIONS:\nPlease see the weakness part above,", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper introduces a model for generative modeling of stochastic trajectories. The model bridges the traceability of individual entities and the latent system representation. It uses the identifiers (IDs) that allow for the retrieval of entity properties. The author conducted a comprehensive experiment to validate the model.

STRENGTHS AND WEAKNESSES:
**Strengths**
1. The model introduces a novel approach by using identifiers (IDs) to allow traceability, closing the gap in previous generative modeling methods. 
2. The model has good performance across different domains, showing the general effectiveness of the approach.  

**Weaknesses**
1. The paper offers only limited discussion and ablation studies. More ablations on the design choices, such as identifiers (IDs), will be beneficial. 
2. The number of parameters for LAM-SLIDE is not indicated in Table 14.

QUESTIONS:
1. Could the author provide more insights into the results on the ETH-UCY and the NBA dataset, where LAM-SLIDE only shows marginal or no improvement? 
2. Does the author have any insights on whether the number of particles in the system affects the model performance? 
3. Could the author provide some explanation on why LAM-SLIDE 2.1M seems to underperform GeoTDM 1.9M (all→each) in Table 13?

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03