Neural Force Field: Few-shot Learning of Generalized Physical Reasoning
Authors: Shiqian Li
Ruihong Shen
Yaoyu Tao
Chi Zhang
Yixin Zhu
Venue: ICLR 2026 Poster (2026)
Keywords: Physical reasoning
few-shot learning
Primary area: causal reasoning
TL;DR: Few-shot learning and generalization in physical reasoning tasks

Abstract: Physical reasoning is a remarkable human ability that enables rapid learning and generalization from limited experience. Current AI models, despite extensive training, still struggle to achieve similar generalization, especially in Out-of-distribution (OOD) settings. This limitation stems from their inability to abstract core physical principles from observations. A key challenge is developing representations that can efficiently learn and generalize physical dynamics from minimal data. Here we present Neural Force Field (NFF), a framework extending Neural Ordinary Differential Equation (NODE) to learn complex object interactions through force field representations, which can be efficiently integrated through an Ordinary Differential Equation ( ODE) solver to predict object trajectories. Unlike existing approaches that rely on discrete latent spaces, NFF captures fundamental physical concepts such as gravity, support, and collision in continuous explicit force fields. Experiments on three challenging physical reasoning tasks demonstrate that NFF, trained with only a few examples, achieves strong generalization to unseen scenarios. This physics-grounded representation enables efficient forward-backward planning and rapid adaptation through interactive refinement. Our work suggests that incorporating physics-inspired representations into learning systems can help bridge the gap between artificial and human physical reasoning capabilities.

Published as a conference paper at ICLR 2026
NEURAL FORCE FIELD: FEW-SHOT LEARNING OF GENERALIZED PHYSICAL REASONING
Shiqian Li 1,2.5,6 Ruihong Shen 3,2.5,6* Yaoyu Tao 4.13
Chi Zhang 1.5%
Yixin Zhu 2,1,5,604
"qqual contribution
X Corresponding author
School Psychological and Cognitive Sciences, Peking University 3 School of EECS, Peking University School of Integrated Circuits, Peking University
Institute for AI, Peking University
State Key Lab of General AI, Peking University
Beijing Key Laboratory of Behavior and Mental Health, Peking University
htps://neuralforcefilldd htt.giecfiiiddggiuubooooooo
ABSTRACT
Physical reasoning a remarkable human ability that enables rapid learning and generalization from limited experience. Current AI models, despite exten- sive training, still struggle to achieve similar generalization, especially Out- of-distribution (OOD) settings. This limitation stems from their inability to ab- stract core physical principles from observations. A key challenge is developing representations that can efficiently learn and generalize physical dynamics from minimal data. Here we present Neural Force Field (NFF), a framework extend- ing Neural Ordinary Differential Equation (NODE) to learn complex object in- teractions through force field representations, which can be efficiently integrated through an Ordinary Differential Equation (ODE) solver to predict object trajecto- ries, Unlike existing approaches that rely on discrete latent spaces, NFF captures fundamental physical concepts such as gravity, support, and collision in continu- ous explicit force fields. Experiments on three challenging abstract physical rea- soning tasks demonstrate that NFF, trained with only a few examples, achieves strong generalization to unseen scenarios. This physics-grounde representation enables efficient tive refinement. Our work suggests that incorporating physics-inspire represen- tations into learning systems can help bridge the gap between artificial and human physical reasoning capabilities.
planning and rapid adaptation through interac-
INTRODUCTION
Physical reasoning. the ability to understand and predict how objects interact in the physical world, is fundamental to both human intelligence and artificial systems (Spelke, 2022). This capability under- lies crucial applications ranging from robotics to scientific discovery, making it central challenge in Al research (Lake et al., 2017). One of the remarkable aspects of human cognitive capabilities is the ability to rapidly learn from limited examples (Kim al., 2020; Jiang et al., 2022; Lake & Baroni, 2023; Zhang et al., 2024), especially evident in intuitive physics (Kubricht et al., 2017; Bear al., 2021). Humans can quickly abstract core physical principles after observing limited physi- cal phenomena, enabling them to predict complex dynamics and interact with novel environments (Spelke& & Kinzler, 2007; Battaglia al., 2013; Bonawitz al., 2019; Xu et al., 2021).
In contrast, current AI systems face significant limitations in physical reasoning. Despite being trained on gigantic datasets, these models still struggle achieve human-level generalization, par- ticularly in -itt----iittiittioo (OOD) settings (Lake et al., 2017; Zhu et al., 2020). The core issue lies in their tendency to overfit observed trajectories rather than capturing inherent physical prin- ciples, severely limiting their ability to compose known knowledge and predict outcomes in novel contexts (Qi et al., 2021; Li et al., 2022; Wu et al., 2023). This stark contrast between human ca- pabilities and current model limitations has motivated the search for new approaches that can learn generalizable physical representations from minimal data.
To bridge this gap, we aim develop agents with few-shot physical learning abilities that achieve robust generalization across diverse environments. This ambitious goal presents three fundamen- tal challenges: (i) Diverse Physical Dynamics: Physical systems exhibit intricate and nonlinear

---

Published as a conference paper at ICLR 2026
(a) Dynamic interaction graphs T-2
(c) Generalization
T-4
T-5
Falling
Sliding
Suppor
Collisiin
(b) Neural operator force fields
Rotation
Contaimment
Trunk
Force
Query objects
Force fields
Learned physicul
Branch
concepts
training examples
OOD template
Input space
DeepDNet
Output function
Figure I: The framework of NFF. (a) NEF models complex physical interactions by constructing dynamic interaction graphs from seenes and performing continuous integration force fields. (b) The force fields inferred by a neural operator that takes object interactions as input. (c) After learning from a few examples, NFF can capture various physical concepts represented by force fields and generalize unseen OOD scenarios.
dynamics shaped by complex object properties and interactions. OOD scenarios often present dras- tically different dynamics from training examples, requiring sophisticated representations that ex- plicitly capture core physical principles. (ii) Risk of Overfitting: The few-shot learning setting dramatically increases the challenge of generalization compared to large-scale training approaches. Models must carefully balance between fitting observed examples and extracting broader physi- cal principles. (iii) Interactive Reasoning: Effective physical reasoning demands more than pas- sive observation agents must actively engage with their environment through experimentation and feedback, adapting their understanding based on limited examples.
To address these challenges, we introduce Neural Force Field (NFF), a neural function map from objects to forces acting continuous Euclidean coordinates for efficient interactive learning and reasoning. At its core, NFF employs a neural operator to learn dynamic force fields from external interventions and object interactions. These predicted forces are then integrated through an ODE solver to compute explicit physical variables such as velocity and displacement, producing inter- pretable results that align with established physical principles. Please refer Figure for details.
Our framework offers three advantages. First, by representing physical interactions low- dimensional neural operator force fields, NFF can rapidly learn fundamental physical concepts from just a few training examples. Second, due to the ODE-grounded dynamic graph, NFF can effec- tively generalize to OOD scenarios. Third, the integration of forces through an ODE solver enables fine-grainee physical interactions, supporting precise modeling of collisions and gravity effects.
We evaluate our method using two state-based physical reasoning Menchmarks-I-PPYRRE eLi 2023) and N-body problems (Newton, 1833)-as well as one vision-based benchmark, PHYRE (Bakhtin et al., 2019). These tasks feature complex rigid-body dynamics ranging from shont-range forces (collision and sliding) to long-range forces (gravity). Our experiments demonstrate that NFF not only learns dynamics efficiently by abstracting physical interactions into force fields but also achieves strong generalization in both framework's ' physics-based representation enables effective forward and backward planning in goal- directed tasks, consistently outperforming existing methods such RPIN (Qi et al., 2021), EGNN (Satorras et al., 2021), SEGNO (Liu et al., 2024b), and transformer based methods (Wu et al., 2023).
and cross-scenane settings. Moreover, the
RELATED WORK
Physical reasoning Research in physical reasoning has progressed along two main trajectories: passive observation and interactive platforms. The passive observation approach, exemplified by the Violation-of -frr--ettatttiiion (VoE) paradigm (Speike et al., 1992), evaluates physical understanding by measuring agents' ability to detect violations of physics principles (Baillargeon, 1987; Hespos & Baillargeon, 2001; Dai et al., 2023). While this approach has provided valuable insights into basic physical comprehension, it is limited in assessing active interventions and complex reasoning.
2

---

Published as a conference paper at ICLR 2026
To enable more comprehensive evaluation, interactive platforms such PHYRE (Bakhtin et al., 2019), the virtual tools game (Allen et al., 2020), and I-PHYRE (Li et al., 2023) have emerged. These environments require agents to actively manipulate objects to achieve specific goals, testing not only prediction capabilities but also planning and reasoning skills. However, current physical reasoning models face primary challenges: the need for extensive training data (Qi et al., 2021; Liet al., 2023) and limited cross-scenario transfcrability. While some methods achieve strong perfor- mance within specific scenarios (Allen et al., 2020), they often fail generalize their understanding to novel situations, falling short of human-level reasoning capabilities (Kang et al., 2025).
Object-centric Dynamic prediction The prediction of physical dynamics between objects is fun- damental to intuitive physics. Many approaches (Battaglia et al., 2016; Qi et al., 2021) adopt Graph Neural Networks (GNN) as their backhone, capitalizing on their relational inductive bias for model- ing object-centric interactions. Key enhancements include integrating E(3) equívariant architectures to generalize across rotations and translations (Satorras et al., 2021; ; Han al., 2022), and incor- porating NODE for intrinsic continuity modeling to improve long-term prediction accuracy (Poli et al. 2019; Yuan et al., 2024; Liu et al., 2024b). Some works inject physics inductive bias into simulation such mesh or SDF (Allen et al., 2023; Rubanova et al., 2024) for collision handling and spring-mass mass models or particle-grid representations (Jiang et al., 2025; Zhang et al., 2025) for deformable objects. However, these methods are mainly designed for particle-based systems which hardly fit into abstract physical reasoning tasks like PHYRE. More importantly, these methods fail to extract robust physical representations, which severely hinders their performance in few-shot learn- ing scenarios. More recent methods like SlotFormer (Wu et al., 2023) and SlotSSM (Jiang et al., 2024) leverage transformer or mamba for temporal sequence modeling with a slot attention (Lo- catello et al., 2020) module for unsupervised object detection, but still struggle with cross-scenario generalization.
Our work advances continuous-ting methods to model complex rigid body interactions via learnable dynamic force fields. This approach enables robust cross-scenario generalization while handling non-conservative energy systems. By combining the stability advantages of continuous-tinn model- ing with flexible force field representations, NEF achieves both accurate long-term predictions and strong generalization capabilities without requiring explicit physical priors. More related work can be found in Section A.
METHOD
3.1 THE NEURAL FORCE FIELD (NFF) REPRESENTATION
Traditional approaches to modeling physical interactions typically rely on implicit representations through hidden vectors to describe object interactions. These methods use neural networks to learn state transition functions that map current scene features to future states. However, this purely data- driven approach cannot guarantee physically grounded representations, leading to poor generaliza- tion despite intensive training. We introduce NFF, which addresses these limitations by learning physics- physics-grounded, grounded, generalizable representations through explicit force field modeling; see Figure for comparison.
Predicting force field The core of NFF is built on the physical concept of force field-a vector field that determines the force experienced by any query object based on its state z For a scene with objects, we model the force field F.)) at time t using a neural network operating on a rela- tion graph G. The graph contains object nodes V = (t), (t), ..., ~N-((t). Following neural operator methods (Lu et al., 2021) and graph neural models (Battaglia et al., 2018), we formulate the force field function as:
F(2(())= = Σ w (blz + +b,
(1)
(Eg(a)
where ((g) represents the neighbors of query determined by contacts or attractions, fo and fa are neural networks with parameters 9 and d. and W E Rdhus x diece b E Rows map hidden features to dwwddmmnsiioall forces. The state vector incorporates geometry states (angle, length, radius), optionally physical attributes (mass), zero-order states (position), and first-order states (velocity, angular velocity). This representation handle various physical interactions including collision,
3

---

Published as a conference paper at ICLR 2026
sliding, rotation, and gravity, as shown in Figure For image-based NFF, utilize object masks to replace the geometry states in imput.
Decoding trajectory with ODE Rather than relying on neural decoders, NFF employs a second- order ODE integrator to compute object trajectories from the learned force field. This approach ensures physically consistent trajectories governed by fundamental principles. The dynamic change of motion for query state follows:
d') d12
(t) di
F(z WAR
(2)
While Equation (2) explicitly incorporates object mass m our formulation is flexible regarding mass availability. In scenarios with known physical properties, m serves Conversely, when mass is not provided as state input, the network F(.) learns to implicitly encode mass information into its parameters via end-to-end optimization. In this implicit setting. F(-) effectively predicts mass-normalized forces (i.e., accelerations) by leveraging correlated features such object geometry semanties.
explicit divisor.
Solving ODEs We solve the ODE sys- tem using integration methods such as Lunge-Kuttte and Euler:
Prediction
ODESolve (0),
(3)
x(t) x(0)
v(t) dt,
Ja v(t) v(0)
(4)
(()
dt,
uajectory velocity
Transition model
The NFF framework offers three key ad- vantages over existing approaches. First, it enables few-shot learning of force fields from minimal examples-en even single majectories-througgh its low-dimensional, physically-grounded
T+1
d+
T+1
representation Equation (2).
(a) Discrete latent transition
(b) Continuous explicit transition
(t)) adhering Second, by representing interactions as neural operator force fields, as shown in Equation (1), NFF can generalize the learned force patterns to novel scenarios through simple force summation new interaction graphs, allowing modeling varying object compositions, interaction types, and time horizons. Third, the high- precision integration decoder, formulated in Equations (3) and (4), enables both fine-grained modeling of continuous-time interactions and efficient optimization for planning tasks through its invertible information flow. This precision supports various applications, from determining initial conditions for desired outcomes designing targeted force fields for specific behaviors. The effectiveness of these three components- -ODE grounding, Neural Operator Learning (NOL)), and precise integration-is systematically validated through ablation studies in Section 4.5.
Figure 2: Comparison between discrete latent transition in traditional interaction modeling and continuous plicit transition NFF. The figures in the first com- pare the predicted discrete steps and continuous trajectories, in which the discrete decoding cannot explain how the green ball goes through the black wall. In the sccond line, the black arrows indicate velocity vectors and the red and green arrows indicate force vectors. Traditional interaction models such as Interaction Network (IN) and SlotFormer decode discrete frames from a learnable decoder. In contrast, NFF decodes trajectories by integrating the continuous force using ODE solver. which benefits learning detailed interactions such as collisions.
Training NFF The framework employs autoregressive prediction, using previously predicted states to generate current states. Network optimization minimizes the Mean Squared Error (MSE) loss between predictions and ground truth. To promote learning of local, generalizable dynamics rather than global patterns, segment long trajectories into smaller units during training. This al- leviates the issue of accumulated error teacher forcing while reducing the complexity recursive training. In evaluation, the model predicts all future dynamics given only the initial states.
4

---

Published as a conference paper at ICLR 2026
0.6
Ground truth
0.4
0.2
0.0
Learned one
0.4
0.2
(a) PHYRE force field
(b) Gravitational field
Visualization of learned force fields after few-shot learning. (a) NFF successfully inverts force fields across different templates PHYRE, consisting of physical hehaviors like falling under gravity, sliding with friction, and colliding with momentum transfer. (b) The learned gravitational field distributions N-body also show accurate force field reconstruction.
Figure 3:
3.2 INTERACTIVE PLANNING
NFF extends beyond forward prediction to enable mental simulation for planning tasks. Given goal- directed scenarios, NFF can serve as a learned simulator either search for action sequences that achieve the desired goal state through forward simulation, or optimize initial conditions and system parameters through backward computation.
Forward planning For physical reasoning tasks requiring specific action sequences, we extend the NFF framework incorporate action effects by including action features in fo- Through for- ward simulation with the NFF model, we sample multiple action sequences A = [ao,..., ...,, @r } and optimize according to evaluation metrics R: A" = argmax.((F, A). The optimal sequence A can
then be executed in the physical environment. While the forward simulation may initially deviate from actual observations, NFF's soysscc--asedd design enables efficient adaptation through new ex- perimental data, inspired by human trial-and-error learning (Allen al., 2020). When executed trajectories deviate from the desired goal state, the new state sequences can be directly incorporated into model optimization through the MSE loss, following the same training procedure used in the initial learning.
Backward planning The invertible nature of the NFF formulation makes backward computation particularly efficient. By inverting the time direction of ODE integrator, NFF enables the inversion of initial conditions given desired goal state: x(0) x(f)+ + S vt) dt, v(0) v(t) S F(z dt, consistent with the learned physical dynamics encoded in the NFF model, as shown in Section 4.4.
4
EXPERIMENTS
4.1
ENVIRONMENTS DATASETS
We evaluate state-based NFF on I-PHYRE and N-body dynamics and vision-based NFF on PHYRE. Each task is evaluated across three distinet settings: within- sichin-cceeario prediction, cross-scenario pre- diction, and planning. Detailed environment and dataset configurations provided in Section B.
I-PHYRE I-PHYRE (Li et al., 2023) presents a suite of complex physical reasoning puzzles requiring multi-step interventions. The environment incorporates diverse physical interactions in- cluding gravity, collision, friction, rotation, spring dynamics, and pendulum motion. It challenges Al agents solve puzzles with minimal environmental interactions while generalizing to unseen scenarios. For within-scenario prediction, we evaluate on 10 training games that share similar see- narios but require different solutions. The cross-scenario prediction setting extends to 30 novel
5

---

Published as a conference paper at ICLR 2026
True
NFF
SlotFormer IN
I-PHYRE
N-body
Figure 4: Trajectory predictions on unseen scenarios after few-shot learning. After learning from 100 and 200 trajectories on I-PHYRE and N-body systems, respectively, our NFF predictions closely match the ground truth behaviors across diverse scenarios, from rigid body interactions gravitational dynamics. In contrast, other baselines fail predict physically plausible dynamics. Additional visualizations are provided Section I.
games featuring noise, compositional elements, and multi-ball scenarios, with varying object prop- erties such as block lengths, ball sizes, and object positions. In the planning setting, models must generate optimal action sequences to successfully complete each game.
N-body The N-body problem (Newton, 1833) tests trajectory prediction for small comets orbiting massive central planet. Using REBOUND simulator (Rein & Liu, 2012), we generate dynamics by randomly sampling orbital parameters including radii, angles, and masses. This task evaluates model's ability to infer gravitational laws from limited observations. The within-scenari predic- tion introduces novel initial conditions and masses, focusing on systems with 1-2 orbiting bodies tracked over 50 timesteps. Cross-scenario prediction significantly increases complexity by intro- ducing systems with or orbiting bodies tracked over 150 timesteps. The planning setting challenges models to optimize initial conditions that will evolve to specified target states after 50 timesteps.
PHYRE Building on the previous datasets that used state-based representations of objects, we explore extending NFF to a vision-based physical reasoning benchmark, PHYRE (Bakhtin et al., 2019). Designed for goal-oriented physical reasoning, PHYRE features 25 task templates involving interactive physics puzzles, all solvable by strategically placing a red ball. Each template generates 100 variations of core task, supporting two distinct evaluation protocols: (i) the within- -ccenaio setting, where models train on 80% of the tasks in each template and are tested on the remaining 20%; and (ii) the cross-scenario setting, which evaluates generalization by training on tasks from 20 templates and testing on tasks from the 5 unseen ones.
4.2 LEARNING FORCE FIELDS FROM A FEW EXAMPLES
We qualitatively evaluate NFF's ability inverse force fields from limited observations without direct access to ground-truth forces; training details are provided in Section C. For I-PHYRE, we train NFF on 10 basic games with 10 trajectory samples each. From these 100 trajectories, the model successfully learns fundamental physical concepts through unified force fields. To verify that NFF learns generalizable physical principles, we examine force responses to varied ball-block interactions under controlled conditions. Figure A1 shows our systematic evaluation where we in- dependently vary individual parameters such as position, velocity, and angle while holding others constant, demonstrating NFF's accurate force response modeling. For the N-body system, we train NEF using 200 randomly sampled trajectories from 2-body and 3-body dynamics. As shown in Figure 3b, the model successfully learns to capture the inverse gravitational field, correctly model- ing the distance-dependen centripetal forces governing the mutual attraction between bodies. For PHYRE, train vision-based NFF on 12,000 trajectories (20 templates x 20 tasks 30 actions),

---

Published as a conference paper at ICLR 2026
Table 1: NFF outperforms baselines prediction tasks. Comparison of baselines on I-PHYRE and N- body tasks. Results are reported with Standard Error of the Mean (SEM); lower is better (L) for error metrics (RMSE, FPE, PCE), and higher is better (f) for correlation (R). NFF consistently yields lower errors and higher correlations, especially under cross-scenario generalization. The [0,3T] settings extend prediction from 50 to 150 steps.
I-PHYRE
N-body
Metric
Model
Within
Cross
Within [O,T]
Within [0,3T]
Cross [0,3T]
IN
0.124 a 0.007
0.194 0.004
0.200 0.015
0.752 0.026
6,942 a 0.235
RMSE
Slotformer SGENO
0.067 0.006
0.206 0.005
0.214 0.018
1.092 0.043
2.533 0.068
0.203 0.070
0.314 0.106
0.079 0.009
0.690 = 0.144
2.759 = 0.076
NFF
0.048 = 0.004
0.131 + 0.004
0.097 =: 0.009
0,525 = 0.026
1.226 + 0.023
IN
0.129 0.010
0.178 0.005
0.198 0.014
0.785 = 0.027
.568 0.492
FPE |
Slotformer
0.068 0.007
0.198 0.005
0,215 0.017
1.311 = 0.064
2.751 = 0.060
SEGNO
0,212 =: 0.082
0,289 0.109
0,074 0.009
1.535 0.056
2.302 = 0.062
NFF
0.042 = 0.004
0.111 0.004
0.104 0.009
0.592 0.029
1.418 0.029
IN
0.004 + 0.000
0.005 0,000
0.012 0.001
0.029 0.002
0,101 0.003
PCE! 1
Slotformer SEGNO
0,002
0.004 0.000
0.013 0.002
0.035 0.002
0.053
0.008 0.008
0.017 0.018
0.005 + 0.001
0.022 = 0.004
0,063 = 0.001
NFF
0.001 = 0.000
0.003 + 0.000
0,007 = 0.001
0.021 = 0.002
0.035 = 0.002
IN
0.993 a 0.001
0.974 0.001
0,984 0.002
0.895 a 0.009
0,287 a 0.015
R 1
Slotformer
0.997 = 0.001
0,967 + 0.001
0,977 0.003
0,854 a 0.010
0,4384 = 0.019
SEGNO
0.977 0.026
0,928 0.064
0,996 0.001
0,947 0,010
0,291 = 0.013
NFF
0.997 0.001
0.980 0.002
0.996 0.001
0.949 0.006
0.803 0.010
which is about 267 times less data than prior SOTA (Qi et al., 2021). As shown in Figure 3a, the model accurately captures force fields across diverse templates.
4.3 PREDICTION ON UNSEEN SCENARIOS
We evaluate the learned force fields' predictive accuracy in both within- and cross-scenario settings. For I-PHYRE, we test against 20 ground truth trajectories per game, while for N-body, we eval- uate using 200 unseen initial conditions. We compare against established state-of-the-arr (SOTA) interaction modeling methods: vanilla IN (Battaglia et al., 2016), SEGNO (Liu et al., 2024b), and SlotFormer (Wu et al., 2023). Note that for fair comparison, we directly feed encoded state in- formation into SlotFormer to avoid inaccurate perception caused by slot attention. More baseline performances on N-body dataset can be found in Section
Evaluation metries We employ multiple complementary metrics to assess prediction quality. Beyond standard Root Mean Squared Error (RMSE), we introduce Final Position Error (FPE) and Position Change Error (PCH) for detailed performance analysis. FPE quantifies terminal position ac- curacy, while PCE evaluates the model's ability to capture motion dynamics. Additionally, Pearson Correlation Coefficient (R) measures trajectory shape alignment independent of speed variations. This comprehensive metric suite enables thorough evaluation across temporal and spatial dimen- sions. Detailed definitions are provided in Section C.2.
Results As shown in Figure 4, NFF generates physically plausible trajectories that closely match ground truth behavior, even in previously unseen scenarios. Quantitative results in Table 1 demon- strate NFF's superior performance across all metrics for both I-PHYRE and N-body tasks, with particularly strong results in cross cross-scenario generalization. While other models exhibit overfitting tendencies that limit the cross-scenario performance, NFF's ability to learn generalizable physical principles enables robust prediction across diverse scenarios.
4.4 PLANNING ON UNSEEN SCENARIOS
The trained NFF model generate plans for novel tasks after learning from limited demonstra- tions. Unlike prediction tasks that evaluate trajectory accuracy, planning tasks require generating action sequences to achieve specific goals.
I-PHYRE planning We implement a 5-round interactive learning protocol. NEF acts as a mental simulator to evaluate 500 randomly sampled action candidates (when eliminate which block in
7

---

Published as a conference paper at ICLR 2026
True
NFF
SlotFormer RPIN
PHYRE within
PHYRE CTOSS
Figure 5: Trajectory Predictions PHYRE. We compare the iision-based NFF, trained 12,000 trajecto- ries, with SOTA methods trained 3.2 million trajectories. Prior methods decoding objects from latent spaces often suffer from object inconsistency. into a gray ball, indicating overfitting to training shapes. SlotFormer also shows object disappearance in cross- scenario tasks. In contrast, NEF produces more accurate predictions while maintaining object consistency.
example, the last column, RPIN mistakenly transforms a gray cup
order drop the balls into the abyss), selecting the optimal sequence for physical execution. After each execution, the model updates its parameters based on observed outcomes, refining its physics understanding. This updated model then guides subsequent action proposals.
We quantitatively evaluate planning by calculating success probability p, for each game as the suc- cess rate over 20 trials in round i, with NEF updating after failures; detailed results in Section G. Figure 6 compares NFF against random sampling, human performance (from Li et al. (2023)), IN, and SlotFormer using cumulative success probability: 1 11"", (1 P:) for current round n. NFF outperforms baselines and approaches human-level performance after refinement (Allen et al., 2020), demonstrating effective few-shot learning. The performance of IN and SlotFormer, falling below random sampling. indicates how inaccurate dynamics modeling compromises planning.
N-body planning We focus on determin- ing initial conditions that achieve desired final configurations under celestial dynamics. NFF's inverse simulation enables direct computation of initial conditions through reverse time evo- lution, while IN and SlotFormer rely on iter- ative gradient- based refinement. As shown in Table NEF achieves superior planning perfor-
IPHYRE interactive planning results
Probabiitty
mance in both within- and cross-scenare set- tings. Moreover, its reliance gration provides a substantial speed advantage, further evidenced by the planning time eval- uation in Table A3.
backward inte-
Modal
refine
NF
IN
Stotformer
Random
Number of rounds
PHYRE planning In PHYRE, extend NFF to learn dynamics directly from RGB videos with segmentation masks by applying sparse convolutions to extract geometric fea- tures from the masked objects. Following PHYRE's standard evaluation protocol, we first utilize NFF to predict trajectories and then use pretrained TimeSFormer (Bertasius et al., 2021) as the classifier to determine task suc- cess or failure. Planning performance is measured by the AUCCESS metric (Bakhtin et al., 2019), which captures both effectiveness and efficiency by assigning higher weight to solutions that suc- ceed with fewer attempts. Specifically, each attempt count k. 1,..3..... 100 is weighted by
Figure 6: Interactive planning performance I- PHYRE. Comparison cumulative success probabil- ity over planning rounds among human, random sam- pling, our NFF, IN, and SlotFormer. Error bars show SEM across trials. Our NFF with refinement (Allen al., 2020) shows continuous improvement rounds, achieving performance comparable human (data from et al. (2023)), while others show limited performance even with refinement.
8

---

Published as a conference paper at ICLR 2026
wi log(k + 1) - log(()), and the final AUCCESS score computed as: success rate within attempts.
N weitk where 83 the E. wi
Table 2: N-body system initial condition reconstruc- tion from target configurations. Results show aver- age MSE with SEM across trials for both within- and cross-scenario evaluations.
Table 3: Planning performance on PHYRE in the cross-scenario setting, Training sample size (in mil- lions) and AUCCESS 1 score are reported. Our NFF achieves the best performance with the least data.
Sccnario IN
SlotFormer
NFF
Metric
DQN
RPIN
SlotFormer
NFF
Within L 0.651 + 0.021 0.837 + 0.029
0.067 + 0.010
Sample (M)
200.000
3.200
0.120
0.012
Cross 1 4.654 + 0.193 4.018 0.133
0.140 = 0.011
AUCCESS
30.96
36.58
21.04
49.22
We compare NFF against the RPIN (Qi et al., 2021) and SlotFormer (Wu et al., 2023). As shown qualitatively in Figure 5, NFF, trained on just 0.012 million trajectories, learns physically grounded dynamics that support effective planning while maintaining higher fidelity and stronger object per- manence than RPIN and SlotFormer, particularly in cross-scenario generalization. Quantitative re- sults in Table 3 further demonstrate that NFF outperforms SOTA baselines in the challenging cross- scenario tasks. In contrast, SlotFormer's slot attention leads to limited generalization in unseen objects, resulting in planning failures in cross scenarios; see Figure A6 for more analysis.
4.5 ABLATION STUDY
We investigate three key factors that may affect NEF performance, including integration precision, ODE grounding, and NOL. For integration precision, we evaluate NFF on the N-body task using Euler integration at two precision levels: 1e - 3, 5e - 3 and adaptive integration with tolerance level: 1e 5. Table 4 demonstrates that higher integration precision consistently yields better performance. We also explore more integration methods in Section E. To study ODE grounding's impact on generalization, we compare two configurations: our NFF using predicted force fields with ODE integration, and IN using learned state transitions matching step sizes. Table 4 shows that ODE integration enhances generalization performance. For NOL, we replace the DeepONet in NEF with vanilla MLP. The results in the last column of Table 4 show a consistent increase in generalization errors across settings compared with NFF with DeepONet (column one). These demonstrate that NFF achieves optimal few-shot learning of physical dynamics when combining high integration precision, ODF-based integration, and NOL.
5 DISCUSSION
Extending NFF to real stimuli While NFF has demonstrated effectiveness in modeling funda- mental physical forces like gravity, support, and collision on synthetic datasets, extending NFF to real-world environments remains unexplored. A promising avenue lies in harnessing the strong gen- eralization capabilities of vision foundation models to translate complex, real-world visual inputs into object-centric representations compatible with NFF, Despite the challenges of perceptual noise and diverse physical interactions, we believe this could enable the framework to ground high-level visual perception in explicit force-based dynamics modeling, effectively bridging the gap between video prediction and human-level physical reasoning.
Limitations While our experiments focus on generalization across varying object masses and object compositions in controllable abstract reasoning tasks, training a single model with varying friction and elasticity may pose additional challenges. Furthermore, for clear and constrained validation of learning forces, we assume a deterministic environment with rigid bodies and have not
Table 4: Ablation results emphasize the need for high integration precision, ODE grounding, and NOL. We test NFF with different integration precision. The NEF without ODE grounding degenerates to IN with the same integration precision (5e-3) and the NFF without NOL utilize an MLP as force predictor instead of DeepONet. NEF with high integration precision, ODE, and NOL yields the best generalization results.
Scenario
NFF le-3
NFF 5e-3
NFF Adaptive
w/o ODE
w/o NOL
Training 0.08020.009
0.107
0.07220.008
0.04420.005
0.05620.005
Within
0.097
0. 001
0.092+0.007
0.354=0.023
0. 0.120
Within [0,3T]
0.525+0.026
0.557+0.025
0.613++0037
0.949#n.041
0.540+0.042
Cross
1.226*0.023
1.25 +0.005
1.788+0.375
3.51 3.51880.061
1.347+0.128

---

Published as a conference paper at ICLR 2026
explored the model's performance in stochastic environments those containing soft bodies and fluids.
The proposed NFF framework could enable more sample-efficient and inter- pretable physical modeling, reducing data needed for training. We acknowledge potential risks misused for simulating harmful physical scenarios. However, the research focuses basic physical principles and publicly available phenomena, prioritizing transparent and reproducible science.
Impact statements
CONCLUSION
We present NFF, force field-based representation framework for modeling complex physical inter- actions that exhibits human-like few-shot learning, generalization, and reasoning capabilities. Our experiments on three challenging abstract reasoning datasets with complex multi-object interactions demonstrate NFF's ability to not only learn diverse physical concepts and rules such as gravity, col- lision, friction, and attraction from limited observations but also generalize to unseen scenarios for both prediction and planning tasks. This initial exploration of the force field may provide per- spectives for developing physical world models through representation learning, potentially bridging the gap between human physical understanding and learning-based methods.
Reproducibility statement To support reproducibility, provide detailed documentation on the data and environment configurations in Section B, describe the model implementation in Section C, explain the evaluation metries in Section C.2, and present the ablation studies and additional results in Section D, Section E, Section F, and Section G. Both the datasets and code will be made publicly available.
ACKNOWLEDGMENT
The authors would like to thank Miss Chen Zhen (BIGAI) for making the nice figures, Prof. Yizhou Wang (Peking University) for the helpful discussion, and Minyang Yu (Peking University) for work- ing on the GPT's results. This work is supported in part by the Brain Science and Brain-like Intel- ligence Technology National Science and Technology Major Project 2025ZD0219400) the Na- tional Natural Science Foundation of China (62376009), the State Key Lab of General Al Peking University, the PKU-BingJi Joint Laboratory for Artificial Intelligence, the Wuhan Major Scientific and Technological Special Program (2025060902020304) the Hubei Embodied Intelligence Foun- dation Model Research and Development Program, and the National Comprehensive Experimental Base for Governance of Intelligent Society, Wuhan East Lake High-Tech Development Zone.
REFERENCES
Kelsey R Allen, Kevin A Smith, and Joshua B Tenenbaum. Rapid trial-and-error learning with sim- ulation supports flexible tool use and physical reasoning. Proceedings of the National Academy of Sciences (PNAS), (17447))2332--9311 2020. 8,
Kelsey R Allen, Yulia Rubanova, Tatiana Lopez-Guevarg William Whitney, Alvaro Sanchez- Gonzalez, Peter Battaglia, and Tobias Pfaff. Learning rigid dynamics with face interaction graph networks. In Proceedings of International Conference Learning Representations (ICLR), 2023.
Renée Baillargeon. Object permanence 2-and 41 2-month-old infants. Developmental Psy- chology, (3(5):655-664 1987. 2
Anton Bakhtin, Laurens der Maaten, Justin Johnson, Laura Gustafson, and Ross Girshick. Phyre: A new benchmark for physical reasoning. In Proceedings of Advances in Neural In- formation Processing Systems (NeurIPS), 2019. 2, 3,
Peter Battaglia, Razvan Pascanu, Matthew Lai. Danilo Jimenez Rezende, al. Interaction net- works for learning about objects, relations and physics. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), 2016. 3.7
10

---

Published as a conference paper at ICLR 2026
Peter W Battaglia, Jessica B Hamrick, and Joshua B Tenenbaum. Simulation an engine of physical Scene understanding. Proceedings of the National Academy of Sciences (PNAS), 2013.
Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanzhez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv: 17X.v:800.0012600 01261, 2018. 3
Daniel M Bear, Elias Wang, Damian Mrowca, Felix Binder, Hsiao- Yu Fish Tung. RT Pramod, Cameron Holdaway, Sirui Tao, Kevin Smith, Fan- Sun, et al. Physion: Evaluating physical prediction from vision in humans and machines. In Proceedings of Advances in Neural Informa- fion Processing Systems (NeurIPS), 2021.
Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video understanding? In Proceedings of International Conference on Machine Learning (ICML), 2021.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rom- bach. Stable video diffusion: Scaling latent video diffusion models to large datasets, 2023. URL
https: /arxiviivvvvv./////////////aaaiv org/ abs/ /2311111112 15127. Al
Elizabeth Bonawitz, Tomer D Ullman, Sophie Bridgers, Alison Gopnik, and Joshua B Tenenbaum. Sticking the evidence? behavioral and computational case study of micro-theory change in the domain of magnetism. Cogmitive science, 43(8)=e12765, 2019. I
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Proceedings of Advances in Neural Information Processing Systems
(NeurIPS), 2018. Al
Ricky TQ Chen, Brandon Amos, and Maximilian Nickel. Learning neural event functions for ordi- nary differential equations. arXiv preprint arXiv:2011.03902 2020. A1
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The Inter- national Journal of Robotics Research, 2023. A1
Mengyu Chu, Lingjie Liu, Quan Zheng, Erik Franz, Hans-Peter Seidel, Christian Theobalt, and Rhaleb Zayer. Physics informed neural fields for smoke reconstruction with sparse data. ACM Transactions Graphics (ToG), 4144):1-14 2022. Al
Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, and Shirley Ho. Lagrangian neural networks. In ICLR Workshop on Integration of Deep Neural Models and Dif- ferential Equations, 2020. A1
Bo Dai, Linge Wang. Baoxiong Jia, Zeyu Zhang, Kong-Chun Zhu, Chi Zhang, and Yixin Zhu. X-voe: Measuring explanatory violation of expectation in physical events. In Proceedings of International Conference Computer Vision (ICCV), 2023. 2
Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. In Proceed- ings of Advances Neural Information Processing Systems (NeurIPS), 2019. A1
Jiaqi Han, Wenbing Huang, Hengbo Ma, Jiachen Li, Joshua B. Tenenbaum, and Chuang Gan. Learn- ing physical dynamics with subequivariant graph neural networks. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), 2022.
Susan J Hespos and Renée Baillargeon. Infants' knowledge about occlusion and containment events: A surprising discrepancy. Psychological Science, 12(2):141-147, 2001. 2
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi. and David J Fleet. Video diffusion models. In Proceedings of Advances Neural Information Processing Systems (NeurIPS), 2022. A1
11

---

Published as a conference paper at ICLR 2026
Ayush Jain, Andrew Szot, and Joseph Lim. Generalization to actions in reinforcement learning. In Proceedings of International Conference on Machine Learning (ICML), 2020. A1
Michael Janner, Yilun Du, Joshua B Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.0999) 2022. A1
Hanxiao Jiang, Hao- Yu Hsu, Kaifeng Zhang, Hsin-Ni Yu, Shenlong Wang, and Yunzhu Li. Physt- win: Physics-informedt informed reconstruction and simulation of deformable objects from videos. ICCV, 2025. 3
Huaizu Jiang. Xiaojian Ma. Weili Nie, Zhiding Yu, Yuke Zhu, and Anima Anandkumar. Bongard- hoi: Benchmarking few-shot visual reasoning for human-object interactions. In Proceedings of Conference on Computer Vision and Puttern Recognition (CVPR), 2022.
Jindong Jiang, Fei Deng, Gautam Singh, Minseung Lee, and Sungjin Ahn. Slot state space models. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), volume 37, pp- 11602-11633 2024. 3
Bingyi Kang, Yang Yue, Rui Lu, Zhijie Lin, Yang Zhao, Kaixin Wang, Gao Huang, and Jiashi Feng. How far is video generation from world model: A physical law perspective. arXiv preprint arXiv:2411.02388 2025. 3, Al
Youngsung Kim, Jinwoo Shin, Eunho Yang, and Sung Ju Hwang. Few-shot visual reasoning with meta-analogical contrastive learning. In Proceedings of Advances in Neural Information Process- ing Systems (NeurIPS), 2020.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. Proceedings of International Conference on Learning Representations (ICLR), 2017. A3
Miltiadis Miltos Kofinas, Erik Bekkers, Naveen Nagaraja, and Efstratios Gavves. Latent field dis- covery in interacting dynamical systems with neural fields. In Proceedings of Advances Neural Information Processing Systems (NeurIPS), 2023. A1
James Kubricht, Keith Holyoak, and Hongjing Lu. Intuitive physics: Current research and controversies. Trends in Cognitive Sciences, 21(10):749-755 2017. 1
Brenden Lake and Marco Baroni. Human-like systematic generalization through a meta-learning neural network. Nature, (557985))115-12)) 2023.
Brenden M Lake, Tomer D Ullman, Joshua Tenenbaum, and Samuel Gershman. Building machines that learn and think like people. Behavioral and brain sciences, 40:e253, 2017.
Shiqian Li, Kewen Wu, Chi Zhang, and Yixin Zhu. On the learning mechanisms in physical rea- soning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), 2022.
Shiqian Li, Kewen Wu, Chi Zhang, and Yixin Zhu. I-phyre: Interactive physical reasoning. In
Proceedings of International Conference on Learning Representations (ICLR), 2023. 2, 3, 5, 8, Al
Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physics- grounded image-to-vin generation. In Proceedings of European Conference Computer Vi- sion (ECCV), 2024a. Al
Yang Liu, Jiashun Cheng, Haihong Zhao, Tingyang Xu, Peilin Zhao, Fugee Tsung, Jia Li, and Yu Rong. Improving generalization in equivariant graph neural networks with physical inductive biases. In Proceedings of International Conference on Learning Representations (ICLR), 2024b.
2, 3.7
Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot atten- tion, 2020.
12

---

Published as a conference paper at ICLR 2026
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence. 3(3):218- 222222, 2021. 3
Xiao Luo, Jingyang Yuan, Zijie Huang. Huiyu Jiang, Yifang Qin, Wei Ju, Ming Zhang, and Yizhou Sun. HOPE: High-order graph ODE for modeling interacting dynamics. In Proceedings of the 40th International Conference on Machine Learning, 2023. A3
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning, Nature, 5 518(7540):529-533 2015, Al
Saman Motamed, Laura Culp. Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do generative video models understand physical principles? In Proceedings of International Conference on Computer Vision (ICCV), 2025. Al
Isaac Newton. Philosophiae naturalis principia mathematica, volume 1. G. Brookman, 1833. 2, 6
Alexander Norcliffe, Cristian Bodnar, Ben Day, Nikola Simidjievski, and Pictro Liò. On second order behaviour in augmented neural odes. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), 2020. A1
Carlota Parés-Morlans, Michelle Yi, Claire Chen, Sarah A Wu, Rika Antonova, Tobias Gerstenberg, and Jeannette Bohg. Causal-pik: Causality-based physical reasoning with a physics-informed kernel. arXiv preprint arXiv:2505.22880 2025. Al
Michael Poli, Stefano Massaroli, Junyoung Park, Atsushi Yamashita, Hajime Asama, and Jinkyoo Park. Graph neural ordinary differential equations. arXiv preprint arXiv: 1rXiv:1911.07532, 2019. 3, A3
Haozhi Qi, Xiaolong Wang, Deepak Pathak, Yi Ma, and Jitendra Malik. Learning long-term visual dynamics with region proposal interaction networks. In Proceedings of International Conference on Learning Representations (ICLR), 2021. 1, 2, 3,7,9
Hanno Rein and S-F Liu. Rebound: an open-source multi-purpose n- n-body code for collisional
dynamics. Astronomy & Astrophysics, 537:A128, 2012. 6
Yulia Rubanova, Tatiana Lopez-Guevara, Kelsey R. Allen, William F. Whitney, Kimberly Stachen- feld, and Tobias Pfaff. Learning rigid-body simulators over implicit shapes for large-scale scenes and vision. In Advances in Neural Information Processing Systems, 2024. Oral. 3
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling, E(n) Equivariant Graph Neural Net- works. Pruceedings of International Conference on Machine Learning (ICML), 2021. 2, 3,
A3
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Mare Lanctot, et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484488) 2016. Al
Elizabeth Spelke. What babies know: Core Knowledge and Composition volume 1, volume 1.
Oxford University Press, 2022.
Elizabeth S Spelke and Katherine D Kinzler. Core knowledge. Developmental Science, 10(1):89-99)
2007. 1
Elizabeth S Spelke, Karen Breinlinger, Janet Macomber, and Kristen Jacobson. Origins of knowl- edge. Psychological Review, 99(4)660 1992. 2
Ziyi Wu, Nikita Dvorikk Klaus Greff, Thomas Kipf, and Animesh Garg. Slotformer: Unsuper- vised visual dynamics simulation with object-centric models. In Proceedings of International Conference Learning Representations (ICLR), 2023. 1, 2, 7,9
Kai Xu, Akash Srivastava, Dan Gutfreund, Felix Sosa, Tomer Ullman, Josh Tenenbaum, and Charles Sutton. A aayesian-symbolle approach to reasoning and learning in intuitive physics. In Proceed- ings of Advances in Neural Information Processing Systems (NeurIPS), 2021. 1, A1
13

---

Published as a conference paper at ICLR 2026
Jingyang Yuan, Gongbo Sun, Zhiping Xiao, Hang Zhou, Xiao Luo, Junyu Luo, Yusheng Zhao, Wei Ju, and Ming Zhang. Egode: An event-attended graph ode framework for modeling rigid dynamics. In Proceedings Advances in Neural Information Processing Systems (NeurIPS),
2024. 3, A1
Chi Zhang. Baoxiong Jia, Yixin Zhu, and Song-Chun Zhu. Human-level few-shot concept induction through minimax entropy learning. Science Advances, 10(16)readg2488, 2024.
Kaifeng Zhang, Baoyu Li, Kris Hauser, and Yunzhu Li. Particle-grid neural dynamics for learning deformable object models from rgb-d videos. In Proceedings of Robotics: Science and Systems (RSS), 2025.
Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Symplectic ode-net: Learning hamiltonian dynamics with control. In Proceedings of International Conference Learning Representations (ICLR), 2020. A1
Yixin Zhu, Tao Gao, Lifeng Fan, Siyuan Huang, Mark Edmonds, Hangxin Liu, Feng Gao, Chi Zhang, Siyuan Qi, Ying Nian Wu, al. Dark, beyond deep: paradigm shift to cognitive ai with humanlike common sense. Engineering, 6(3):310-33) 345, 2020. I
14

---

Published as a conference paper at ICLR 2026
A
MORE RELATED WORKS
Physics-informed neural dynamics Physics-informed methods address the representation learn- ing issues by incorporating physical inductive biases via explicit modeling of state derivatives within dynamical systems (Chen et al., 2018; Zhong et al., 2020; Norcliffe et al., 2020). While they exhibit good adherence physical laws, current systems typically assume conservtvvee systems with simplified dynamics (Greydanus et al., 2019; Cranmer et al.. 2020) or static fields (Kofinas et al,, 2023). More importantly, these methods face significant challenges in few-shot learning and strug- gle with cross-scenario generalization (Chen et al., 2020; Yuan et al., 2024), often requiring specific priors and grammars achieve meaningful transfer (Xu et al., 2021). Although Physics-informed Neural Networks (PINN) can be precise in modeling physical dynamics, they depend heavily on prior knowledge of the underlying equations (Chu et al., 2022).
Mental simulation for planning Mental simulation is a core mechanism for both human and machine planning. Classic reinforcement learning (RL) agents learn policies through trial-and-eree (Mnih et al., 2015; Silver et al., 2016), while model-based RI. incorporates explicit rollouts for decision making (Jain et al., 2020). Human studies, such as the Virtual Tool Game, show that people exploit internal physics models to predict imagined outcomes and guide interventions (Allen et al., 2020), inspiring computational work that embeds neural simulators into planning et al,, 2025). More recently, generative approaches like diffusion- based planners leverage stochastic simulation to generate diverse, adaptable trajectories in high-din high-dimensional spaces (Janner et al., 2022; Chi et al., 2023), though at the cost of large data requirements. Yet, much prior work still assumes known dynamics or heavy data availability, leaving the few-shot nature of physical reasoning an open challenge.
Physics-aware video generation Recently, video generation models have made significant progress in producing visually realistic scenes, highlighting their potential to serve as "world simu- lators" by predicting the next frame (Ho et al., 2022; Blattmann et al., 2023). However, research has shown that these models, despite trained on millions of videos, still lack physical commonsense and violate Newtonian laws (Kang et al., 2025; Motamed et al., 2025). To tackle this problem. (Liu et al., 2024a) introduced physies simulator for producing high-quality ever, these methods mainly focus on video synthesis, falling short of physical reasoning in few-shot scenarios.
conssss-cnnisttnt videos. How-
B ENVIRONMENT AND DATASET CONFIGURATIONS
In this section, describe the environments and datasets used in our experiments, covering three distinct domains: I-PHYRE, N-body simulations, and the PHYRE physical reasoning benchmark. Each environment presents unique challenges and dynamics, requiring tailored data generation and simulation procedures. We outline the structure of the training, within--eenario, and cross-scenario datasets, along with the physical parameters and recorded features essential for model training and evaluation. These configurations ensure both diversity and rigor in testing the generalization capa- bilities of our models across varying physical setups.
B. I-PHYRE
We adopt the original settings from the I-PHYRE work (Li al., 2023). The training dataset con- sists of 10 basic games: support, hinder, direction. hole, fill, seesaw, angle, impulse, pendulum, and spring. For each game, we randomly generate successful and failed action sequences. The within-scenario setting includes an additional 10 successful and 10 failed action sequences for each of the 10 basic games. The cross-scenario setting contains 10 successful and 10 failed action se- quences from 30 unseen games, which include 10 noisy games, 10 compositional games, and 10 bulti-ball games. All object masses are set to be equal, and small friction and elasticity coefficients applied. For each game, record the center positions, lengths, angles, radii, horizontal veloc- ities, vertical velocities, and rotational velocities of each object, as well as the spring pair indices. Each sequence contains up to 12 objects and spans 150 timesteps.
A1

---

Published as a conference paper at ICLR 2026
B.2 N-BODY
We employed the REBOUND engine to simulate N-body dynamies and considered two types of N-body problems for 3D trajectories: the planetary n-body problem and the cometary n-body prob- lem. In the planetary n-body problem, the comets are initialized with Keplerian velocities, while in the cometary n-body problem, the planets are initialized with escape velocities. Initial positions are sampled from the spherical coordinate system and then converted to Cartesian coordinates. For the training and within-scenario datasets, the radii are sampled within the range of to 3, azimuthal angles are sampled from 0 to 2n, and polar angles are sampled from - to n/6. Masses for the orbiting bodies are sampled from 0.05 0.1, while the central massive body is assigned mass sampled from 3 to 5 or from 7 to 9. All sampling is performed using Latin Hypereube Sampling to ensure comprehensive space coverage, even with sparse samples. The training dataset includes 50 two-body planetary trajectories (1 orbiting body), 50 three-body planetary trajectories (2 orbiting bodies), 50 two-body cometary trajectories (1 orbiting body), and 50 three-body cometary trajecto- ries (2 orbiting bodies), spanning across 50 timesteps. The within-scenario dataset consists of the same four trajectory types as the training set but with different initial conditions, with trajectories spanning 150 timesteps. The cross-scenario dataset contains 50 eight-body planetary trajectories, 50 ten- ten-body planetary trajectories, 50 eight-body cometary trajectories, and 50 ten-body cometary trajectories. For these cross-scenario trajectories, initial radii are sampled from I to 5, masses of the orbiting bodics are sampled from 0.05 to 0.15, and the central body mass is sampled from to 9. The trajectory length in the cross-scenario data is 150 steps. We record the masses, 3D Cartesian coordinates, and velocities for training.
B.3 PHYRE
We conduct experiments the PHYRE-B tier, using the standard split from fold I-one of the most challenging folds indicated by its low baseline performance. The goal is to make the green ball touch the purple blue objects by placing red ball. This physical reasoming benchmark presents high level of difficulty for several key reasons. First, it features a diverse range of objects- such as balls, bars, sticks, and jars fars-with varying shapes and uneven mass distributions, adding complexity to interaction dynamics. Second, the solution space is extremely narrow relative to the vast array of possible actions, rendering random attempts largely ineffective. Third, achieving the goal requires navigating a sequence of intricate physical events, including toppling, rotation, collisions, and fric- tional forces. We train the model using 20 within-template scenarios, each containing 20 tasks, with 30 actions allocated per task. The model is evaluated on the remaining 5 templates, which were not seen during training. For each sample. we simulate dynamic frames 256 x 256 resolution using a stride of 60 and extract object masks for supervision. To ensure stable training, we also record initial linear and angular velocities for each training chunk. To visualize the force field, we place red balls on 64 x 64 grid, run simulations to generate trajectories, and compute second derivatives field vectors using finite difference methods.
C TRAINING DETAILS
C.I HYPERPARAMETERS
In I-PHYRE, we train the models using a single NVIDIA A100 Tensor Core 80GB GPU. The force field predictor in NFF is based on a DeepONet architecture, consisting of trunk and branch networks, each a 3-layer MLP with hidden size of 256. The ODE solver uses the Fuler method with step size of 0.005. The 150-step trajectories are divided into 6-step segments, with 6 trajectories per batch. The learning rate starts at 5e-4 and gradually decays to le-5 following cosine annealing schedule. Training occurs over 3000 epochs with a weight decay of le-5 for regularization.
The interaction predictor in IN is a 3-layer MLP with hidden size of 256, while the decoder is also an MLP of the same size. SlotFormer uses transformer encoder layers, each with attention heads and feedforward network of size 256. Both IN and SlotFormer trained with a batch size of 50 trajectories, employing the same segmentation technique. All other hyperparameters are kept consistent with those used in NFF.
A2

---

Published as a conference paper at ICLR 2026
In N-body, we train the models using a single NVIDIA GeForce RTX 3090 GPU. For all models, the 50-step trajectories are divided into 5-step segments, with 50 trajectories per batch. The learning rate starts at 5e-4 and gradually decays to lc-7 following cosine annealing schedule. Training occurs over 5000 epochs with a weight decay of le-5 for regularization. All the other hyperparameters are the same as those used in I-PHYRE.
For training the vision-based NFF in PHYRE, we perform feature extraction from 256 x 256 object masks using four sparse convolutional layers with channel sizes of 16, 32, 32, and 32. The model architecture includes a DeepONet, composed of a trunk network and a branch network, each with layers and 128 hidden units. DeepONet takes as input combination of mask, positional, and velocity features, and ouiputs the corresponding 2D forces and torques acting on each object. Then, we perform a transform and rotation on the object masks to obtain the frames the next states. The ODE solver uses the Euler method with step size of 0.025. We adopt a cosine annealing schedule to adjust the learning rate from 3e-4 to 5e-5, with a batch size of 8. Training is conducted for 500 epochs to ensure convergence, with the 18 18-step trajectories divided into 2-step segments during training.
C.2 EVALUATION METRICS
In this study, the chosen evaluation metrics include, RMSE, FPE, PCE, and R. Each of these metrics provides valuable insights into different characteristics of the model's predictions, allowing for a comprehensive evaluation.
Root Mean Squared Error (RMSE) The RMSE, the square root of MSE, shares the same char- acteristics MSE in terms of penalizing larger errors. However, it provides the error in the same units as the original data, allowing for a more intuitive understanding of how far off the model's predictions are, average, in the context of physical trajectories:
1
RMSE
Σ n
(2 21)
(A1)
Final Position Error (FPE) The FPE specifically measures the discrepancy between the predicted and actual final position of the object at the end of the trajectory. This metric is crucial for goal- driven physical reasoning tasks where the objects are expected to finally move into a specific area. FPE helps ensure that the model is not only capturing the intermediate trajectory but also predicting the final destination with high accuracy:
FPE-
&&nmal
2final
(A2)
Position Change Error (PCE) The PCE quantifies the error in the predicted change of position over time, which can be interpreted as measure the model's accuracy tracking the object's velocity during its motion:
PCE = | Azl.
(A3)
Pearson Correlation Coefficient (R) The R measures the linear relationship between the pre- dicted and actual trajectories. R is useful for assessing how well the model captures the overall trend or pattern of the trajectory, even when the absolute errors might vary. A high correlation suggests that the model is effectively tracking the overall movement, while a low correlation might indicate that the model fails to capture the underlying trajectory pattern:
En - {) - Z)
R
(A4)
1 - 3)22 (Zc 2)2
D MORE BASELINES ON N-BODY DATASETS
We further compare NFF with other advanced GNN-based methods including Graph Convolution Networks (GCN) (Kipf & Welling, 2017), EGNN (Satorras al., 2021) and GraphODE (Poli al., 2019; Luo et al., 2023) on N-body datasets. Since most of these methods assume strong priors (c.g., central forces), they cannot be applied abstract reasoning tasks such as PHYRE or I-PHYRE.
A3

---

Published as a conference paper at ICLR 2026
Table A1: More baselines N-body datasets
Model
Within [O,T]
Within [0,3T]
Cross
NFF
0.079 = 0.000
0.525 = 0.026
1.226
=0023
GCN
0.256 a 0.021
0.890 = 0.034
4.343
+0.159
EGNN
0.096 + 0.009
0.718
0.034
8.394 +0,278
GraphODE
0.252 0.020
0.974 0.039
3.920 0.005
Table A2: Results of different kinds of integration methods. The computational complexity of the model is proportional to the number of average integration steps (relative to Euler 5e 3). Among non-adaptive methods, with the increase of computational complexity, the Heun3 method achieves the best performance. For uniformity across tasks and stability during training, we simply selected the Euler method for integration.
Method
Average steps (relative)
Within
Cross
Adaptive methods (tolerance)
Adaptive (le-4)
0.28
0.095 + 0.008
1.712 + 0.088
Adaptive (1e-5)
0.36
0.092 0.007
1.788 1 0.375
Adaptive (1e-6)
0.56
0.097 0.008
1.609 1 0.104
Adaptive (1e-7)
1.07
0.095 + 0.007
1.556 + 0.050
Non-adaptive methods (step size)
Euler (5e-3)
1.00
0.120 + 0.012
1.251 + 0.025
Midpoint (5e-3)
2.00
0.097 + 0.009
1,736 + 0.091
Heun3 (5e-3)
3.00
0.096 0.008
1.489 + 0.050
RK4 (5e-3)
4.00
0.094 0.008
1.292 1 0.038
E ABLATION STUDIES ON INTEGRATION METHODS
In Table A2, we present additional ablation studies comparing various integration methods. Specif- ically, we evaluate the performance of different integration orders on the N-body task, including Euler, Midpoint, Heun3, RK4, and adaptive methods. Computational complexity is measured by the average number of integration steps. The results indicate that the Euler method excels cross- scenario generalization. Despite this, Euler integration has lower computational complexity than the higher-order methods, while maintaining better training stability compared to adaptive methods. These findings suggest that Fuler integration is sufficient for our physical reasoning tasks.
F PLANNING TIME ON N-BODY
We demonstrate how backward integration accelerates planning in N-body tasks. As shown in Ta- ble A3, backward planning is up to 10 times faster than traditional optimization-base planning methods.
Table A3: NEF enables fast planning through backward integration. NFF performs planning approximately 10 times faster than optimization baseed based methods in the N-body task. The value represents the time in seconds required to plan 200 scenarios in parallel, with each experiment conducted three times.
Model IN SlotFormer NFF
Type
Within
Cross
Optimization
13.886 + 0.225
11.693 + 0.153
Optimization
22.098 + 0.156
22.489 + 0.068
Backward integration
1.717 + 0.023
1.716 = 0.013
G DETAILED PLANNING RESULTS
We present the detailed planning results of different models across trials in Table A4.
Besides those dynamic prediction models, we also benchmark the planning performance of the large language model GPT-4o with the refinement mechanism. The core idea is utilizing In-Context Learning (ICL) abilities and high-level reasoning abilities of large language models to find correct solutions.
The prompt in Round consists of parts: (1) I-PHYRE description and elaborately explained example. (2) Other training examples. (3) Description of the testing game setting. Part 2, we provide GPT-4o with successful solution and wrong solution together with resulting trajectories
A4

---

Published as a conference paper at ICLR 2026
of objects for each of the 10 basic games. In the refinement stage (i.e., Round and Round 3), run the action sequence proposed by GPT-4o on the simulator, Then we offer GPT-4o the resulting trajectory and ask it to modify the previous solution or design one.
We test the performance of GPT-4o for both within-sccnario planning and cross-scenario planning. Detailed results are presented in Table A5. Within-scenario games contains the same 10 games used in the prompt in Round 1. So the language model can memorize corresponding solutions provided in the prompt achieve good performance. Cross-scenario games contains the other 30 game settings, on which the performance of GPT-4o is better than other baselines (Random, IN, SlotFormer) in Table A4. The improvement of performance through refinement is witnessed in both within- sccenrio and cross-scenario games.
The prompt used in Round is shown below.
A5

---

Published as a conference paper at ICLR 2026
Table A4: Planning results in I-PHYRE. Average probability of succeeding cross-scenario games after trails from IN, SlotFormer, NFF. human, refining. The gray line indicates the best results among all the Al methods.
Method
Refine
Round
Round 2
Round 3
Round 4
Round 5
Random
0.24 + 0.04
0.37 + 0.05
0.45 + 0.06
0.51 + 0.06
0.56 + 0.06
IN
0.25 0.06
0.33 + 0.07
0.39 + 0.07
0.42 1.22+0 0.08
0.45 + 0.08
IN
0.25 0.06
0.39 0.07
0.48 0.07
0.57 + 0.07
0.63 i 0.06
SlotFormer
0.30 0.07
0.36 0.008
0.39
0.41
0.43 1 0.08
SlotFormer
0.30 + 0.07
0.42
0.49 0.07
0.55 0.07
0.60 1 0.07
NFF
0.51 0.08
0.58 + 0.08
0.60 0.08
0.62 + 0.08
NFF
0.51 0.08
0.62 0.08
0.69 0.07
0.74 0.06
0.77 + 0.06
Human
0.52 + 0.06
0.66 + 0.06
0.73 + 0.06
0.77 + 0.06
0.80 + 0.05
Human
0.52 0.06
0.70 0.06
0.79 0.05
0.83 0.04
0.86 I 0.04
Table A5: GPT-4o's planning results in I-PHYRE. The GPT-4o refines itself after each round of play and shows increasingly better performance.
Scenario
Round
Round
Round 3
Within
0.74 + 0.08
0.78 + 0.07
0.82 + 0.07
Cross
0.35 + 0.07
0.45 + 0.08
0.51 i 0.04
/s PART This is a 20 physics simulation few red balls, and may appear black blocks, grey blocks and blue blocks. Lengths of blocks
There will be one or
different.
All these objects are rigid bodies. when we talk about physical laws, we censider gravity (there is gravity in this environment), frictien (but the friction coefficients are small), collision between rigid bodies, rotation of balls and blocks.
However, grey blocks and black blocks are fixed and stay still to their initial positions. They do not apply any laws of physics (even collision won't affect them) The red ball and blue blocks are controlled by physical Laws.
Initially, all objects are at rest (their velccity and angular velocity are all 0).
There may exist 2 types of constraints between objects: spring joint. "Spring" constraint objects. Springs in all the games have the same rest length, stiffness and damping rate (You can infer these properties from trajectories provided below that contain springs). "Joint" constraint means there is rod supporting 2 objects their distance is fixed
there is spring linking
We emphasize that grey and black blocks are not affected by any constraints. They are fixed to their initial positions.
going play game. The goal is to make all the red balls fall down to the bottem of the canvas.
Now,
Let's first define a coordinate on the canvas. Let the origin be the top left corner. The x-axis is horizontal, from left to right. The y-axis is vertical, from top to bottom. The canvas size is (600, 603), so the coordinate
the lower-right corner The only operation the player can do is to eliminate any grey blocks at any time they want (however, we limit the whole game in 15 seconds. If after 15s the zed ball still doesn't reach the bottom, the player loses the game). The player cannot eliminate other objects including the red ball, blue or black blocks. Note that the player eliminates a grey block which is connected to some other object with spring or joint, then the epring/Joint between them will disappear.
(600, 600).
A6

---

Published as a conference paper at ICLR 2026
Now, I will show you game named "fill". The is actually key solve this problem.
this gane, a blue block placed above a grey block. You need to eliminate the grey block so that the blue block falls down to fill the pit on the "ground" (the ground and pit are built by a few black blocks). Otherwise, the red ball will roll into and get trapped in the pit.
Below the initial setting the game. will describe the meaning of each parameter.
l'block': 111100.0, 420.01, 1400.0, 420.0]J, 11100.0, 400.0), 1250.0, 400.011, [[350.0, 400.01, [400.0, 400.0]l, [[270.0, 200.0], [330.0, 200.011, [[270.0, 180.01, [330.0, 180.0]1, [C100.0, 150.0], [180.0, 200.011, [[150.0, 100.01, [150.0, 150.0]]l, 'ball'; [[120.0, 120.0, 20.0]1, feli': [0, 0, 0, 1, 0, 1, 1, 0], 'dynamic': [0, 0, 0, 0, 1, 111
First comes the 'boock' items. Each item is in the form of 'I[ left_x, left_yl, [right_x, right_yll's namcly, the coordinates of the left and right corner of the block. Note that each block is generated by moving circle (radius 1C) centered from the left corner to the right corner.
Next comes the 'ball' items. Now we only have 1 ball in this game This iter. looks like 1x, y, rl', namely, the coordinates of the center and the radius of the ball.
Then cones the 'eli' item specifying which of these objects are eliminable. In this game, 'eli': [0, D, C, 1, 0, 1, 1, 0]. This is an 8-dimensional vector. The first 7 elements describes the blocks there are 7 elements in the 'block' item), and the last element describes the ball. means the object is not eliminable, and the object eliminable.
Finally comes the 'dynamic' item, which specifies whether the objects are dynamic. In this game, 'dynamic'; [0, 0, 0, 0, 1, 0, 0, 1]. The first 7 elements are the 7 blocks, and the last element is the ball. 0 means the object is static (not apply physical laws), and means the object is dynamic (epply physical laws)
You can infer from 'eli' and 'dynamic' that the 4th, 6th, and 7th blocks are grey, the 5th block is blue. A11 other blocks are black. other game settings, there may exist 'spring' or 'joint items They lock like 'spring': [16, 711, which means there is only 1 spring connecting the 6th and 7th object (object can be blocks or balls). 'joint': [16, 71] means there is only 1 joint connecting the 6th and 7th object.
Here, we also provide an image of the game.
<image here>
Now you are provided with successful action sequence that can solve the game. The action sequence is as follows:
11150.0, 125.0, 0.31, 1300.0, 200.0, 1. 1.00000000000000011
Action sequence is list eliminations [pos_x, pos_y, tl, where 'pos_x' and 'pos_y' must be the center of some block (formally, the center is the average of the coordinates of 2 corners: pos_x = 2 (left_x + right_x/ x), pos_y 1 (left_y right_y)) and is the time when the block is eliminated should be in 0.1 15.0). In this exampie, there are 2 grey blocks to eliminate. [150,
125, 0.3] means climinate the vertical block (next to the red ball) at t = 3s; [300.0, 200.0, 1.4] means climinate the horizontal block (which supports the blue block from falling down) at 1.4s. In this setting, all positions that [[300.0, 200.0], [140.0, 175.0], [150.C, 125.011.
eliminated are
Now you are provided with the trafectory of the red ball and the dynamic (blue) block. Other blocks either static (the same in
A7

---

Published as a conference paper at ICLR 2026
the initial setting) eliminated only one blue block. there are multiple blue blocks, their position at some timestamp will be given by the same order as in 'dyn item. If there are no blue balls, the 'dynamic' item will be an empty list.
time. In this case, there
Below the trajectory.
t 0.000s: ('ball': [[120.0, 120.0]], 'dynamic_block'; [[270.0, 180.0, 330.0, 180.011:
t = 0.167s: l'pall'; 11120.0, 121.25]], 'dynamic_block';
11270.0, 180.03, 330.0, 180.03111
0.333s: ("ball': [[120.0, 125.2811, 'dynamic_block',
11270.0, 180.03, 330.0, 180.0311)
/+ PART 2 */
Now we provide you with more examples from different game settings. We will provide 10 gane sottings. For each gane setting, we will provide you with the initial setting (data + image), and 2 action sequence trajectory (ineluding 1 successful and failed ones).
Game 1: name:angle
Initial setting: ('block'; 111100.0, 400.0), 1400.0, 400.0]1,
11230.0, 350.0), 1210.0, 350.011, 11200.0, 300.01, 1210.0, 300.011, 11480.0, 400.0), 1550.0, 400.011, 11390.0, 350.01, 1400.0, 350.011, [1390.0, 300.0], [400.0, 300.0]1, [[100.C, 380.0], [100.0, 360.011, [[550.0, 380.0], (550.0, 360.011, [[200.0, 280.0), [400.0, 280.0111, 'ball'; "ball': [[300.0, 250.0, 20.0,, 01, "dynamic': 10, 0, 0, 0, 0, 0,
[0, 1, 1, 1, 1, 0, 0, 0, 1, 11:
Eliminable posttionn: ([205.0, 350.0], [205.0, 300.0),
[395.0, 350.0], [395.0, 300.0]]
Game setting image:
<image here>
Successful Case 1:
Action sequence: [[395.0, 300.0, 1.4000000000000041 [205.0, 350.0, 900000000000000011 [205.0, 300.0, 3.9000000000000121, (395.0, 350.0, 5.9000000000000002111 This action sequence leads to success. Trajectory:
t 0.000s: ('ball': [1300.0, 250.011 dynamic_block' 1,200.0, 280.0, 0, 400.0, 280.011}
t 0.167s: ('ball's [[300.0, 250.32]], "dynamic_block [[200.02 02, 280.3, 400.02, 280.311)
Failed Case 1:
Actions: 1 11395.0, 350.0, 0, 3. $.0000000000000001, 1205.0, 300.0, 4. 8.8000000000000021, [205.0, 350.0, C, 6.400,,660000000 6.4000000000000022], [395.0, 300.0, 6.700000000000000211
This action sequence leads to failure. Trajectory:
0.000s: ('ball': [[300.0, 250.01], Synamic_blocke [1200.0, 280.0, 400.0, 280.01];
t 0.167s: ('ball'; [[300.0, 250.32 32]], 'dynamic_bie bbock'' [[200.02, 280.3, 400.02, 280.3]1)
<more examples>
/+ PART 3 +/
A8

---

Published as a conference paper at ICLR 2026
Your goal is to provide successful action sequence (that makes ball fall to the bottom the canvas) under new game setting. Below is the new game setting, Game nane: "angle".
Game setting:
Eliminable positions: Game setting image:
<image here
Your task is to provide a successful action sequence in the same format as examples given previously. You should make some analysis or explanations.
Now please provide successful action sequence. the end of your respense, please give the entire action sequence the format of
Action sequnce: [[pes_1_x pos 1,,, t_1l, 2_x,___x, pos __y_2_y,
t_2], ...]
80 that we can extr and test your proposed action sequence easily.
The prompt used GPT-4o.
A9

---

Published as a conference paper at ICLR 2026
Fx60
-03-
-0.5
0.2
-10
Foro8
-14
-15
-022
-0.4
-2.
8.2 1.00 0.00
0.75 angle
1.00
vy
Spring
(a) Support interaction
Platform orientation
(c) Impact dynamics
(d) Spring mechanies
Figure A1: Force response analysis under controlled state variations. Each plot shows the predicted components (Fx in blue, Fy in orange) from our trained NEF model, measured in normalized units against different state parameters. The subplots systematically analyze: (a) support forces showing characteristic contact respouse when the ball meets the platform, (b) force decomposition the plat- form rotates from horizontal (0") ) vertical (90"). (c) impact forces scaling with the ball's downward velocity, and (d) spring forces varying with horizontal displacement from the equilibrium position. Each plot varies one parameter while keeping all other scene variables constant, demonstrating NFF's learned physical principles, including contact mechanies, geometric reasoning, impact dynamics, and harmonic motion.
function of ball height,
Figure A2: The inverted forces from NFF and its predicted trajectorics of dynamic objects in I-PHYRE within scenarios. Ground-truth trajectories are represented by black dots, while predicted trajectories are shown with colorful dots. Horizontal and vertical forces are depicted as arrows, while rotational forces are indicated by red or blue dots, signifying negative and positive forces, respectively. The color of the forces indicates which object exerting the force. Only the initial states shown for static objects, Some objects will be eliminated during the process. Best in videos.
H MORE VISUALIZATION
We provide additional visualizations of the learned forces along trajectories in I-PHYRE, as shown in Figure AI, Figure A2, and Figure A3. Further prediction results on the N-body dataset are presented in Figure A4. Additionally, examples of failure cases in NFF predictions are shown in Figure A5. We also visualize the failure encoding slot-attention SlotFormer Figure A6.
A10

---

Published as a conference paper at ICLR 2026
The inverted forces from NFF and its predicted trajectories of dynamic objects in I-PHYRE cross scenarios. Ground-truth trajectories represented hy black dots, while predicted trajectories are shown with colorful dots. Horizontal and vertical forces depicted arrows, while rotational forces are indicated by red blue dots, signifying negative and positive forces, respectively. The color of the forces indicates which object is exerting during the process. Best seen in vidcos.
Figure A3:
the force. Only the initial states are shown for static objects. Some objects will be eliminated
A11

---

Published as a conference paper at ICLR 2026
(a) Ground truth Figure A4: Additional prediction results on N-body. The figure displays predictions from simple N-body
(b) Our NFF
(c) SlotFormer
(d) IN
dynamics to complex dynamics with more bodies and extended prediction horizons.
A12

---

Published as a conference paper at ICLR 2026
True
NFF
(c)
Figure A5: Examples of prediction failures in PHYRE cross-ccenario setting. (a) Minor prediction inac- curacies accumulate, causing the green ball fall the cup unexpectedly. (b) Extremely tiny red balls introduce unforeseen interactions, potentially resulting in significant dynamic changes, such as cup rotation. (c) Complex interactions between the ball and the seesaw can hinder accurate force prediction.
True
Slot att-ttttntinn
Figure A6: Examples of failures in Slotformer's pretrained slot-attention module in cross-scenario setting. The pretrained slot-attention encoder in slotformer, responsible for encoding visual features, exhibits poor generalization to cross- scenario scenes, leading to an inability capture the precise geometric information crucial for physical reasoning.
A13


Reviews:
[{"review_id": "TAcUyf0PQy", "reviewer": "Reviewer_fnPM", "summary": "The paper aims to better model and learn the physical interactions, and proposes to use a neural operator to learn the force field from external interventions and object interactions. \n\nThe force field is parameterized by a graph neural network. An ODE integrator (such as the Euler integrator) is applied to generate object trajectories from the force field represented by the neural network. To train the model, the network is optimized by minimizing the Mean Squared Error (MSE) loss between predictions and ground truth.\n\nThe method is evaluated by several benchmarks: I-PHYRE, a suite of complex physical reasoning puzzles;  N-body problem for trajectory prediction; and PHYRE, a vision-based benchmark of physical reasoning puzzles. Several methods are used to compare against the proposed method: vanilla IN, SEGNO, and SlotFormer. From the experiment results, the proposed method generates trajectories that closely match ground truth behavior, and shows good generalization to new scenario. The trained NFF model also shows the ability to generate plans for new tasks after learning from limited examples.", "questions": "* The paper uses graph based model as the backbone network to model  the force field, would it suffer the issue that hard to model long-distance interactions among multiple objects? \n\n\n* In traditional physical simulations, force-based method usually needs small time steps to make it stable. Does the same apply to this paper’s force-based method? (Does the model needs a smaller dt to maintain stability, or just need the similar dt as the other compared methods?)\n\n\n* For the training trajectory that contains multiple time step, is the MSE loss is computed on each time step's predication or only on the final state of the trajectory?\n\n\n* For the rigid body collision problems tested in the paper, it might be more convincing to also demonstrate such cases in 3D scenes (if an appropriate dataset is available). Most of the examples (excepts for the N-body problem) presented in the paper are 2D cases.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* It appears novel (to me) to learn the force field using a neural operator instead of directly learning physical interactions or trajectories.\n\n* Besides comparing experiments with the ground-truth trajectories, the paper also demonstrates the learned force field, as well as testing it with planning tasks.\n\n* Additionally, the paper provides a clear video to demonstrate the experiment results.", "weaknesses": "I just have some additional questions listed in the section below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4tE6jzmEj9", "reviewer": "Reviewer_mX4Q", "summary": "This paper proposes a new representation of scene dynamics using neural force fields. The model takes the state of the system as input, represented using position and velocity observations for various objects in the scene as an interaction graph, and produces an estimate of the forces that are operating in the scene. These force estimates can then be used to solve an ODE that would produce future states (object positions and velocities) of the world, conditioned on past observations. \n\nThe authors show that this framework enables both prediction and planning tasks with strong performance on multiple synthetic datasets.  The model also is capable of generalizing from much fewer observations compared to existing approaches.", "questions": "My main questions are about the details of the method, and the assumptions being made on each task introduced in the results section (see point 1-4 in weakness).\n\nI’m also curious to know what the authors think about extending this method to real stimuli and what roadblocks one might face.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The methods introduced in this paper are important advances. The force field representation provides clear advantages over the standard interaction network based architectures which are based on latent space transitions. By modeling physical transitions through forces, NFF provides better physical grounding to enable generalization from a lesser number of observations.\n2. Neural force fields enable multiple tasks to be performed under the same underlying framework. The authors report strong results on future prediction, forward and backward planning on multiple synthetic datasets like I-Phyre, Phyre and N-body.\n3. Experiments in this paper are extensive. Different aspects of the framework such as grounding the ODE and cross scenario generalization are rigorously tested.", "weaknesses": "The paper is easy to follow overall, but I found that some aspects of the method are not very clear. \n \n1. The formulation requires the past history of the object states as input in order to predict forces, but it seems like eqn (1) only contains the current state of the world as input. In theory this would not work, unless you make the assumption that acceleration is not required for estimating forces, which seems incorrect. Later in L194, it’s said that the framework uses autoregressive prediction of future states given past states - it seems like this is inconsistent with eqn (1) somehow.   \n2. I’m wondering how object geometry gets modeled in this framework, because that doesn’t seem to be part of the description of the system state which includes only position and velocity. Isn’t that necessary to predict forces that result from collisions? It seems like there’s some underlying assumptions being made which is not evident when you read the paper.\n3. I think another assumption being made is that all collisions are elastic and objects are rigid. To generalize this, we would need a representation of the material properties of objects, which the authors do not discuss. Moreover, it seems like the mass of the object is not part of the state in eqn (1). It might be worth first talking about a more generic physical system which doesn’t have these assumptions, and then introduce the approximations being made. Otherwise it seems like the equations are physically incorrect. \n4. The authors talk about an image space NFF, but don’t describe how the state of the system is represented in this case. Do they use an off the shelf model to get object masks/positions, or use pre-trained features as state representation? This part was not clear to me. \n\nThe goals laid out in the intro are too broad and ambitious. The paper doesn’t really solve them, which is okay, but the limitations and the way forward should still be discussed (i.e. how might one extend this to real stimuli). Otherwise, it’s hard to understand the contributions of the paper and makes the intro a bit misleading. In fact, the conclusions seem to talk about other domains like migration patterns, etc. which I think is not useful in the light of physical reasoning goals setup in the intro.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NNgSje8D5m", "reviewer": "Reviewer_SNun", "summary": "This paper proposes a pipeline, Neural Force Field (NFF), for predicting force fields using neural networks—as opposed to directly predicting velocities or state changes. From what I understand, the main motivation is to exploit ODE integration methods by learning to predict force fields. This allows for continuous integration at arbitrary resolutions. Experiments on three different datasets show the advantage of using NFFs.", "questions": "Please also consider my concern in the weaknesses section.\n\nApart from that, I’m not sure whether training the model with new data should be called interactive reasoning. Could you provide some details on the model update? How many optimization steps are done in each round in Figure 6?\n\nAll of the compared methods are doing pixel reconstructions as far as I can tell. While the results support that predicting force fields is a better alternative when compared with pixel reconstruction, I’m not sure if it compares the model itself (i.e., the force field + ODE integration) with, say, SlotFormer, since the predictions are in different lands. I wonder how SlotFormer would perform if it’s at least predicting positions (if not forces). Or is it the case for only PHYRE tasks?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is written clearly and quite well-organized. The motivation behind most of the design decisions are explained clearly. Experiments are designed in a way to directly test the claims. Overall, I enjoyed the paper and I think it might draw some attention.", "weaknesses": "I didn’t follow the reasoning after Eq. 2. If we do not take mass into account, how can we predict the velocity accurately in general? It’s mentioned that mass is absorbed in force through correlated features like size, but is there any experiment that tests this assertion with counterintuitive sizes? Or, to put it in another term, how do the results change with objects of highly varying densities? Do you think adding a mass prediction would help? This part needs to be discussed and motivated in more detail.\n\nI’m not sure if this is due to the space constraints but, Figures 1 and 2 (especially Fig. 2) are a bit condensed and hard to parse.\n\nI'm not fully familiar with the work in this subfield, but it doesn't strike me as something brand new to do force prediction and combine it with ODE integration. Most of the related work is mentioned as it lacks robustness and sample efficiency, which makes it feel like the main contribution is the few-shot performance rather than the force + ODE combination. If that is the case, then I would wonder what's different in this method that makes it more sample efficient and applicable in few-shot scenarios---I don't think it's clear from the related work section. I tried to search for other methods, and found some, but I couldn't be sure if they are working with the same set of assumptions, and therefore, didn't want to speculate. So, I am relying on my fellow reviewers on this point, and might change my score if they think this is very close to some other work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JCGtgTeOKN", "reviewer": "Reviewer_2R3x", "summary": "This paper presents a method for learning physical systems behavior and dynamics by combining an explicit ODE solver working on top of a learned, implicit, force field. The force field is learned as a mapping between a pairs of objects states (position, velocity etc.) in the scene (given the scene graph which denotes which objects interact with which) and the scene space. The training signal comes through prediction error using ground truth trajectories and back-proping through the ODE solver into the neural networks which govern the implied force field.\n\nThe method is shown to work nicely on a variety of tasks and an ablation analysis is performed to investigate the effect of different components in the model. Furthermore the model is shown to be useful for physical based reasoning. It is demonstrated that the method is able to learn from significantly", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Generally speaking I found the paper interesting, well motivated, well executed and enjoyable to read.\n\n* The main contribution of the paper - a combination of an explicit inductive bias in the form of an ODE solver with an implicitly learned force field plays well to the strength of neural networks from one side, and the strength of explicit knowledge on the other side.\n* I think this is the first paper I see this specific combination being used.\n* The paper is clearly written and enjoyable to read. It is well structured, and well organized. Figures mostly help in understanding and the accompanying video is helpful.\n* Experimental validation is satisfactory - covering several tasks, several baselines and ample ablation. Results are convincing, especially on the data efficiency side where one would expect this model to excel at (due to the strong inductive biases).\n* While not ground breaking (see below) I think there is a valid contribution to the community here.", "weaknesses": "There are several issues, I think, with the paper, some addressable at this time-frame, some probably less so - but none, I think, are critical.\n\n* I think it would be good to show more explicitly how the mapping between the force field acting on each object and scene space - essentially, write down explicitly how to produce the force fields visible in Figure 3. In other words - how is the force field parameterized exactly in terms of coordinates? or in other words, what is d-force and how does it relate to space?\n\n* While there is a nice set of experiments in the paper - they all involve rigid object dynamics. I am wondering if it is possible to apply this method to non-rigid dynamics data - i.e, fluid dynamics, weather systems etc. This would be very compelling.\n\n* On a similar note - all the experiments here assume known ground truth states (either provided explictly, or inferred from masked images). How senstitive is the method to errors in that provided state? Is there a way to learn a mapping, for example, from images to state, by utilizing these inductive biases, or would the system collapse due to inaccuracies?\n\nGiven clarifications to the above I will consider raising my score after rebuttal as I like the paper!", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)