Test-Time Accuracy-Cost Control in Neural Simulators via Recurrent-Depth
Authors: Harris Abdul Majid
Pietro Sittoni
Francesco Tudisco
Venue: ICLR 2026 Poster (2026)
Keywords: Neural Simulator
Recurrent Depth
AI4Simulation
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Accuracy-cost trade-offs are a fundamental aspect of scientific computing. Classical numerical methods inherently offer such a trade-off: increasing resolution, order, or precision typically yields more accurate solutions at higher computational cost. We introduce \textbf{Recurrent-Depth Simulator} (\textbf{RecurrSim}) an architecture-agnostic framework that enables explicit test-time control over accuracy-cost trade-offs in neural simulators without requiring retraining or architectural redesign. By setting the number of recurrent iterations $K$, users can generate fast, less-accurate simulations for exploratory runs or real-time control loops, or increase $K$ for more-accurate simulations in critical applications or offline studies. We demonstrate RecurrSim's effectiveness across fluid dynamics benchmarks (Burgers, Korteweg-De Vries, Kuramoto-Sivashinsky), achieving physically faithful simulations over long horizons even in low-compute settings. On high-dimensional 3D compressible Navier-Stokes simulations with 262k points, a 0.8B parameter RecurrFNO outperforms 1.6B parameter baselines while using 13.5\% less training memory. RecurrSim consistently delivers superior accuracy-cost trade-offs compared to alternative adaptive-compute models, including Deep Equilibrium and diffusion-based approaches. We further validate broad architectural compatibility: RecurrViT reduces error accumulation by 90\% compared to standard Vision Transformers on Active Matter, while RecurrUPT matches UPT performance on ShapeNet-Car using 44\% fewer parameters.

Published as a conference paper at ICLR 2026
TEST-TIME ACCURACY-COST CONTROL IN NEURAL SIMULATORS VIA RECURRENI-DEPTH
Harris Abdul Majid The University of Edinburgh
Pietro Sittoni
Francesco Tudisco
Gran Sasso Science Institute
The University of Edinburgh
h, a Abdulme ji ac
ABSTRACT
Accuracy-cost trade-offs are fundamental aspect of scientific computing. Classi- cal numerical methods inherently offer such a trade-off: increasing resolution, or- der, or precision typically yields more accurate solutions at higher computational cost. We introduce Recurrent-Depth Simulator (RecurrSim) an architecture- agnostic framework that enables explicit test-time control over accuracy trade-offs in neural simulators without requiring retraining or architectural re- design. By setting the number of recurrent iterations K, users generate fast, less-accurate simulations for exploratory runs or real-time control loops, or in- crease K for more-accurate simulations in critical applications or offline stud- ies. We demonstrate RecurrSim's 's effectiveness across fluid dynamics bench- marks (Burgers, Korteweg-DD Vries, Sarrmoto-SSvvshinskyy achieving physi- cally faithful simulations over long horizons even in low-compute settings. On high-dimensional 3D compressible Navier-Stokes simulations with 262k points, 0.8B parameter RecurFNNO outperforms 1.6B parameter baselines while using 13.5% less training memory. RecurrSim consistently delivers superior accuracy- cost trade-offss compared to alternative adaptive-compun models, including Deep Equilibrium and diffusion-baate based approaches. We further validate broad architec- tural compatibility: RecurVViT reduces error accumulation by 90% compared standard Vision Transformers on Active Matter, while RecurrUPT matches UPT performance on ShapeNet-Car using 44% fewer parameters.
INTRODUCTION
Simulations are fundamental to science and engineering, They enable scientists to study and predict the behavior of complex systems, and engineers to quickly iterate and optimize designs, without the need for expensive or impractical experiments. Early scientific computing, limited by computational resources, produced crude simulations with limited practical value. Today, with the availability of enormous computers, simulations have led to breakthroughs across different domains, including numerical weather prediction, fluid and particle flow, and drug and material design. Still, even with today's computational resources, less accurate but fast simulations are essential for early-stage studies and prototyping.
In scientific computing, techniques for explicit control of the accuracy-cost trade-off are well- established. Heuristic search methods, such as genetic algorithms and simulated annealing. enable users to balance desired accuracy against available computational resources by controlling the size of the search space. For instance, genetic algorithms obtain better solutions with larger population sizes or by running more generations. Similarly, numerical methods, which have traditionally under- pinned practically all simulations, have an inherent accuracy-coo trade-off: using finer discretiza- tions, higher-order methods, and lower tolerances yields more accurate solutions but requires more computational resources to execute. For high- nigh-dimensional or large-scale problems, this trade-off becomes extremely unfavorable, rendering many real-world problems computationally intractable.

---

Published as a conference paper at ICLR 2026
Machine learning provides promising avenue to overcome this trade-off. Unlike numerical meth- ods, which rely on explicitly defined formulations or heuristics, machine learning methods are peneros-purpoee general purpose learners that learn directly from the vast amounts of available measurement and observational data, and are capable of generating simulations for a wide range of problems, ge- ometries, discretizations, and boundary conditions. Machine learning methods also benefit from hardware and software advancements specifically developed for the machine learning ecosystem, including GPU acceleration frameworks and automated parallelization tools. Additionally, in favor- able settings, machine learning methods can achieve comparable accuracy with less computational resources, or deliver greater accuracy within the same computational budget. Several works have successfully demonstrated these advantages on challenging applications including atmosphere and weather forecasting and industrial automotive design (Price et al., 2025; Bleeker et al. 2025).
At train-time, there are a number of tunable knobs available for controlling the test-time accuracy- cost trade-off of a neural simulator. Generally, allocating more computational resources during training leads to more accurate predictions through data acquisition, data augmentation, or synthetic data; by increasing the model size through stacking more layers using wider layers; or by improving the optimization process through more advanced optimizers, higher numerical precision, or training for more steps. Each of these train-tima adjustments directly affect the test-time accuracy and cost.
that is by increasing the training dataset size
At test-time, once a neural simulator has been trained, there are fewer tunable knobs. Deep Equi- librium models iterate for more steps by increasing the maximum iteration limit lowering the convergence tolerance, with additional iterations theoretically yielding solutions closer to the
true fixed point (Bai et 2019). Diffusion models can make use of additional denoising steps or more advanced samplers to generate higher-quality outputs at greater cost (Ho et al. [020 Lu et al.)
2022 Recent advances in large language model inference has developed adaptive computation methods that dynamically allocate computational resources based on input complexity, enabling fast inference on simple prompts while spending additional compute on more challenging prompts (Wei)
et al. 2022 DeepSeek-Al et al. 2025; Geiping et al. 2025).
Our contributions are as follows:
1. We introduce RecurrSim, an architecture-agnostiic, plug-and-play framework that enables explicit test-time control over accuracy-cosi trade-off's in neural simulators, allowing a sin- gle model to generate both fast simulations and high-aceuracy simulations.
2. We validate RecurrSim across diverse datasets (Burgers, Korteweg-De Vries, Kuramoto Sivashinsky, Active Matter, and ShapeNet-Cait and architectures (FNO, ViT, UPT), demonstrating physically faithful simulations even in low-compute settings, with superior memory and parameter efficiency on nigh-dimensional problems.
3. We establish accuracy-cost curves as an evaluation framework for adaptive neural sim- ulations, demonstrating that RecurrSim overcomes key limitations of existing methods: architecture dependence, early plateauing, lack of anytime prediction capability, poor gen- eralization to OOD recurrent iterations, and parameter sensitivity.
2 BACKGROUND
Partial Differential Equations. form
We consider time-dependent partial differential equations of the
+ N X, u, Ux Uxx:
0,
where L E [O, represents the temporal dimension, E X' represents the (possibly multiple) spatial dimension(s), and u(t, x) [0, x X' -> R represents the state at (1, x). Here, N is a non- linear operator that governs the systems' dynamics, describing the interactions among the different variables and their derivatives. We consider initial conditions given by u(0, x) = ux(x), and unless otherwise specified, assume periodic boundary conditions.
Discretizing the partial differential equations transforms the continuous equations into a discrete form, yielding sequence of states discrete time steps U N where N T/ At is the number of time steps At. This discretization induces an evolution operator G, which maps the state at any given time step to the state at the subsequent time step g U, -> Un41.
2

---

Published as a conference paper at ICLR 2026
Neural Simulators. A neural (physics) simulator approximates the evolution operator G with learned operator Go. often by minimizing the one-step loss C = =IU/+1 (I) using data from high-fidelity simulations real-world measurements. Repeated application of Go generates trajectory. Because the one-step loss does not measure trajectory performance, accuracy is typically quantified by a trajectory error:
N N 10 g0mh (Un)
where d denotes the number of spatial points and ()S²) denotes the n-fold application of the neural simulator. However, for chaotic systems where small errors grow exponentially, the trajectory error becomes unreliable measure of trajectory performance. Instead, we define
Ta = = i-
mátlo (Un.G." (U,) < <a).
to be the earliest time at which the Pearson correlation coefficient p between the true and predicted state falls below specified threshold E (0, 1). Computing Ta for each test trajectory yields (i) the average correlation horizon, obtained by averaging all Ta values, and (ii) the worst-case corre- lation horizon, obtained by selecting the minimum Ta. Together, the trajectory error and correlation horizons capture both long-term accuracy and stability.
Related Work. A wide range of architectures have been explored for neural simulators. For reg- ular domains, convolutional-base architectures such as the Residual Network (ResNet (He et al.) 2016)) and the U-shaped Encoder-Decoder (UNet (Ronneberger et al. 2013)) effectively capture local interactions, whereas spectral-based basee architectures, such as the Fourier Neural Operator (FNO
(Li et al. 2020h) and its factorized variant (F-FNO (Tran al. (02T)). leverage global frequency- domain features. For irregular domains, BBannstetter et al. (2022) propose a message-passing graph neural network, while [i al. (2023) extend the FNO architecture with geometry encoder and decoder, deforming an irregular mesh into uniform latent space suitable for FNO application, and subsequently reversing this deformation. Pokle al. (2022) propose FNO-DEQ, a Deep Equilibrium Model (DEQ (Bai et al. 2019) variant with Fourier layers, to solve steady-state state PDEs, showing im- provements accuracy and robustness to noise over baselines with four times as many parameters. Kohl et a (023) demonstrated that diffusion models are viable for turbulent flow simulation. Their results show that diffusion models outperform, in terms of long-term accuracy and stability, more efficient (and more commonly used) neural simulators. Recently, transformer-based architectures have gained prominence, Alkin et al.) (2024) introduce the Universal Physics Transformer, a unified Eulerian Ialerian-Lagrangian framework capable of handling large-scale simulations. Separately, |McCabe et 2023) show that a single transformer pre-trained on multiple physics tasks can match or exceed task-specific baselines without additional fine-tuning.
These diverse architectures have demonstrated the potential of neural simulators across various sci- entific domains. Luz et al. 2020) performed experiments on broad class of problems demon- strating improved convergence rates compared to highly efficient numerical methods. Pathak et al.) (220)) found that the machine learning-assister coarse-grid evolution resulted in corrected solution trajectories that were consistent with the solutions at a much higher resolution in space and time us- ing highly-efficient spectral solvers. [Stevens& & Colonius (2020) proposed method that outperforms highly efficient numerical method in simulations where the numerical solution becomes overly dif- fused due numerical viscosity[] Aurora (Bodnar et al., 2024), a foundation model for the Earth system, outperforms operational forecasts in predicting air quality, ocean waves, tropical cyclone tracks and high-resolution weather, all at orders of magnitude lower computational costs. Aurora generates these predictions in approximately 0.6s per hour lead time single 1100 GPU-this yields roughly a 100,000 times speed-up over CAMS. The fine-tuned model improves on all tar- gets with average magnitude of 54%. While these advances demonstrate the potential of neural simulators, explicit test-time control of the accuracy-cost trade-off remains largely unexplored.
'The results of Luz et al. 2000 Pathak et al. 2020) and Stevens & Colonius 2020) have been indepen- dently verified by McGreivy & Hakim (2024) and have been deemed "Tair". The verification process ensured comparisons satishidd two critical criteria: (i) comparisons equal accuracy equal runtime, and (ii) compar- ison against efficient numerical methods appropriate for each specific PDE.
3

---

Published as a conference paper at ICLR 2026
set recurrent steps K
low
hhgh
2g
x
recurrent-depth block
encoder
decoder
Z
Schematic of the Recurrent-Depth Simulator (RecurrSim) framework. RecurrSim con- sists of three main components: an encoder, recurrent-depth block, and decoder. At test-time, the user able to control the accuracy-cost trade-off by setting the number of recurrent iterations K.
Figure I:
RECURRENT-DEPTH SIMULATOR
Overview. The proposed Recurrent- Simulator (RecurrSim) framework consists of three main components: an encoder, a recurrent-depth block, and decoder (see Figure 1). The encoder transforms the input state into conditioning vector An initial latent Zo drawn from fixed distribution pzz) (which may be deterministicc). For a user-chosen number of recurrent iterations K, the decurrent-depth block R(- 0x) conditioned on iteratively updates the latent:
2x = R ([c, 2x-1), 0R), h == 1...., K.
After the final recurrent iteration, the decoder maps to the predicted state y.
Training. RecurrSim is trained end-to-end. For each training sample, number of recurrent itera- tions K is drawn from distribution p(K), the recurrent-depth block is applied that many times, supervised loss is evaluated, and gradients are mack-propagater through the computation (seel (see/Algo) Algo- rithm I). Sampling wide range of recurrent iterations K encourages the recurrent block to contract toward a fixed point.
A large number of recurrent iterations K inflates memory because every intermediate activation must ordinarily be stored. To bound the memory footprint, we use truncated backpropagation-tiee thhrouhh- depth with fixed backpropagation window B (Williams & Peng 1990). Gradients propagated through, at most, the last recurrent iterations, while earlier iterations are treated as constants. This caps memory at O(B) regardless of K and has proved sufficient for optimization. Empirically, performance saturates at B = 4, with larger values yielding diminishing returns but significantly increasing memory usage (see Appendix D
Inference. At test-time, the user is free to choose the number of recurrent iterations K according their desired accuracy and available computational resources (see Algorithm 2). Choosing small K' generates fast, less-accurate simulations ideal for exploratory runs, or real-time control loops; whereas choosing large K value generates more-accurate, but slow, simulations suitable for critical applications or offline studies. Empirically, the first few recurrent steps make the largest adjustments to the latent vector zæ: and subsequent steps contribute progressively smaller, yet still beneficial, adjustments. This behavior mirrors numerical methods, such as fixed-point and Newton methods, endowing RecurrSim with a strong inductive bias suitable for scientific computing tasks.
Modularity. RecurrSim is modular: each of, the encoder, recurrent-depth block, and decoder may be instantiated with the architecture primitive best suited to the problem-e.. e.g., convolutional layers without altering
for Eulerian simulations training or inference algorithms. The entire framework remains a standard end-to-end, supervised model with custom losses, schedulers, tricks, adoption essentially plug-and-play.
craph-convolutional layers for Lagrangian timulations-wihhoutt

---

Published as a conference paper at ICLR 2026
Algorithm Recurrent-Depth Simulator Training
Input:
training data x.
Output: model parameters Og, OR. OD
repeat
for i E: do
for every training example index in batch compute conditioning vector > sample initial latent representation sample number of recurrent iterations > unroll K recurrent iterations concatenate conditioning and latent representation apply recurrent block
E(x,, 0.)
Zo ~ p(z)
K p(K)
for k = to do
1 + [c, 1k
24 +- R(z 0R)
end for
y: D (zz 0p)
decode latent representation D compute individual loss
4 + ly. 8u1l
end for accumulate losses for batch and take gradient step until converged
Algorithm SecuurenttDeplee Simulator Inference
Input: input state x, number of recurrent iterations K, model parameters 0g, AR. OD
Output: output state - E(x, 0g) Zo pzz) for = to K do
D> compute conditioning vector sample initial latent representation D unroll K recurrent iterations D concatenate conditioning and latent representation apply recurrent block
4 c, Zk-
Z₁: f R(z OR)
end for
D(zk, 0p)
decode latent representation predicted state
Initial Latent Distribution. The initial latent vector 2o is drawn from fixed distribution p(z). Common choices include degenerate distributions (e.g., zeros or average of target fields), standard normal distributions, heavy-tailed alternatives like Student-t priors (Pandey al. 2025). Empiri- cally, this choice primarily affects early iterations, with minimal impact later as the latent converges toward the fixed point. We use the standard normal distribution (0, I) for consistency with DEQ and diffusion models.
Recurrent Iteration Distribution. log-normal distribution:
The number of recurrent iterations drawn from a Poisson
2~ N (ogg K 1 -0)
K ~ Poisson (E") +
where K is the desired mean (Geiping et al. 2025). This distribution exposes the model broad spectrum of compute budgets during training: it is positively skewed with most draws landing near K, but occasional very small and very large values are sampled, encouraging the recurrent- block to remain stable across both shallow and deep rollouts. Empirically, performance saturates around K 32 (a 0.5), with larger values providing diminishing returns (see Appendix E
Merging Conditioning and Latent Vectors. At each recurrent iteration, the conditioning vector с is merged with the current latent vector žj. The most straightforward scheme plain addition: 2/_ = Zạ. A slightly richer variant introduces learnable scalar weights: z/ ac + 32z: the weights can be made element-wise: z/. = = BOzz- Alternatives include point-wise projection, concatenating and passing the result through width-halving layer. Empirically, addition with element-wiie wise weights offers the best balance between parameter efficiency and performance (see Appendix H).
5

---

Published as a conference paper at ICLR 2026
4 RESULTS
Complete experimental details, including hardware specifications, data acquisition, data generation, preprocessing pipelines, training pipelines, and architectural configurations, are provided in Ap- We conduct experiments across diverse datasets including Burgers Equation, Korteweg- Equation, Compressible Navier-Stokes Equations, Ac-
pendix A C
De Vries Equation, tive Matter, and ShapeNet-Car, and across multiple architectural backbones including Fourier Neu- ral Operator (RecurFNO), Vision Transformer (RecurViiT), and Universal Physics Transformer (RecurrUPT).
4.1 EQUATIONS
Burgers Equation. The Burgers equation is second-order nonlinear partial differential equation derived to model convective steepening and diffusive smoothing. Its one-dimensional variant can be expressed
Ut + uMy = Vugz-
Here, y plays the role of kinematic viscosity. Setting v = O yields the inviscid form de + WMy = 0, whose solutions develop finite-time shock discontinuities; the viscous term Vulz regularises these shocks but introduces extremely thin internal layers that remain numerically stiff. Machine learning methods must learn to represent sharp gradients, moving shocks and the delicate interplay between nonlinearity and diffusion.
Korteweg- Vries Equation. The Korteweg- De Vries (KdV) third-order nonlinear partial differential equation derived to model weakly nonlinear, weakly dispersive unidirectional waves. Its one-dimensional variant can be expressed as:
We + auug + Uzer 0.
Here, (often set to +1 or =) controls nonlinear steepening while the third-order derivative Urzz introduces dispersion. The exact balance of these effects produces solitary- waae solutions (solitons) that preserve their shape and speed and undergo only phase shifts upon interaction -small amounts of artificial dissipation can destroy these very structures making KdV an ideal candidate for evaluat- ing whether machine learning methods can maintain accuracy, stability and conservation over long horizons.
Kuramoto-Sivane Sivashinsky Equation. The nonlinear partial differential equation derived model aeffusive-thermall instabilities in laminar flame fronts. Its one-dimensional variant can be expressed as:
(KS) equation is fourth-order
Ut Usz + Ugarz + uu 0.
Here, the fourth-order derivative Urazz and the nonlinear term uuy contribute to complex and chaotic behavior which present challenge for traditional numerical solvers. The challenges and the wide applicability of the KS equation make it an ideal candidate for evaluating machine learning methods.
Compressible Navier-Stokes Equations. The three-dimensional Compressible Navier-Stokes (CNS) equations model complex phenomena such shock formation and propagation. They are widely used across various engineering and physics applications, including aircraft wing aero- dynamics and the formation of interstellar gases. The equations be expressed
0,p+V (pv) = O, p(ä,v + vv) Vp + πAv + (C 7/3)V(V v),
ac(c pv2 + V (p+ p22 =
where p is the mass density, v is the fluid velocity, iss the pressure, and is the internal energy deter- mined by the equation of state. The term o' denotes the viscous stress tensor, while and Ç represent the shear and bulk viscosities, respectively. In this case, using classical numerical solver to approx- imate the fluid flow is particularly challenging due to strict stability constraints, high computational cost, and the need for accurate yet robust numerical schemes that handle shocks, dissipation, and grid adaptivity in large-scale domains. Even though machine learning can overcome several of the challenges posed by traditional solvers, training neural simulator three- dineesiieenii data comes with considerable engineering complexity. The primary limitation arises from storing the activations during training, increasing the memory requirement compared to smaller dimensions problems.

---

Published as a conference paper at ICLR 2026
Burgers Equation
Kdv Equation (Short)
Kdv Equation (Long)
Trajectory Error (L2)
Pareteters
D.at
Peranntere
0.55
EM
2m
10-1
10-2
1
16
32
1
4
16
32
Recurrent Steps (K)
Recurrent Steps (K)
Recurrent Steps (K)
Predicted
Absolute Error
Predicted
Absolute Error Predicted
Absolute Error
Figure 2: Top: Trajectory Error (L2) versus Recurrent Steps (K) for the Burgers (left), short-horizon KdV (middle), and long-horizon KdV (right). Bottom: Trajectories K 4 (orange) and K 16 (purple) (highlighted above). Increasing sharpens shocks in Burgers and aligns soliton crests KdV, illustrating how recurrent depth controls the accuracy-cost trade-off. See Figure 13 for detailed visualization of Bottom with ground truth and colorbars.
4.2 EXPERIMENT: ACCURACY-COSS TRADE-OFF
Commonly used neural simulators are trained for single accuracy-cost setting: once the model trained, every forward pass delivers the currSim, on the other hand, has a tunable knob for controlling the accuracy-cost setting (the number of recurrent iterations K). The purpose of this experiment to empirically demonstrate whether rolling out the trajectory across values of recurrent iterations K is viable.
expected accuracy and incurs the same cost. Re-
Experimental Setup. We conduct experiments three datasets: Burgers, short-horizon KdV, and long-horizon KdV. Two instantiations of RecurrSim are benchmarked. The first variant (RecurFFNO wo/ EncDec) lifts the input with a point-wise operation, recursively applies recurrent-eepth block with a single Fourier layer, and projects back to physical space; the second variant (Recur- IFNO w/ EncDec) inserts additional Fourier layer in, both, the encoder and decoder. For each variant, we target three parameter budgets (~ 1.0M, 3.5M, 7 7.5M). yielding six models per dataset. We K 32 and B 4. After convergence, generate trajectories for every K E (1,....33) 32) and measure the trajectory error. All experiments are repeated with three seeds and averaged.
Results. Across all three datasets, both variants show the same qualitative accuracy-cost curve (Figure 2). but RecurrFNO w/ EncDec achieves consistently lower trajectory error. As K in- creases, the trajectory error falls steadily and plateaus around K = 16 for Burgers and K = 8 for both, short- and long-horizon KdV further steps neither help nor harm. For each dataset, we plot low-compute (K = 4) and high-compute (K = 16) trajectories. In Burgers, the two settings reproduce the same shock patterns, with the low-compute run showing slightly larger absolute error around the fronts. In both KdV datasets, the low- low-compute the absolute error is almost entirely a small amplitude and/or phase offset, visible as narrow streaks along the soliton trajectories. Increasing to K 16 sharpens the shocks and aligns the soliton crests. These results demonstrate that RecurTENO delivers physically faithful simulations over a range of accuracy-cost settings, Extended results presented in Appendix I
already recovers the full soliton train;
7

---

Published as a conference paper at ICLR 2026
Burgers Equation
Kdv Equation (Long)
Tra Pejeectoyy Error (L2)
10a
Recurrhi HAB DED ACOM PSE Refliner
Recurifin FMO 200 KLUM
FRE Refliee
10
16-2
8
16
32 8
16
32
Recurrent Steps (K)
Recurrent Stops (K)
Figure 3: Trajectory Error (L2) versus Recurrent Steps (K) for the Burgers (left), and long-horizon
KdV (right). Curves compare RecurFNO (teal), FNO-DEQ (blue) (Marwah et al. (023). ACDM (orange) (Kohl et al., 2023), and PDE-Refiner (green) (Lippe al. (023). Across both tasks, Re- currENO achieves the best accuracy-cost curve and reaches the lowest plateau.
4.3 EXPERIMENT: ALTERNATIVES
There are a few recent neural simulators that have test-time controllable knobs. FNO-DEQ is Deep Equilibrium Model with Fourier layers whose runtime is set by a maximum number of iterations or minimum update. ACDM-a autoregressive conditional diffusion model-is able adjust the prediction quality by varying the number and schedule of denoising steps. PDE-Refiner applies the same diffusion principle in direct prediction and refinement process. In this experiment, benchmark RecurFNNO against the three alternatives under identical data and training setups.
Experimental Setup. We conduct experiments on three datasets: Burgers, long-horizon KdV, and long-horizon Kuramoto-Sivaahinsky. For RecurrSim, we carry over the best variant from the pre- vious set of experiments: point-wise lift Fourier layer encoder, recurrent-depth block with one Fourier layer, Fourier layer with point-wise projection decoder conoderr-onfferreed with 7.5M parame- ters, K 32, and B 4 steps. FNO-DEQ follows the setup of Pokle et al. 2022) with its width scaled to match a parameter count of ~ 7.5M. ACDM and PDE-Refiner use a modern UNet back- bone from their original implementations (Kohl et al. 22033 Lippe al. 2023). In early tests, both diffusion-basee models proved parameter-inefffiieen and could not rollout beyond a few steps, so we train them with 15M parameters for Burgers and KdV, and 50M parameters for KS (the scale used by Lippe et al. (2023)). After convergence, we generate trajectories for every K E (1..... 32} (where K equal the recurrent steps for RecurrSim, iterations for FNO-DEQ, and denoising steps for ACDM and PDE-Refiner). On Burgers and KdV, we measure and report the trajectory error. Since the KS equation produces chaotic behavior, we measure the average and worst-case correlation horizon over a sweep of 30 thresholds (a 0.7-0.99 in increments of 0.01).
Results. On Burgers, FNO-DEQ, ACDM, and PDE-Refiner all plateau by K (see Figure (left)); PDE-Refiner gains practically nothing beyond its second refinement step. FNO, by contrast, continues improve until K 16, while using half the parameters of the diffusion-based models. On Kdv, FNO-DEQ exhibits the convergence limitation reported by Sittoni & Tudisco (2024) the latent representation oscillates around, rather than converges to, the fixed point-so ad- ditional iterations provide no improvement. The ten-fold larger training dataset helps the diffusion- based models, however, once again, ACDM plateaus near K 11 before degrading because larger K values are out-of-distribution. RecurrFNO delivers the best accuracy-cost curves and lowest trajectory errors. On KS (see Appendix I), where the diffusion-based models have 7-fold the amount of parameters as RecurFFNO, ACDM plateaus early, and PDE-R Refiner shows erratic worst-case correlation horizons. Taken together, RecurFFNO consis- tently outperforms alternatives while using fewer parameters.
4. PDE-Refiner improves up to
8

---

Published as a conference paper at ICLR 2026
Training Memory
Training Epochs
Training GFLOPs
MSE x10 Density
MSE x10 Pressure
MSE x10 Velocity
Model
Params
FNO
0.5B
38 GB
100
1 10"
9.60 = 0.03
9.59 = 0.03
9.55 + 0.03
FNO
1.0B
57 GB
100
2 x
7.83 0.02
7.79 0.02
7.82 + 0.03
FNO
1.6B
73
100
3
7.61 0.01
7.59 0.02
7.62 = 0.03
RecunFNO
0.8B
64
82
3 10"
7.57 0.04
7.51 0.01
7.53 + 0.03
RecurFNO
0.88
64
10?
7.37
7.33 0.01
7.36 + 0.03
Table Comparison between FNO and RecurrFNO on 3D Compressible Navier- Stokes Equations.
Model
MSK Steps
MSE MSK Stepa
MSE
Steps
Original Work
UPT
2.35
RecuMMIT
75M
1.31
RecurUPT
32'
92M
NIA
2.19
Table 2: Comparison between VIT and Recur- Table 3: Comparison between UPT and Recur- rVIT on Active Matter.
rUPT on ShapeNet-Car.
4.4 EXPERIMENT: HIGII-DIMENSIONAL SIMULATIONS AND TRANSFORMER VARIANTS
Many real-world scientific simulations involve high- eigh-dimensional problems where memory inttesie approaches like ADCM and FDE-Refiner become computationally prohibitive. Additionally, the generalizability of RecurrSim across different architectural primitives and problem domains remains to demonstrated. We address these challenges through three targeted experiments: (1) evaluat- ing memory efficiency high-dimensional 3D compressible Navier-Stoker equations (262k grid points) where traditional deep networks face memory constraints, (2) demonstrating architectural
flexibility by adapting Kohl et al. (2023) 's vision transformers for active matter simulations (Ohanal et al. 2025), and (3) validating the framework's drop-in compatibility by recreating UPT's (Alkin 2024) ShapeCar- experiments, only adapting their approximator module by decreasing the depth and wrapping it in a recurrent-block significantly lowers the number of trainable parameters.
Results. RecurrSim variants consistently achieve superior accuracy with dramatically reduced computational requirements across all domains (Talle Table 2 Table 3). On 3D compressible Navier-Stokes equations, RecurFNNO with K 8 outperforms all FNO baselines, including 1.6B parameter model, while requiring less memory (64GB vs 73GB). When training epochs are matched (K = 16), RecurFNNO achieves MSE improvements on density, pressure, and velocity compared to the strongest FNO baseline. On Active Matter, RecurrViT with 75M parameters (58% of VIT's 130M) reduces error accumulation by 83% at timesteps and maintains 87% lower error at 12 timesteps. On ShapeNet- RecurUPT with 92M parameters (56% of UPT's 164M) achieves better performance to the original work, demonstrating perfect drop-in compatibility. These results establish that RecurrSim provides universal framework for test-time accuracy-cost control: users can deploy a single trained model across diverse computational budgets simply by adjusting the number of recurrent iterations.
CONCLUSIONS AND FUTURE WORK
We introduce the Recurrent-Dec Depth Simulator (RecurSim),, an architecture-agnostic framework en- abling explicit test-time control over accuracy-cosi trade-offs in neural simulators. By adjusting the number of recurrent iterations K inference, users can deploy single trained model across diverse computational mudgets-fromd fast exploratory runs to high-accuracy critical simulations. RecurrSim demonstrates physically faithful behavior across fluid dynamics benchmarks and con- sistently outperforms existing adaptive methods (FNO-DEQ. ACDM, PDE-Refiner) while using fewer parameters. On high-dimensional problems, RecurFFNO with 0.8B parameters outperforms 1.6B FNO baselines using 13.5% less memory, while RecurVVT and RecurrUPT show similar ad- vantages across transformer architectures. This plug-and aad--aaddply framework fundamentally improves neural simulator utility and opens new research directions in adaptive scientific computing.

---

Published as a conference paper at ICLR 2026
Ethics Statement. We adhere to the ICLR Code of Ethics. Our work does not involve human sub- jects, create potentially harmful insights, raise discrimination/iia/aaanss concerns, or pose privacy and security risks. Our work uses data generated from numerical solvers (which we make publicly available) and publicly available datasets. The aim of this work is to advance Al for Science. Al- though related methods may be deployed in critical applications, we do not deploy in such contexts. We have taken care to report results honestly, acknowledge limitations, and follow best practices for research integrity.
Reproducibility Statement. We have made significant effort to ensure reproducibility. We use clear and consistent terminology throughout the work. Section 3 describes our framework in de- tail, explaining each design choice and linking to ablation studies in the Appendix. Section describes the motivation and practical implementation of each experiment, with additional results in the Appendix. The Appendices provide complete descriptions of hardware specifications (Ap- pendix A). data generation and processing (Appendix B). hyperparameters and optimization details (Appendix CJ and pseudocode (Appendix J). We also provide source code for all experiments. Together, these resources enable independent researchers to reproduce and verify our findings.
REFERENCES
Benedikt Alkin, Andreas Fürst, Simon Schmid, Lukas Gruber, Markus Holzleitner, and Johannes Brandstetter. Universal physics transformers: A framework for efficiently scaling neural opera- tors. Advances in Neural Information Processing Systems, 2024.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34: 11119198--9993 2021.
Shaojie Bai, Zico Kolter, and Vladlen Koltun. Deep equilibrium models. Advances in neural information processing systems, 32, 2019.
Shaojie Bai, Vladlen Koltun, and J Zico Kolter. Multiscale deep equilibrium models. Advances in neural information processing systems, 33:5238-5255 2020.
Maurits Bleeker, Matthias Dorfer, Tobias Kronlachner, Reinhard Sonnleitner, Benedikt Alkin, and Johannes Brandstetter. Neuralefd: Deep learning high-fidelity automotive aerodynamics sim- ulations. arXiv preprint arXiv:2502.09662. 2025.
Cristian Bodnar, Wessel Bruinsma, Ana Lucic, Megan Stanley, Johannes Brandstetter, Patrick Garvan, Maik Riechert, Jonathan Weyn, Haiyu Dong, Anna Vaughan, al. Aurora: A foundation model of the atmosphere. arXiv preprint arXiv:2405.13063. 2024.
Johannes Brandstetter. Daniel Worrall, and Max Welling. Message passing neural pde solvers. arXiv preprint arXiv:2202.0333 2022.
Nanxin Chen, Yu Zhang, Heiga Zen, Ron Weiss, Mohammad Norouzi, and William Chan. Wave- grad: Estimating gradients for waveform generation. arXiv preprint arXiv:2009.00711 2020.
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng. Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang. Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang. Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge. Ruisong Zhang, Ruizhe Pan, Runji Wang, R. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing
10

---

Published as a conference paper at ICLR 2026
Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang. Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma. Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang. Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li. Ziwei Xic, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforce- ment learning, 2025. URL htt.htts::/// //axxvv org/ hte///b///22 abs/ 2501. 12948
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794 2021.
Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach. arXiv preprint arXiv:2502.05171, 2025.
Zhengyang Geng, Xin-Yu Yu Zhang, Shaojie Bai, Yisen Wang, and Zhouchen Lin. On training implicit models. Advances in Neural Information Processing Systems, 34:24247-24260, 2021.
Zhengyang Geng, Ashwini Pokle, and Zico Kolter. One-step diffusion distillation via deep equi- librium models. Advances in Neural Information Processing Systems, 36:41914-41931, 2023,
Davis Gilton, Gregory Ongie, and Rebecca Willett. Deep equilibrium architectures for inverse problems in imaging. IEEE Transactions on Computational Imaging, 7:1123-1133, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 6851, 2020.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 35:26565-26577, 2022.
Georg Kohl, Li-Wei Chen, and Nils Thuerey. Benchmarking autoregressive conditional diffusion models for turbulent flow simulation. arXiv preprint arXiv:2309.00745. 2023.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.0977 2020.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.0889) 2020.
Zongyi Li, Miguel Liu-Schiaffini, Nikola Kovachki, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Learning dissipative dynam- ics in chaotic systems. arXiv preprint arXiv:2106.0688 2021.
Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural oper- ator with learned deformations for pdes on general geometries. Journal of Machine Learning Research, 24(388):1-26 2023.
Phillip Lippe, Bas Veeling, Paris Perdikaris, Richard Turner, and Johannes Brandstetter. Pde-refiner: Achieving accurate long rollouts with neural pde solvers. Advances in Neural Information Pro- cessing Systems, 36:67398 -64333, 2023.
11

---

Published as a conference paper at ICLR 2026
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts, 2017. URL https://arxiz. org/ http/b//////oooo abs/ 1608. 03983
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URL https:
//axxiv.org//bb//11111 05101
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arX1v:2211.01095, 2022.
Ila Luz, Meirav Galun, Haggai Maron, Ronen Basri, and Irad Yavneh. Learning algebraic multigrid using graph neural networks. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research. pp. 6489 6499. PMLR, 13-18 Jul 2020. URL https: ptttps://proceedingg .ml..
press/vl19/ wtw.y11111uu22aaaam...
Tanya Marwah, Ashwini Pokle, Zico Kolter, Zachary Lipton, Jianfeng Lu, and Andrej Risteski. Deep equilibrium based neural operators for steady-state pdes. Advances in Neural Information Processing Systems, 16:15716-157331 2023.
Michael McCabe, Bruno Régaldo-Saint Blancard, Liam Holden Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg. Siavash Golkar, Geraud Krawezik, Francois Lanusse, al. Multiple physics pretraining for physical surrogate models. arXiv preprini arXiv:2310.02994, 2023.
Nick McGreivy and Ammar Hakim. Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations. Nature Machine Intelligence, (10):125--126, September 2024. ISSN 2522-5839.
doi: 10.103064225--02-00035--5 URL httt:://xx.ox/ wwt//o..//... 1038 1et.:1103322222552 422556 020-566--22-00977---
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorcalistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.107411 2021.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint arXiv:2502.0994 09999, 2025.
Ruben Ohana, Michael McCabe, Lucas Meyer, Rudy Morel, Fruzsina J. Agocs, Miguel Beneitez, Marsha Berger, Blakesley Burkhart, Keaton Burns, Stuart B. Dalziel, Drummond B. Fielding, Daniel Fortunato, Jared A. Goldberg, Keiya Hirashima, Yan-Fei Jiang, Rich R. Kerswell, Surya- narayana Maddu, Jonah Miller, Payel Mukhopadhyay, Stefan S. Nixon, Jeff Shen, Romain Wat- teaux, Bruno Régaldo-Saint Blancard, François Rozet, Liam H. Parker, Miles Cranmer, and Shirley Ho. The well: a large-scale collection diverse physies simulations for machine learning. 2025. URL UtWpttps::// /tattps://axivvvii org/ abs 2412. 00568
Kushagra Pandey, Jaideep Pathak, Yilun Xu, Stephan Mandt, Michael Pritchard, Arash Vahdat, and Morteza Mardani. Haive- diffusion models. In The Thirteenth International Confer- ence on Learning Representations, 2025. URL http:: httte::s///peeevveew net fwr.mmrriioooo
toz 1oz10EN4qp
Jaideep Pathak, Mustafa Mustafa, Karthik Kashinath, Emmanuel Motheau, Thorsten Kurth, and Marcus Day. Using machine learning to augment coarse- grid computational fluid dynamics sim- ulations, 2020. [ttpttss:://a/e
org/ abss/al10000 20tp..200..0000 00072
Ashwini Pokle, Zhengyang Geng, and J Zico Kolter. Deep equilibrium approaches to diffusion models. Advances Neural Information Processing Systems, 35:37975-37990, 2022.
Ilan Price, Alvaro Sanchez-Gonzale Ferran Alet, Tom R Andersson, Andrew El-Kadi, Do- minic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, et al. Gen- cast: Diffusion-based ensemble forecasting for medium-rante range weather. arXiv preprint 1577:9 2023.
12

---

Published as a conference paper at ICLR 2026
Ilan Price, Alvaro Sanchez-Gonzalez Ferran Alet, Tom R Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed. Peter Battaglia, et al. Probabilistic weather forecasting with machine learning. Nature, 637(8044):84-900 2025.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint 2rX1v22204.06125, 1(2):3, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and compuler-ussisted intervention- MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings, part III 18, pp. 234-241. Springer, 2015.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022a.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffuusion models with deep language understanding, Advances in neural informa- tion processing systems, 35:36479-36494, 2022b.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE transactions on pattern anal- ysis and machine intelligence, 45(4):4713-4722 2022c.
Arne Schneuing, Charles Harris, Yuanqi Du, Kieran Didi, Arian Jamasb, Ilia Igashov, Weitao Du, Carla Gomes, Tom L Blundell, Pietro Lio, et al. Structure-based drug design with equivariant diffusion models. Nature Computational Science, 4(12):8999009, 2024.
Pietro Sittoni and Francesco Tudisco. Subhomogeneous deep equilibrium models. In International Conference Machine Learning, pp.
PMLR, 2024.
Jascha Sohl-Dickstein. The boundary of neural network trainability is fractal. arXiv preprint arXiv:2402.06114, 2024.
Jascha Sohl-Dickstein. Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2265..265. pmir, 2015.
Ben Stevens and Tim Colonius. Finitenet: A fully convolutional Istm network architecture for time- dependent partial differential equations, 2020. URL bttp:://axxxv..ot///bb//// 03014
Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Daniel MacKinlay, Francesco Alesiani, Dirk Pfltiger, and Mathias Niepert. Pdebench: An extensive benchmark for scientific machine learning. Advances in Neural Information Processing Systems, 15:159611611, 2022.
Eleuterio F Toro, Michael Spruce, and William Speares. Restoration of the contact surface in the hll-riemann solver. Shock wavex, 4(1):25-34, 1994.
Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. arXiv preprint arXiv:2111. 2021.
Bram Van Leer. Towards the ultimate conservative difference scheme. Journal of computational physics, 135(2):229-25) 248. 1997.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-244 24837, 2022.
Ronald Williams and Jing Peng. efficient gradient-based algorithm for on-line training of recurrent network trajectories. Neural computation, 2(4):490-501, 1990.
Sherry Yang, KwangHwan Cho, Amil Merchant, Pieter Abbeel, Dale Schuurmans, Igor Mor- datch, and Ekin Dogus Cubuk. Scalable diffusion for materials generation. arXiv preprint arXiv:2311.09235, 2023.
13

---

Published as a conference paper at ICLR 2026
A HARDWARE DETAILS
For the one-dimensional Burgers, Korteweg-De Vries, and generated the data using an AMD 7950X processor (16 cores/32 threads). Each example trajectory in the Burgers equation and Korteweg-De Vries equation datasets took approximately 10 and 20 minutes generate, respectively. The entire datasets, with 600 examples (500 training examples and 100 testing examples), took approximately 6000 and 12000 minutes to generate, respectively. Each training example in the Suramoto-Sivashine
equations, we
equation dataset took approximately 15 minutes to generate. The testing examples were twice as long, and took approximately 30 minutes to generate. The entire dataset, with 500 training examples and 100 testing examples, took approximately 10500 minutes to generate. All together, the three one-dimensional datasets took approximately 28500 minutes (475 hours) generate.
All mie-dimensional models were trained on single NVIDIA A100 (40GB) GPU per run, with aver- age training times ranging from 15-300 minutes per model smaller models on the Burgers dataset took 15 minutes, whereas larger models trained on Korteweg-De Vries or Kuramoto-Sivashinst datasets, which contained times longer trajectories, took closer to 300 minutes. We trained ap- proximately 1000 models for exploratory experiments (e.g., tuning hyperparameters, evaluating al- ternative architectures) and final experiments, and estimate a total 1000 NVIDIA A100 (40GB) GPU hours.
The dineessimenii models were much larger. Under our experimental setup, only the smallest Fourier Neural Operator with two layers managed fit on single NVIDIA A100 (40GB) GPU. This model did not perform well (approximately 25-30% higher MSE compared to its six layer variant). So all three-dimensional experiments were trained on single NVIDIA A100 (80GB) GPU. On average, each training run took 1200-1500 minutes to complete. We trained approximately 10 models for exploratory experiments and final experiments, and estimate total of 225 NVIDIA A100 (80GB) GPU hours.
B DATA DETAILS
For the one-dimensional Burgers and Korteweg-De Vries equations, we set T 10 and T 100, respectively (for both training and testing datasets). For the one-dimensional Kuramoto-Sivashinsky 200 for the training dataset. For all set At 0.2. The spatial domain was set to X' [0, 2T) for the Burgers equation with Ar = 2zz 88922 X' = [o, 128) for the Korteweg- De Vries equation with Ar = 128/ 20244 and X [o. 64] for the Satamoto-Sivasinnsee equation, the spatial step Ar was chosen to be as small as possible while maintaining trajectory generation under pre-specified computational budget. All three domains had periodic boundaries. The initial conditions were sampled from a distribution over the truncated Fourier series with random coefficients A& U(A, A,), 4 (le, 46, les (a). and de (0),
equation, we set T' = 100 for the training dataset and three equations,
equation with Ax 64/4096. For each
10
uo(x)= = Σ A& sin 12T4 + Φx
where is the length of the spatial domain. Each trajectory was generated using the method of lines with the spatial derivatives computed using the pseudo-spectral method. For each equation, selected a time-stepping method that balances accuracy and cost: RK23 for the Burgers equation, RK45 for the Korteweg-DD De Vries equation, and LSODA for the Curamoto-Sivashing equation. See Table 4 for details.
Equation
Train
Test T
X
AA
{th d.
Time-Stepping
Burgers
10
10
0.2
0m
2 2z/8192
0.5.
(3.4.5.6
RK23
Kav
128)
128 210/1
0.5,
(1,2,3.
2m
RK45
KS
100
200
0.2
[1]
64/4096
0..5
(1,2.3.
[0.2m)
LSODA
Table 4: Data generation settings.
We construct two additional datasets, short-horizon Korteweg-De De Vries and short-horizon Kuratmoto-Sivaste Sivassiis-i.s.isky.... by considering the first 400 time steps to be part of a warmup phase and subsequently discarding them. See Table Table 5 for details.
14

---

Published as a conference paper at ICLR 2026
Equation
Warm-Up Steps
Train T
Test
Short-Horizon KdV
400
20
20
Short-Horizon
400
120
Long-Horizon Horizon KdV
Long-
0
100
100
Long-Horizon KS
100
200
Table 5: Short-horizon and long- settings.
For each the one diee-iinnsional equations, we generate 500 training trajectories and 100 testing tra- jectories. The data initially generated using double-precision floating -ointt format (float and then converted into single-precisie floating-poin formation
for our experiments.
COMPRESSIBLE LAVIER-STOKES DATASET
We the three-dimensional compressible Navier-Stokes turbulence dataset provided by Takamoto et al. (2022). This dataset consists of 600 trajectories, each containing 21 time steps, with 90% of the trajectories used for training and the remaining 10% reserved for testing. The turbulence initial condition considers turbulent velocity with uniform mass density and pressure. The initial velocity defined as
4
v(xt= 0)
Σ
A sin (k; + Φz),
where the amplitude coefficients are
4
k 1
and the characteristic velocity C.M is determined by the Mach number M and the speed of sound
Tp
C2
P
To reduce compressibility effects, the compressible component the velocity field removed us- ing Helmholtz decomposition in Fourier space, resulting in divergence-free free velocity field that preserves turbulent structures while minimizing artificial acoustic modes.
The flow parameters are set to
(n M)
1,00)
where y and ( are the shear and bulk viscosity coefficients, respectively, and M is the initial Mach number.
The data are simulated using a second-order accurate HI.IC Toro al. (1994) scheme for the invis- cid terms, the MUSCL Van Leer (1997) method for spatial reconstruction, and central difference scheme for the viscous terms.
Each time step is composed by five channels: the three velocity components, pressure, and density. and each time steps is represented on 64 grid, resulting in x 64 = 5 x 262, 144 x 1.31 x 10c data points per step. The whole dataset size is 62 GB, indeed, due memory constraints, training is performed by loading sub-bae batches of 32 samples directly from the hard disk where the dataset was stored. While this approach slows down training, it is necessary given the large dataset size.
15

---

Published as a conference paper at ICLR 2026
2n
2
3n/2
1
x
17
0
n/2
-1
2n
2
3m/2
1
x
π7
0
n/2
-1
2n
2
3m/2
1
x
π
0
m/2
2n
2
3m/2
1
x
π7
0
n/2
-1
-2
0.0
2.0
4.0
6.0
8.0 0
10 0
t
Figure 4: Example trajectories from the Burgers dataset. Train and test datasets share the same
16

---

Published as a conference paper at ICLR 2026
128
2
96
1
x
64
0
32
-1
128
2
96
1
x
64
0
32
--1
128
2
96
1
64
0
32
--1
128
2
96
1
x
64
0
32
-
-
0.0
20.0
40.0
60.0
80.0
100.0
t
Figure 5:
Example trajectories from the Korteweg-de Vries dataset. Train and test datasets share the
same 7'.
17

---

Published as a conference paper at ICLR 2026
64
2
48
1
x 32
0
16
-
64
2
48
1
x 32
0
16
-1
64
2
48
1
x 32
0
16
-1
64
2
48
1
x 32
0
16
-1
-2
0.0
40.0
80.0
120.0
160.0
200.0 0
Figure 6: Example trajectories from the -ivassiiiiyy testing dataset. The training dataset
has T = 100, whereas the testing dataset has 7 = 200.
18

---

Published as a conference paper at ICLR 2026
Trajectory 132, Time
05
0.0
8
++1
z=1
0.
000
z=32
0.50
025
0.00
z=32
12
density
-10
z=64
11
pressore
2-64
Vx
Vy
Vz
density
pressure
Figure 7: Example of two consecutive time steps from dimensiimali Navier- Navier-Stokes simula- tion, visualized across different z slicess for all physical fields.
19

---

Published as a conference paper at ICLR 2026
Trajectory 378, Time
2=1
1.0
80
-0.5
-1.0
2=1
1.0
80
-0.5
-1.0
z=32
0.5
0.0
z=32
-2.9
density
z=64
0.50
1.5
pressure
110
2-64
Vx
Vy
Vz
density
pressure
Figure 8: Example of two consecutive time steps from dirensiimnnioona Navier- Stokkes-tttkssssse simula- tion, visualized across different z slicess for all physical fields.
20

---

Published as a conference paper at ICLR 2026
C TRAINING DETAILS
Data Preparation. To minimize the one-step loss L - we require input-output pairs. Consistent with prior work (Li al. (.21), residual prediction (U, ~ Un+1 ,,)) to balance short-term (one-step loss) and long-term (trajectory) performance. We also spatially downsample 256 points. We scale each target by dividing by the maximum value across all trajectories, time steps, and spatial points; we found this perform marginally better than normalizing unit standard deviation.
the prediction step size Atp 0.8 and use
Neural Simulator Architectures.
Fourier Layer. The Fourier layer transforms the input into the frequency domain using a fast Fourier transform (FFT), applies truncated linear transformation to selected Fourier modes, and then maps the result back to the spatial domain via an inverse FFT. This spectral transformation is typically combined with skip connection consisting of point-wise convolution, bias term, and an activation function. Formally, for an input x E RR", the layer computes:
F(x) = oF (R F(x)) + Wx + b),
where F and FF1 denote the FFT and inverse FFT respectively, R R + R" is a learned linear transformation in frequency space, W R" R'' represents point-wise convolution, and b is bias term.
Several variations of the Fourier layer have been proposed. One such variant Tran al. 2021) modifies the layer by introducing a residual connection and a two-layer feedforward network, while omitting the point-wise convolution and bias term:
F(x) = x + (W2o(W;F-3 F(x)) + b,) + b2).
In our early experiments, this modification did not yield noticeable improvements. We also explored simply adding skip connection without the feedforward block and inserting normalization layers at various points in the architecture, but these did not result in noticeable improvements.
The Fourier Neural Operator is made of a point-wise lifting layer, followed by a se- quence of Fourier layers, and then point-wise projection. The Recurrent Depth Simulator (Re- currSim) with Fourier layers can be interpreted in two ways: 1) RecurrFNO wo/ EncDec, wisnt-wise lifting layer encoder, followed by sequence of Fourier layers (that make up the recurrent-depth block), and then a point-wise projection layer decoder, or 2) RecurTNNO w/ EncDec, where the first Fourier layer part of the encoder and the last Fourier layer part of the decoder. We find that RRecurrFNO w/ EncDec often leads to more consistent and superior performance.
RecurrENO
RecurrSim. The Recurrent Depth Simulator is highly flexible framework. Each component- the encoder, recurrent-depth block, and decoder-may may be instantiated with any layer(s) depending on the task. For example, in problems with periodic boundaries and requirement of parameter efficiency, where the Fourier Neural Operator would typically shine, Fourier layers can be used. On the other hand, if the goal is develop a foundation model for physies on irregular meshes, where one might use a graph-based encoder, with an attention-basee bottleneck, and a graph-based decoder, the RecurrSim framework can be configured accordingly. With just few additional lines of code, RecurrSim enables explicit control over the accuracy-cost trade-off (see Appendix for pseudocode).
Fourier- and attention- layers are well-suited for recurrent-depe -erth blocks due to their ability to model infinite receptive fields. In contrast, convolutional- based layers have a fixed receptive field that grow with the depth. For example, standard convolutional layer in PyTorch with dialation=1, and stride = 1 has a receptive field of size 3. Stacking two such layers increases he receptive field to capatruingg the center point and two neighbor- ing point on each side. More generally, the receptive field after stacking L such layers is given by L- (kernelsi 2e/2) + 1. To achieve receptive field of size 64. effectively model the Burgers equation, one would need to stack 63 layers. In RecurrSim, where K = 1 could be sampled, 63 lay- would need to be distributed across the encoder, recurrent-depth block, and decoder. To mitigate
21

---

Published as a conference paper at ICLR 2026
this, some alternatives be considered to expand the receptive field more efficiently: increasing the kernel size, incorporating attention-baan basod layers, or adding downsampling blocks.
FNO-DEQ. Similarly Marwah al. (2023), we use Anderson acceleration with maximum of 16 iterations. For the backward pass of the DEQ layer, we follow the phantom gradient approach proposed by Geng et al. (202T) using parameters
3 and T 0.8. To match the parameter count of RecurFNNO, we employ ID FNO with 8 layers and 120 channels.
ACDM. We follow the original setup from Kohl et al. (023), using linear scheduler and training with maximum of 50 diffusion steps. For conditioning, we concatenate the snapshot from the previous time step, i.e., the solution 24 when predicting U++1- To ensure a fair comparison, we condition only u, and do not include earlier time steps.
PDE-Refiner We use the same scheduler proposed by Lippe et al. (2023). with o 2-10 and K 10. Following similar approach Kohl al. (2023) we implement the following algorithm from scratch:
Algorithm 3 Reffneeiner: Training and Inference Procedures
1: procedure TRAINSTEP(M, Upev)
2: < random integer in [0, num_steps]
3: if k 0 then
4: pred < Neural@perrator(zers like(w)), Uprevs k)
5: target 4- U1
6:
else
noise std <- min www._ooiieescem std 4/mum.sstspp
7:
8:
noise -- randn
randn_like(u))) like(U)) (t))
9:
uninoeeed We + noise noise_sst _td
10:
pred <- NeuralOperator((u (M, nniseed Uprevs k:)
11:
target noise
12: end
loss (SS((pre)) target)
14:
return loss
end procedure
16: procedure PREDICTNEXTSOLUTIIO ((
17:
Mg < NeuralOpperator(Zerus like likee())rev). Uprev 0)
18:
for to num steps do
noise std < min noise //umm//ssss
20:
noise +- randn like(w))
"i iinoised f Ug + noise noise_std
22:
pred <- NeuralOperraoo((uu (Mjjera noised Uprevy k)
23:
Ug f- "ijnissed pred noise_std
24:
end for return ua
25:
26:
end procedure
Algorithm 3 is taken from Lippe et al. (0033) and the number of inference num-steps is fixed at test time. To adapt the original algorithm, we investigated two variations: (Agorithm and 5 When K = num steps, both methods recover the original procedure proposed in Lippe et al. (023
The first variation, Algorithm 4 adjusts the noise scheduler based on the number of inference steps. However, this strategy only performs well when the number of steps matches the training setup. To address this limitation, we introduce Algorithm 5) which retains the noise scheduler from training while allowing the number of inference steps vary. This consistency in noise levels enhances stability and performance by preserving the distribution the network was trained on.
22

---

Published as a conference paper at ICLR 2026
Algorithm 4 Predict Next Solution
procedure PREDICCTNEXTSSOLUTIONCCCCC U1 < ZeuraroOperatorrzeros ("ppre)) Upreve 0) for k = 1 to K do
noise_std -- citiinnois__td/////
noise randn randn_ike((u)
innissed - 4i + noise noise std
pred f- NeuralOperator((uu noiseed: Uprev k))
U; "i,noised pred noise std
end for return uę end procedure
Algorithm 5 Predict Next Solution 2
1: procedure PRIDICTNEXTSOLUTIOON
2: Ug Neural@peratto(zerros like ), Uprev 0)
3: for k = 1 to K do
4: noise_std < wwn moiseestdd//mnan
5: noise +- randn (i,)(())
6:
iniised f- 06 + noise noise std
7: pred 4 Neurallpperatoortuu noised noised Uprovs k)
8: "g 4- miniseed pred noise_std
9:
end for
10: return uz
11: end procedure
Optimization. All optimization hyperparameters are listed in Table and remain fixed across all experiments, except where explicitly stated. We train each model for 100 epochs using the Adam W optimizer (Loshchilov & Hutter 2019), starting with learning rate of 3 10 and a weight decay of 1 x 10 A cosine annealing schedule is applied to gradually reduce the learning rate to 3 x 10-6 [Loshchilov & Hutter 2017). In early experiments, we observed that using a higher initial learning rate (e.g.. 1x >) led to less consistent performance, though it occasionally improved performance
Sohl-Diccstein 2024).
Hyperparameter
Value
Epochs Batch Size
100
256 AdamW
Optimizer
Starting Learning Rate
3 10
Weight Decay
x 10-5
Scheduler
Cosine Annealing
Ending Learning Rate
3 10
Table 6: Optimization hyperparameters used in all experiments.
For the three-dimensional experiments, a batch size of 32, and perform gradient accumulation to have effective batch size of 256.
23

---

Published as a conference paper at ICLR 2026
D BACKPROPAGATION WINDOW
During training, the recurrent-depth block repeated times in the forward pass, after which gra- dients are propagated backward through the same computation. If K is large, which could happen because K drawn from a long-tailed distribution, the backward pass must retain every interme- diate activation, quickly exhausting GPU memory. To cap the memory usage, we use truncated eeckpropagation-thougggge
with a fixed backpropagation window B: gradients are backprop- agated through most the last B steps, and earlier steps are treated constants. This bounds memory at ( independent of K. In this experiment, we study it truncated backpropagation- through-depth viable and the effect of different backpropagation windows.
Burgers, long-horizon KdV, and long-horizg hong-horizon KS. We train a recurrent depth simulator with a point-wise lifting layer, a recurrent- depth block with a single Fourier layer, and a point-wise projection layer with ~ 1M parameters. We set K 32, and the backpropagation window is swept over B {1, 4, 16, 32). With B I the compute for the forward pass is equivalent to Fourier layer with 33 layers, but the backward pass stores only single activation; with B 32 the backward pass stores every activation whenever K S 32 and the last 32 when K > 32. This would be infeasible for higher-dimensional problems.
Experimental Setup. We conduct experiments on three datasets:
Across all equation B performs worst and moving from B = 1 to B = yields the largest gain, and improvements largely saturate by B = 4. Beyond = 4, larger windows offer only marginal benefit while reinstating substantial memory cost. Note that although trajectory error is not the preferred metric for KS, the same saturation is evident. Based on these results, and to balance accuracy and memory, we set B = in all main experiments.
Results. We report the trajectory errors in Table 7
Backpropagation Window B
Burgers
Korteweg-De Vries
Kuramoto-Sivashinsky
]
0.0849
0.1046
1.6341
2
0.0315
0.0522
1.4097
4
0.0199
0.0317
1.3972
16
0.0181
0.0302
1.3960
32
0.0178
0.0298
1.3910
Table 7: Impact of the back poccppropagation window B trajectory error. Accuracy improves sharply up to B and then plateaus.
E DISTRIBUTION PARAMETER K
The distribution parameter K controls the expected number of recurrent steps during training. Set- ting K too low shortens training time but may leave the model under-exposed large K values during inference; setting it too high increases training time. In this experiment, we wish to identify the optimal K.
Experimental Setup. We conduct experiments on three datasets: Burgers, long-horizon KdV, and long-horizon KS We train a recurrent depth simulator with a point-wise lifting layer, a recurrent- depth block with single Fourier layer, and point-wise projection layer with ~ 1M parameters. The backpropagation window is fixed at Doubling K roughly doubles the forward cost, yet backward memory remains capped by B; for in- stances, K matches the forward FLOPS of an 8-layer FNO but the truncated-backe backpooopcckkroaaattinnn through-depth keeps the backward pass FLOPs as cheap as 4-layer FNO. After training, each model is evaluated at all values K E [1, 2K] and we report the lowest trajectory error achieved.
4, and K is swept over {1, 2, 4, 8, 16, 32, 64, 128).
plot trajectory error as a function of K. Increasing K consistently lowers the best achievable trajectory error, but we observe diminishing returns beyond K x 32, We also notice that models trained with larger K underperform with small K values (see Figure 20p. In other words, the additional training compute shifts the accuracy-cos. curve to the right and gains appear only once K is allowed to grow. Based these results, set K 32 in our main experiments as captures the bulk of the benefit of high-compute settings while leaving the model competitive in low-compute settings.
Results. Figure 11
24

---

Published as a conference paper at ICLR 2026
1e-01-
Trajec Trajectory Error (L2)
1e-02 02
816
32
64
128
R
Figure 9: Choosing the distribution parameter K on the Burgers dataset.
1e-01-
Trajectory Error (L2)
1e- 02
816
32
64
128
K
Figure 10: Choosing the distribution parameter K on the KdV dataset.
1e+01
Trajectory Error (L2)
1e+00
8
816
32
128
K
Figure 11: Choosing the distribution parameter K on the KS dataset.
25

---

Published as a conference paper at ICLR 2026
F FIXED POINT
At test-time, the user is free to select the number of recurrent iterations according to their desired accuracy and available computational resources. Figure 2 (Top) and Figure demonstrate that in- creasing recurrent iterations K yields more accurate trajectories. To understand this behavior, we investigate how the latent vectors 2 evolve with increasing recurrent iterations K and whether they converge to fixed points 2".
Experimental Setup. We conduct experiments on two datasets: Burgers and long-horizon Kdv. the trained RecurrENO w/ EncDec models from Section 4.2 with K 32. For each timestep in the trajectory, we measure the Euclidean distance between latent vector 21: and approximate fixed point 2 2 for all l [0, K]. We consider the convergence criterion || 20 for a chosen tolerance e.
Results. Figure shows predicted trajectories for Burgers (Top Left) and long-horizon KdV (Top Right). The heatmaps (Bottom) show the Euclidean distance between latent vector and approx- imate fixed point 2 2 for all k E [0, K] across time. For both equations, we see successive recurrent block applications generate increasingly similar latent vectors || - = < || =1||, eventually satisfying our convergence criterion || =*1| < -2, This indicates practical conver- gence to fixed point. Remarkably, the convergence behavior reflects the underlying physics. For the Burgers equation, which exhibits two distinct regimes, an early nonlinear phase that develops shocks and later convergence during the methods. For long-horizon KdV, we find spikes throughout the trajectory due to the more difficult task of soliton interactions. These results demonstrate that RecurrSim's 's latent vector does converge toward fixed point and that the model behavior is aligned with the underlying physies.
phase, we find that more recurrent iterations required for period. This behavior matches that of adaptive numerical
Burgers Equation
KdV Equation (Long)
x
X
|| -2-1
1 12-211
10a
10a
Recur rent Steps (K)
Recur rent Steps (K)
101
10
100
10
10-1
10-1
10-2
10-2
Figure 12: Convergence analysis showing predicted trajectories (top) and heatmaps of ||= - (bottom) for Burgers (left) and KdV (right) equations. Darker colors indicate larger distances from the approximate fixed point.
26

---

Published as a conference paper at ICLR 2026
G INITIAL LATENT DISTRIBUTION
RecurrSim's ss initial latent vector 2g is drawn from a fixed distribution p(z) and iteratively updated through the recurrent-depth block. A good initial latent vector could potentially require fewer itera- tive updates to reach better latent vectors for decoding. In this experiment, we study how different initial latent distributions affect performance across varying recurrent iterations K.
Experimental Setup. We conduct experiments two datasets: Burgers and long-horizon KdV. We train a recurrent depth simulator with a point-wise lifting layer, a recurrent-depth block with single Fourier layer, and point-wise projection layer with ~ 1M parameters. We set the backprop- agation window B and K = 32. We ablate four different initial latent vector choices:
1. Standard normal: Zu IO,
2. Zero initialization: Zg (Dirac delta zero, 50)
3. Conditioning initialization: Zo (Dirac delta at conditioning vector, 8e)
4. Learnable initialization: Zo = (learned parameter independent of input)
After training, we evaluate the model at recurrent iterations K E {2, 4, 8, 16,32) 32} and report the trajectory error.
Results. Table 8und Table show trajectory errors for all four initial latent vectors across different values of K. For Burgers equation, standard normal and zero initialization (ôp) perform the best. At K = 2. the lcarnable latent vector performs poorly, but by K = 16. all initial latent vectors reach similar performance. For long-horizon KdV, zero initialization performs best, but we observe similar behavior: all initial latent vectors reach similar performance. Across both datasets and all recurrent iterations K 4, 8, 16, (2). we find zero initialization provides the most consistent performance. However, standard normal performs comparably and is consistent with DEQ and diffusion models. These results demonstrate while the initial latent distribution can play a part early given sufficient recurrent iterations, different distributions reach similar performance.
Recurrent Iteration
K
Zg
Zg
5g
Zg
de
Zg 2g0
2
0.3609
0.2183
0.5553
1.5921
0.0968
0.0789
0.1287
0.1500
8
0.0192
0.0199
0.0293
0.0205
16
0.0101
0.0101
0.0132
0.0102
32
0.0100
0.0099
0.0129
0.0102
Table 8:
Trajectory error on Burgers for four initial latent distributions across five recurrent itera- tions. Best result in each row is bold, second-best italic.
Recurrent Iteration K
Zu ~ A(,0 I)
Zg ~ 5g
Zg ~ &e
Zu = Zg
2
0,5892
1.2367
1.8251
0.7057
4
0.1570
0.1176
0.2412
0.1181
8
0.0188
0.0167
0.0194
0.0201
16
0.0187
0.0164
0.0176
0.0205
32
0.0187
0.0164
0.0177
0.0205
Table 9: Trajectory long-horizon KdV for four initial latent distributions across five recur- rent iterations. Best result in each row is bold, second-best italic.
27

---

Published as a conference paper at ICLR 2026
H MERGING
At each recurrent step, the recurrent depth simulator must merge the condition vector с with the latent vector Zk. We consider six merging methods of increasing capacity. Add simply sums the two vectors. Add, introduces two learnable parameters and [ (z4 = ac BZz)). Add, generalizes this element-wise vectors and [ (2 x hiddenchannels additional trainable parameters). Projection concatenates [c, Z₁ and applies a point-wise linear map (2 x hiddenchanne nideecchaneels additional trainable parameters); Projection, uses the same layer but is ini- tialized with 1s along the diagonals and Oss everywhere else, so that it is equivalent to Add, at initialization but with increased capacity. Concat feeds the raw concatenation into the first layer (in the recurrent-detth block), doubling its input channels, and thus, trainable parameters. In this experiment, our goal is to test these merging methods.
x
the one-dimensional Burgers equation. The base architecture is fixed-e point-wise lift, single Fourier layer encoder, a one-layer Fourier recur- rent block, and Fourier decoder with point-wise projection projection-trained trained with K 32 and back- propagation window B = 4. We sweep five parameter budgets {0.2M, 0.5M, 1.0M, 2.0M, 4.0M} by scaling channel width, and implement each of the six merging methods every budget. After training. each model is evaluated at all values K [1, 2 and we report the lowest trajectory error achieved.
Experimental Setup. All experiments
Results. Table 10 reports the lowest trajectory enror for every configuration. The three addition variants perform almost identically and improve monotonically with parameter count. The Projec- tion variant lags behind, but when initialized with 1s along the diagonals (Projectiony), it matches or exceeds the additional family. Concat attains the lowest error overall, but at the price of ~ 33% extra parameters in the recurrent-block's first layer; hypothesis that part of its gain stems from increased model size rather than superior merging mechanism.
Parameters
Add
Add,
Adde
Projection
Projection
Concat
0.2M
0.0234
0.0230
0.0229
0.0240
0.0240
0.0214
0.5M
0.0176
0.0173
0,0172
0,0223
0.0135
0,0146
1.0M
0.0129
0.0126
0.0126
0.0169
0.0101
0.0151
2.0M
0.0116
0.0115
0.0115
0.0094
0,0093
0.0090
4.0M
0.0100
0.0098
0.0099
0.0110
0.0100
0.0083
Table 10: Trajectory error on Burgers for six merging methods across five parameter budgets. Best result in each row is bold, second-best italic.
28

---

Published as a conference paper at ICLR 2026
1 MORE EXPERIMENTS
1.1 EXPERIMENT: ACCURACY-COST TRADE-OFF (EXTENDED)
Ground Truth
Predicted
Absolute Error
10/2
0.075
658
3n/2
0. 0.075
4.8
7.2
9.6
7.2
9.6
9.6
Ground Truth Predicted
Absolute Error
06-
5,975
64
8.850
9.025
8.389
9.975
5.050
32
5.925
3.000
28
12
16
28
12
Ground Truth
Predicted
Absolute Error
128
8.188
0.075
X
0,850
128
0.675
X
0.025
0,000
Figure 13: Supplementary visualization for Figure 2 (Bottom) showing ground truth, predictions, and absolute errors with colorbars. Top: Burgers, Middle: short-horizon KdV, Bottom: long- horizon KdV. Within each equation: low-compute K 4 (orange border) and high-compute K 16 (purple border). RecurrSim maintains physical fidelity at both computational budgets, with errors decreasing K increases.
29

---

Published as a conference paper at ICLR 2026
Predicted
Absolute Errar
Predicted
Absolute Error
Predicted
Absolute Error
Predicted
Absolute Errar
Predicted
Absolute Error
Predicted
Absolute Error
Predicted
Absolute
Predicted
Absolute Predicted
Absolute
Figure 14: Burgers: Trajectories K (orange) and K 16 (purple).
30

---

Published as a conference paper at ICLR 2026
Predicted
Absolute Errar
Predicted
Absolute Error
Predicted
Absolute Error
Predicted
Absolute Errar
Predicted
Absolute Error
Predicted
Absolute Error
Predicted
Absolute
Predicted
Absolute
Predicted
Absolute Error
Figure 15: Short-Horizon KdV; Trajectories
4 (orange) and K 16 (purple).
31

---

Published as a conference paper at ICLR 2026
Predicted
Absolute Errar
Predicted
Absolute Error
Predicted
Absolute Error
Predicted
Absolute Errar
Predicted
Absolute Error
Predicted
Absolute Error
Predicted
Absolute
Predicted
Absolute
Predicted
Absolute Error
Figure 16: Long-Horizon KdV: Trajectories at K 4 (orange) and K 16 (purple).
32

---

Published as a conference paper at ICLR 2026
1.2 EXPERIMENT: ALTERNATIVES (EXTENDED)
Ground Truth
Predicted
Absolute Error
2n
0.100
30/2
0.075
2n
9. 100
2n/2
075
0.0
7.2
9.6
0. 0.000
Ground Truth
Predicted
Absolute Error
2n
0.100
2n/2
0.075
25
8. 100
3n/2
075
8.0
7/2
0.89
Ground Truth
Predicted
Absolute Error
2n
0.109
3n/2
0.075
0.025
2c
0.100
3p/2
7/2
8.825
8.0
8,089
4.3
2.4
t
Figure 17: Performance of baseline methods on Burgers equation. Top: FNO-DEQ. Middle: ACDM, Bottom: PDE-Refiner. Within each method: low-compute setting (orange border) and high-compute setting (purple border). FNO-DEQ shows minimal improvement with additional compute. ACDM and EDE-Refiner fail to accurately reproduce shock structures at low compute and show limited improvement high compute.
33

---

Published as a conference paper at ICLR 2026
Ground Truth
Predicted
Absolute Error
96
8.975
126
9.389
9,050
32-
3.925
3.069
20
40
89
120
46
50
80
160
20
48
68
100
Ground Truth
Predi cted
Absolute Error
9.075
64-
8.058
8.925
96
8.975
32
5,000
Ground Truth
Predicted
Absolute Error
128
9.198
0.058
0,025
0.775
X
0.859
8.925
28
189
Figure 18: Performance of baseline methods on long-horizon KdV equation. Top: FNO-DEQ. Middle: ACDM, Bottom: PDE-Refiner. Within each method: low-compute setting (orange border) and high-compute setting (purple border). All three baselines fail to accurately reproduce solitons.
34

---

Published as a conference paper at ICLR 2026
Table 11 summarizes the best performance achieved by each method across all recurrent iterations K" (mean = standard deviation over 3 seeds). RecurFFNO consistently outperforms all baselines across both trajectory error (Burgers, KdV) and correlation horizons (KS), achieving 82% lower error than the best baseline on Burgers, 21 % lower on KdV, and up to 98% longer correlation horizons on KS.
KS Avg. Corr. 0.8(1)
KS Arg. Corr. 0.9(1) a
Model
Burgers MSE()
Kdv MSE(()
a'
RecurFFNO
0.008 + 0.001
0.022 =
0.005 0.005
74.7 + 4.2
69.2 = 3.2
FNO-DEQ
0.045 0.007
0.094 + 0.028
41.9 + 5.6
37.0 + 4.6
ACDM
0.044 + 0.004
0.043 + 0.015
40.5 + 9.3
35.6 9.2
PDE-Refiner
0,161 + 0.024
0.028 = 0.010
61.6 = 7.7
56.6 + 8.8
Table 11: Best performance across all recurrent iterations K for RecurFNNO and baseline adaptive- compute methods (mean + std over 3 seeds).
On the chaotic Kuramoto- case correlation horizon metrics. Figure 19 shows the behavior of the four adaptive-compute simu- lators across 30 correlation thresholds (a = 0.70 - 0.99) and all inference depths K E (1...., 16). RecurFNOO (first column) shows the desired monotone pattern: both the average and the worst- cast correlation horizons rise steadily with K. FNO-DEQ delivers flat surfaces-in its iterations leave the horizon essentially unchanged-so it cannot expliot extra compute. ACDM begins with short horizons, improves up to K = 4. and then flattens; only a narrow band of K values is usable, lim- iting its test-time flexibility. PDE-Refiner gains up to K = 8 but then oscillates, making hard to pick reliable stopping point. Across both average and worst-case statistics RecurrSim attains the longest horizons and is the only model whose accuracy scales predictably with additional compute, confirming its advantage for controllable accuracy-cost trade-offs in chaotic regimes.
Kuramoto-Sivashinse Sivashinsky dataset we replace trajectory error with the average and worst-
RDS
FNO-DEQ
ACDM
PDE-Re DDE-Refiner
100
Average
80
Steat (93
Street
StOP
Stoot
Poredail 4.75
60
0.7
Time
worst -oort--Case Case
40
20
Recurrest Steee
Recorrent Stopt
becorrest Steee
17
4.13
0
Figure 19: Kuramoto- Kuramoto-Sivashinsky average (top) and worst-case (bottom) correlation horizons and threshold a versus inference depth K.
35

---

Published as a conference paper at ICLR 2026
1.3 EXPERIMENT: NIGH-DIMENSIONAL SIMULATIONS AND TRANSFORMER VARIANTS (EXTENDED)
As shown in fig. 20 the two models present distinct trade-offs. When the number of recurrent steps during inference exceeds 8, the model trained with a higher FLOPs budget and a higher K yields significantly lower MSE. contrast, for fewer than 8 recurrent steps, the model trained with lower computational budget performs better.
Pressure
Density
Velocity
Lover training FLOPs
Lower training FLOPS
training FLOPs
Trajectory Error (MSE)
Higher training FLOPs
Higher training FLOPs
Higher training PLSPs
9 9x10-
8 8x
Recurrent Steps
(K)
Recurrent Steps (K)
Recurrent Steps (K)
Trajectory error (MSE) over the number of recurrent steps for two RecurrFNO models, trained with lower and higher FLOPs budgets
Figure 20:
Figure 21
shows the trajectory error as a function of recurrent steps K for RecurrViT on the Active Matter dataset, evaluated at three different rollout horizons (steps 0:3, 0:6, and 0:12). The dashed lines represent the performance of a standard ViT baseline with comparable parameter count.
RicurVVT demonstrates monotonic improvement with increasing K across all rollout horizons. For short rollout horizons (0:3). Recurr ViT achieves 83% lower error than ViT (0.005 vs 0.029). For medium rollout horizons (0:6), RecurrViT achieves 90% lower error than ViT (0.013 0.124), and already surpasses ViT performance by K = 8. For long rollout horizons (0:12), RicurrVTT achieves 87% lower error than VIT (0.057 vs 0.432).
Steps
Recor/ti1
VIT
VIT
Vit
Trojectory Enror (PS5)
10/7-
Recurront Stops
Recurrent Stops
Recurront
Figure 21: Trajectory error (MSE) over the number of recurrent steps K for RicurrViT (solid lines) and standard ViT (dashed lines) on the Active Matter dataset, evaluated at three rollout horizons: steps 0:3 (left), 0:6 (middle), and 0:12 (right).
36

---

Published as a conference paper at ICLR 2026
J PSEUDOCODE
2.1. lass Network (Module)
det _iiii_itttt_ (self):
super f) init_
Encoder Layer
selffencoder Layer ()
Collect L Intermediate Layers layers [] for in range (L)
10
layers.appende (Layer )
11
12
# Decoder Layer
Layer
15
def forward (self, x)
16
# Apply Encoder
= salEdencoder (x)
19
20
ААНА Main Blook UAAAA
21
Apply L Intermediate Layers for layer in self. z layer (z)
24
25
26
#*## Main Block #
28
29
30
33
34
35
37
38
39
42
43
44
47
48
52
53
Apply Decoder
56
x sel f. decoder (z)
57
return x
Listing 1: Pseudocode of standard neural simulator. The neural simulator contains an encoder or lifting layer (self. encoder). L intermediate layers of any type (residual layers, Fourier layers, etc.), and an decoder or projection layer (self f. decoder).
37

---

Published as a conference paper at ICLR 2026
class Network (Module):
def _init_ (self)
super _innt__iiitt # Encoder Layer
self = (ayer()
# Collect Intermediate Layers layers = II
for in range (I)
Layers append (Zayer()) )
12
Decoder Layer
13
self.decome Layer ()
def (sewa x, K=None)
16
Apply Encoder
17
с sclf.encoder (x)
18
A ***## Main Block #***#
21
22
# Sample Noise \w "shape=x.shape" cimple_nossel
25
25
During Inference:
26
if not self.trat traiinnai
27
A Loop K Times
for in range(K):
# Concatenate and
30
z cl=, z), dim=l)
31
# Apply 1 Intermediate Layers for layer in sclf.layers: (z)
32
35
# During Training:
36
if self. self.training:
# Do Not Use Grad with no_grad
39
# Sample K (Using bar)
40
K sample_KX K()
# Loop K - Times for in range B):
43
2 ((c, 2l, dim=1) for layer in self. layors:
44
45
2 (z))r
# Loop Remaining B Times for range (B) 2 cle, 2), dim=1) for layer in self.layere = = layer
48
49
50
52
$*444 Main Block ++E ААААААнннчААКеНил
53
54
# Apply Decoder self.decoden
57
return
Listing 2: Pseudocode of the Recurrent Depth Simulator-fewer than 20 new lines compared to a standard neural simulator. During inference, we apply the intermediate layers K times. During training, we apply the intermediate layers K B times without gradient, and B times with gradient. Nothing else needs to change.
38

---

Published as a conference paper at ICLR 2026
K EXTENDED RELATED WORK
Deep Equilibrium Models. Deep Equilibrium Models (DEQs), introduced by Bai et al. (2019) are implicit, infinite-depth, weight-tied neural networks. A DEQ directly solves for the fixed point of a nonlinear transformation using any black-box root-finding algorithm and instead of backpropa- gating through each layer, which can be infeasible due memory and numerical stability, the DEQ makes use of the Implicit Function Theorem to compute the gradients at the equilibrium this ap- proach has a constant memory requirement regardless of depth. Although the existence of the fixed point, or convergence to the fixed point, is not guaranteed; on large-scale language modeling tasks, Bai et al. 2019) demonstrated that DEQs can achieve performance comparable with state-of-the- art while using significantly less memory. Later, Bai et al. (2020) extended DEQs to large-scale computer vision tasks, showing similar performance and memory benefits. Subsequent research ex- plored DEQs for various applications. Poklee et al. (2022) represent the entire sampling process in denoising diffusion implicit models as a single fixed-point system. Geng et al.) (2023) distill diffu- sion models, directly from initial noise to the final image, into a DEQ. In inverse problems, Gilton eet al. (2021) model a, potentially infinite, iterative reconstruction scheme as DEQ. For partial dif- ferential equations, [Pokle et al. (2022) propose FNO-DEQ, DEQ variant with Fourier layers, to solve steady-state PDEs, showing improvements in accuracy and robustness to noise over baselines with four times as many parameters.
Denoising Diffusion Models. First introduced by Sohl-Dicksterd et al. 20155) diffusion models are probabilistic models with an iterative forward diffusion process and a learned reverse diffusion process. The forward process gradually adds noise to data until only noise remains, and the reverse process gradually removes noise to restore the original data. New samples are generated by sampling noise vector and passing it through the reverse process. HH al. (2020) presented high-quality image synthesis results using diffusion models. Dhariwal & Nichol (021) and Karras et al.] (2022) made further progress leading to state-of-the-ard results and widespread adoption. Diffusion mod- els have been applied to image generation (Nichol et al. [2021 Ramesh et al. 2022; Saharia et al,, 20226), image inpainting and outpainting (Saharia et al. (2022a), super-resolution Saharia et al. 2022c), audio generation (Chen et al., [2020] Kong et al. (020), text generation (Austin et al. [021] including large language (diffusion) models (Nie et al. 2025). In scientific domains, diffusion mod- els have been applied to medium-range weather forecasting (Price et al. 2023), structure-
drug design (Schneuing et af., (004)), and stable materials generation (Yang et al., (023). Kohl et al. (223) demonstrated that diffusion models viable for turbulent flow simulation. Their results show that diffusion models outperform, in terms of long-term accuracy and stability, more efficient (and more commonly used) neural simulators. Kohl et al. (2023) also compared against PDE-Refine (Lippe et al., 2023), a diffusion-basee multi-step refinement process, but found that PDE-R Refiner is highly sensitive to hyperparameters, and in some cases, generated substantially worse results com- pared to other methods.
L EXTENDED DISCUSSION
To our knowledge, this is the first work to study neural simulators terms of test-time control of accuracy-cost trade-offs. Since the performance varies with the chosen number of recurrent steps K, a scalar metric is no longer adequate: our experiments therefore focus on full accuracy-cost curve, and correlation-oorizon surfaces. Across all tasks, the Recurrent-Depth Simulator provides a smooth, monotone trade-off, demonstrating that adaptive compute is possible, and we hope these results stimulate further work along this new axis.
Although the main experiments concentrate on RecurrSim instantiated with Fourier layers chosen for their infinite receptive field (see Appendix prelimiinay tests with convolutional blocks yield qualitatively similar results. We also use a recurrent-block with single-layer for clarity: it delivers the most predictable behavior, however, deeper blocks also showed strong performance. Explor- ing richer blocks and alternative layer types under this controllable-comput paradigm remains promising direction for future research.
39


Reviews:
[{"review_id": "PGuKei1hbl", "reviewer": "Reviewer_XUFK", "summary": "The authors propose a very simple framework for controlling, at test-time, the accuracy/speed of a neural simulator model, without requiring retraining or architecture adaptations. They show that this technique can be incorporated into a variety of architectures.", "questions": "Could the authors more clearly distinguish their method from DEQ, which also repeatedly applies a function (here the recurrent block) and converges to a fixed point, with the number of function applications being controllable to achieve a desired accuracy? \n\nHow was the recurrent iteration distribution chosen? It would be interesting to see how changing this distribution changes the performance of the model. \n\nThe authors show in the top of Figure 2 that performance saturates relatively quickly with the number of recurrent steps K, with the earliest steps leading to the largest reduction in L2 error. However, for memory purposes, the authors use a fixed backpropagation window where only the last B steps are backpropagated through, with the earlier steps being treated as constant. Would it not make more sense to backpropagate through the earliest recurrent layers, given that the earliest ones are the the ones that lead to the largest reduction in L2 error?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The proposed framework is easily used in multiple different architectures, and that flexibility is a strong point.\n\nNo additional custom losses or tricks are required, and the authors provide a simple explanation of the algorithm, making adoption simple.\n\nThe authors demonstrate improved performance over baselines with reduced compute/parameter counts.", "weaknesses": "The authors say that repeated applications of the recurrent block lead encourage the recurrent block to contract toward a fixed point — what is the justification for this claim? Is there any theoretical proof that the recurrent blocks do indeed converge toward a fixed point?\n\nI would like to see some ablations on the initial latent distribution. The authors claim that the choice “primarily affects early iterations”. The authors also show that the early iterations are the ones that lead to the largest reduction in L2 error and are the most “important” in this sense, and so it would be interesting to see whether the choice of the initial latent distribution makes a big different in terms of overall performance of this method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S8lYQmH7Wa", "reviewer": "Reviewer_N727", "summary": "This paper introduces the Recurrent-Depth Simulator (RecurrSim), an architecture-agnostic framework designed to give neural simulators explicit, test-time control over their accuracy-cost trade-off. This capability is standard in classical numerical methods but largely absent in modern deep learning-based simulators.\nThe core idea is to replace a fixed-depth network with a recurrent block that is iterated a user-specified number of times (K) at inference. The model is trained by sampling K from a distribution and using truncated backpropagation-through-depth to maintain a fixed memory footprint. The authors demonstrate RecurrSim's effectiveness on a wide range of benchmarks and show that it can be applied to various backbones, consistently outperforming standard architectures and other adaptive-compute models.", "questions": "1. Minor comment Line 1215 Optimization typo. Can you correct?\n2. You justify using truncated backpropagation-through-depth as a way to bound memory, which provides an approximate gradient. Could you elaborate on why this was chosen over gradient checkpointing, a standard alternative that computes the exact full-depth gradient by trading compute for memory?\n3. The ICLR guidelines strongly encourage a dedicated 'Reproducibility Statement' paragraph to help reviewers locate the relevant details. While the appendices provide excellent, comprehensive details for reproducibility, this specific statement is missing. Would the authors be willing to add this paragraph in the final version to improve clarity for future readers?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Sufficient Novelty: The paper addresses a critical and practical problem in scientific machine learning. While the core mechanism (using recurrent iterations for an accuracy-cost trade-off) has been explored in other domains like computer vision and natural language processing, this paper's novelty lies in Its application and validation for the neural simulator domain, where this feature is a standard expectation from classical solvers but has been a major missing piece for deep learning methods.\n2. Methodological Simplicity and Generality: The RecurrSim framework is \"plug-and-play.\" It requires minimal code changes to a standard architecture – like LoRa methods. The paper strongly supports its \"architecture-agnostic\" claim by successfully applying it to FNO, ViT, and UPT.\n3. Strong comparison with other baselines and in a wide range of PDE problems.\n4. Scalability and Efficiency: The results on 3D CNS are impressive (a 0.8B param RecurrFNO outperforms a 1.6B param FNO with 13.5% less training memory). \n5. High-Quality Presentation: The paper is exceptionally clear, well-structured, and easy to follow. The appendices provide strong justifications for design choices.", "weaknesses": "1. Lack of a Dedicated Reproducibility Section recommended in author guidelines (although appendix provides enough)\n2. Insufficient Justification for truncated backpropagation-through-depth: The authors propose truncated backpropagation-through-depth to bound memory. However, they fail to discuss or compare this to gradient checkpointing, a standard alternative. Gradient checkpointing would compute the exact full-depth gradient (trading compute for memory) instead of the approximate gradient from truncated backpropagation-through-depth. The paper provides no justification for why an approximate gradient is sufficient or preferable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KVbsLhxauN", "reviewer": "Reviewer_Pk3g", "summary": "This paper proposes an architecture-agnostic framework, the Recurrent-Depth Simulator. During the training phase, the framework randomly samples the number of recurrent iterations K from a distribution and optimizes using truncated backpropagation; during the test phase, users can explicitly specify the number of iterations K to trade off between computational cost and simulation accuracy. The authors validate this framework across multiple datasets, including Burgers, Korteweg-de Vries (KdV), Kuramoto-Sivashinsky (KS), high-dimensional Compressible Navier-Stokes (CNS), Active Matter, and ShapeNet-Car. The method is compared against other adaptive-compute models, such as FNO-DEQ, ACDM, and PDE-Refiner, as well as standard architectures such as FNO, ViT, and UPT. The paper concludes that RecurrSim offers a superior accuracy-cost trade-off curve compared to baselines. On the high-dimensional CNS task, a lower-parameter RecurrFNO variant outperforms a higher-parameter FNO baseline while also reducing training memory.", "questions": "See weaknesses", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The framework's core contribution is providing explicit test-time control, allowing users to flexibly trade computational cost for accuracy by adjusting the number of iterations. Compared to baselines, this method offers a smoother, more predictable trade-off curve, avoiding the early saturation or erratic behavior seen in alternatives.\n- The framework achieves excellent parameter efficiency through weight-sharing, enabling it to match or exceed larger baseline models with significantly fewer parameters and lower training memory consumption.\n- The method is a plug-and-play, architecture-agnostic framework, and its generality has been validated across diverse backbones, including FNO, ViT, and UPT", "weaknesses": "- The core mechanism of this work,a recurrent-depth block trained with truncated backpropagation, is conceptually very similar to a standard Recurrent Neural Network (RNN), making the contribution potentially incremental as it applies existing techniques to a new domain.\n\n- The paper is lacking in visual comparisons. The authors didn't provide corresponding visualizations for baselines like FNO-DEQ or ACDM, making it difficult to visually assess differences in physical fidelity. Also, none of the cases are provided with range.\n\n- The paper suffers from several typographical errors and unclear phrasings. In particular, the descriptions of some experimental setups (e.g., Section 4.3 ) are brief, which may create difficulties for readers attempting to reproduce the results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tKbjnPgI96", "reviewer": "Reviewer_YvLk", "summary": "The paper introduce a new procedure for training any block of neural architectures when learning solutions of PDEs. This procedure consists of incorporating recurrent calls to the block, whose number is controlled by a parameter $K$. The parameter $K$ changes during the training to make the obtained reccurent network able to learn the solution for any $K$, with the intuition that the approximation will be more accurate for high $K$ than for low $K$. As a result, it is possible to tune the accuracy-cost trade-off at test time by toggling $K$. The approach is validated on several benchmark and for several underlying neural architectures.", "questions": "Are there practical limitations that prevented the authors to apply UPT, ViT and FNO and their Recurr variants to all three high dimensional datasets?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper is well written, easy and pleasant to follow\n- The idea is simple, original, and represents a clever approach for adding an inductive bias towards physical solvers in the obtained neural network together with controlling the cost-accuracy trade-off. \n- The approach is thoroughly validated on small to large scale physical learning problem, and its applicability to different existing SOTA architectures is demonstrated (RecurrFNO, RecurrVIT, RecurrUPT).", "weaknesses": "- The benchmark would benefit from a more systematic evaluation of UPT, ViT and FNO, i.e applying those tree models and their Recurr variants to all three high dimensional datasets.\n- The high dimensional benchmark lacks a study on the effect of $K$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)