Diffusion-Based Hierarchical Graph Neural Networks for Simulating Nonlinear Solid Mechanics
Authors: Tobias Würth
Niklas Freymuth
Gerhard Neumann
Luise Kärger
Venue: NeurIPS 2025 spotlight ()
Keywords: Neural PDE Solvers
Diffusion Models
Graph Neural Networks
Learned Simulation
AI4Science
Primary area: machine_learning_for_sciences
TL;DR: We combine denosing diffusion probabilistic models and hierarchical graph neural networks to autoregressively simulate physical dynamics on unstructured meshes.

Abstract: Graph-based learned simulators have emerged as a promising approach for simulating physical systems on unstructured meshes, offering speed and generalization across diverse geometries. However, they often struggle with capturing global phenomena, such as bending or long-range correlations usually occurring in solid mechanics, and suffer from error accumulation over long rollouts due to their reliance on local message passing and direct next-step prediction. We address these limitations by introducing the Rolling Diffusion-Batched Inference Network (ROBIN), a novel learned simulator that integrates two key innovations: (i) Rolling Diffusion-Batched Inference (ROBI), a parallelized inference scheme that amortizes the cost of diffusion-based refinement across physical time steps by overlapping denoising steps across a temporal window. (ii) A Hierarchical Graph Neural Network built on algebraic multigrid coarsening, enabling multiscale message passing across different mesh resolutions. This architecture, implemented via Algebraic-hierarchical Message Passing Networks, captures both fine-scale local dynamics and global structural effects critical for phenomena like beam bending or multi-body contact. We validate ROBIN on challenging 2D and 3D solid mechanics benchmarks involving geometric, material, and contact nonlinearities. ROBIN achieves state-of-the-art accuracy on all tasks, substantially outperforming existing next-step learned simulators while reducing inference time by up to an order of magnitude compared to standard diffusion simulators.

Diffusion-Based Hierarchical Graph Neural Networks for Simulating Nonlinear Solid Mechanics
Tobias Würth 1+ Niklas Freymuth² Gerhard Neumann Luise Kärger' Institute of Vehicle System Technology, Karlsruhe Institute of Technology, Karlsruhe 2 AAtonnomous Learning Robots, Karlsruhe Institute of Technology. Karlsruhe
Abstract
Graph-based learned simulators have emerged as a promising approach for simu- lating physical systems unstructured meshes, offering speed and generalization across diverse geometries. However, they often struggle with capturing global phenomena, such bending long-range correlations usually occurring solid mechanics, and suffer from error accumulation over long rollouts due to their reliance on local message passing and direct next-step prediction. We address these limitations by introducing the Rolling Diffusion-Batched Inference Net- work (ROBIN), novel learned simulator that integrates two key innovations: (i) Rolling Diffusion- -atthhhe Inference (ROBI), parallelized inference scheme refinement across physical time steps
that amortizes the cost of diffusion- by overlapping denoising steps across temporal window. (ii) A Hierarchical Graph Neural Network built on algebraic multigrid coarsening, enabling multiscale message passing across different mesh resolutions. This architecture, implemented via Algebraic-hieree hierarchical Message Passing Networks, captures both fine-scale local dynamics and global structural effects critical for phenomena like beam bend- ing or multi-body contact. We validate ROBIN on challenging 2D and 3D solid mechanics benchmarks involving geometric, material, and contact nonlinearities. ROBIN achieves state-of-the-ae art accuracy on all tasks, substantially outperforming existing next-step learned simulators while reducing inference time by up order of magnitude compared to standard diffusion simulators.
Introduction
Physical simulations enable many engineering and scientific fields gain quick insights into complex systems or to evaluate design decisions. Conventional simulations model the physical system using Partial Differential Equations (PDEs). Usually, the PDE is discretized by numerical methods, such as the Finite Element Method (FEM) (I). the Finite Volume Method (FVM) (2) or the Finite Difference Method (FDM) 33. This process reduces the need for cumbersome, resource-intensive real-world experiments. Recent research aims to speed up simulation with Machine Learning (MI.)-based models 5. These learned simulators promise to allow researchers and practitioners to evaluate large amounts of virtual, simulated experiments. These simulations in turn unlock applications in engineering design and manufacturing optimization [6
This work aims to improve learned simulators, focusing on simulations of nonlinear solid mechanics as a representative class of examples. We combine recent image-based Denoising Diffusion Probabilistic Models (DDPMs) II with Hierarchical Graph Neural Networks (HGNNs) [4] (cf. Figure [
While diffusion has shown promising results on images [15 audio [18] 19 and even policy learning for robotics (0) 21. it suffers from cost-intensive inference due to its iterative denoising procedure. We alleviate this high inference cost on time-dependent domains with Rolling Diffusion-
"correspondence tobias. smerth@kit.dd
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Batched Inference (ROBI), novel scheduling scheme that batches denoising steps of consecutive time steps. ROBI already starts denoising future prediction steps by using partially refined previous steps. This process reduces the number of model evaluations to the number of time steps and preserves the shift equivariance Markovian systems. ROBI only affects the inference process, allowing us to utilize conventional, parallelized DDPM training.
Noisy sample Auf Previous state ub-1
Noise sample A Reconstructed state ub
DDPM- Scheduler
Mosh Coarsening
Algebraic MPN
Predicted Velocity ve
Mesh Coarsening
Intra-Lovel-Messuge Passing Stack
Lnter-Level-Message Passing Stack
Overview of Rolling Ifffusion-Batcher Inference Network (ROBIN) prediction. ROBIN coarsens the fine mesh multiple times with Algebraic multigrid (AMG) to create a graph hier- archy. ROBIN predicts the denoising velocity Vo "AA k, up at time step / using Algebraic- hierarchical Message Passing Networks (AMPNs), given noisy residual sample Aui, the diffusion step k, and previous state uj - The prediction is used draw new noisy residual sample Aui and to update the state unj
Figure 1:
We combine ROBI with DDPMs and Aigebraichhirracche Message Passing Networks (AMPNs) to form the Rolling Seffuuiion-Bachhee Inference Network (ROBIN), which significantly acceler- ates simulation while improving predictive accuracy. ROBIN constitutes the first diffusion- based refiner for simulating physical dynamics on unstructured meshes, surpassing the current accuracy ceiling of HGNNs. We train ROBINs three challenging 2D and 3D solid mechanics datasets involving geometric, material, and contact nonlinearities. Across all datasets, ROBIN significantly improves over state-of-the-art mesh-based simulators [5 22 in terms of predictive accuracy. Lever- aging DDPMs, ROBIN accurately captures low-frequency global solution modes while resolving high-freyuency components. We further find that our proposed inference method, ROBI, speeds up diffusion- inference for learned simulations by up to an order of magnitude while maintaining accuracy. In addition, the AMPN architecture of ROBIN enables efficient transfer much larger meshes, while maintaining near-FEM accuracy
To summarize, i) propose ROBI, novel inference scheduling scheme for diffusion- based simulators that amortizes denoising across time steps. reducing inference to a single model evaluation per step while preserving time-shift equivariance; ii) introduce ROBIN, diffusion- based HGNN for nonlinear solid mechanics that combines multiscale message passing with ROBI to provide fast, accurate diffusion-based simulations; iii) demonstrate state-of-the-art performance challenging 2D and 3D benchmarks, outperforming existing simulators in both accuracy and runtime.
2 Related Work
Simulating Complex Physies. Simulating complex physical systems often requires numerical solvers, such as the FEM 23 24 1. the FVM (2), or the FDM 3 While accurate, numerical
solvers scale poorly with problem complexity, often requiring multiple hours or even days for single rollout on a modern workstation. Recent work shows that ML-based models are able to learn such
numerical simulations from data (25 26 27 9 22 10]. ML-based models provide speed-ups of one to two orders of magnitude while being fully differentiable, which accelerates downstream applications such as design 66) or manufacturing process optimization g Many learned simulators operate autoregressively. They mimic numerical solvers by using their own predictions to estimate
the residual between successive time steps [4. 5 28). Similarly, Neural ODEs 29 30 predict time derivatives and advance solutions via numerical integration. In contrast to these supervised approaches, Physics-Informa Neural Networks [31] directly operate on a PDE loss function to train Multilayer Perceptrons (MLPs) 32 33 8) Convolutional Neural Networks (CNNs) 34] 35
or Graph Neural Networks (GNNs) [6
37 Finally, Neural Operators aim to learn -ndeeedndeenddnnt
solution operators
38 40
27
41 ROBIN is autoregressive learned simulator that replaces direct
Prroject page, code and datasets available at https://tbwwrth.iihhhe github.io/ROBIN.

---

next-step prediction with multiple denoising diffusion steps, leveraging generative inference to improve prediction accuracy.
Learning Mesh-based Simulations with Graph Neural Network (GNN), Pfaff et al. [ introduced MEshGRAPHNETT Message Passing Network (MPN)-based simulator that encodes simulation states as graphs using mesh connectivity and physical proximity. While accurate for small problems, MESHIGAPPINET' (MGNs) does not scale well, as its receptive field is limited by the number of message-passing steps in the MPN. To address this issue, recent work expands the receptive field via global attention 42 4 or hierarchical mesh representations using Graph Convolutional Net- Extensions further improve efficiency and accuracy through rotation equivariance [4), hierarchical edge design [47] bi-stride pooling [48], and attention mechanisms
works 45 and MPNs 46 13.
acting on edges and across hierarchies [22] 49 leverages Adaptive Mesh Refinement (A (AMR) to -PH- NETS [37 integrate FEM-based training to improve accuracy and robustness. Existing methods are generally trained using a next-step Mean Squared Error (MSE) loss, which favors learning lower solution frequencies at the cost of accuracy in higher frequency bands that have less impact on the loss 9. However, autoregressive models trained with a MSE loss often overlook low-amplitude frequencies [9] Our approach is orthogonal, coupling hierarchical GNN with denoising diffusion models. This approach takes advantage of the large receptive field of multi-scale GNNs while pushing accuracy toward diffusion limits.
create mesh hierarchies for multi-scale GNNs. Complementary, Physics-Informen
Diffusion-based Simulations. Diffusion models have been applied to thysics-inffrmee image super-resolution [35 flow field reconstruction [50), and steady-state flow generation on grids using CNNs (5 and more recently to meshes with hierarchical GNNs [52]. These models, however, operate on isolated frames and do not capture time-dependent dynamics. In contrast, our model predicts deterministic physical evolution rather than equilibrium samples via autoregressive rollouts.
While next-step simulators trained with MSE loss capture high-amplitude, low-frequency components, they often miss low-amplitude components (9. PDERefs (PDE- address this via iterative refinement, improving ang-horizonn accuracy of grid grid-based CNN simulators. We extend this idea to unstructured meshes by combining algebraic mesh coarsening [53] [54] with hierarchical GNNs [52] [2] with shared layers. We further introduce a time-parallel denoising scheme at inference, removing the speed bottleneck while maintaining accuracy. Unlike diffusion- based CNN simulators that require K model evaluations per physical step [1] 11555 9. our method requires only single hierarchical GNN call per step post warm-up, reducing inference costs over T time steps from O(KT) to ((T). Compared to video-based approaches DI (56). which need to jointly process N steps and learn time-dependent denoiser with high memory cost, our model, ROBIN, leverages time-translation invariance to train sime-independent denoiser with only one time step in memory. As such, ROBIN can be applied autoregressively and can freely interpolate between fully parallel denoising and memory-efficien one-step denoising at inference time. It also predicts state residuals instead of states, significantly improving morizon rollout fidelity.
3 Rolling Diffusion--atched Inference Network (ROBIN)
Graph Network Simulators (GNSs) for Mesh-based Simulations. We consider solving PDEs for physical quantities u(x, t) that change over time t E j0, T) and inside a time-dependent domain x(t) E Q(t). We focus on simulations on meshes, where G = (V, (V,EM) denotes the mesh graph and the graph nodes V and the graph edges EM correspond to mesh nodes and mesh edges. We seek solutions u,(!)= u(x;, () at discrete node locations (!) E (2()). To obtain discretized PDEs, usually numerical methods, such as the FEM, are applied that define the discretization of spatial operators, such as gradients 0u(x, f)/Ox. Given the discretized operator F, the PDE simplifies to time-dependent Ordinary Differential Equation and requires solving du,/ = F(t, x,, u.) We can solve such systems using numerical time discretization schemes. In this work we use simple Euler forward discretization u uf + At F(t, xf. u²) and set At = 1. We extend PDE-Refinee [9] to Lagrangian systems, where the domain Q2' and node locations x E Q evolve over time. Here, we predict the solution uf at time step / by learning to reverse a probabilistic diffusion process [57] conditioned on the previous state of time step / The proposed methods also apply without any restriction to Eulerian systems, where the domains are time-independent.
3

---

3.1 Denoising Diffusion Probabilistic Models (DDPMs) for time-dependent simulations
Given a time-dependent solution u' from a data distribution q(u), the forward diffusion process is modeled by Markov chain, where Gaussian noise is added gradually to the sample uf at each diffusion step &
q(uj.x (u()
II piuifuug- k=1
a(ufluff == Muffi v1 Bauf Bx1)
M denotes the normal distribution and Br the noise specified by variance scheduler. We learn to reverse the diffusion process, i.e.,
po p [u() =- -N (i0--iiuo((ui (uf. k), E(uj, k:).
consisting of K iterative diffusion steps and starting from & = K. The mean μg depends on the prediction of a learned model with parameters 0. The covariance E is assumed be isotropic and
given as Σ = o = T-ax 3x)1 [5] with a; = Bg and 0g = I[ 04. We train the model to predict the denoising velocity, i.e., the v-prediction target
agg /1 ug
given gaussian noise e' 158 Since this target smoothly interpolates between -u' as ai z and e² (āx se 1), it emphasizes predicting the clean sample in early (high-noise) steps and the noise in later (high-signal) steps, simplifying learning. The model predicts the denoising velocity vo(ug, k) u The training objective is then defined as Bug,e',. .k ( k 1 v
vo uf. k ) autoregressively. conditioned the model on the last time step solution
9 To facilitate
faster denoising, we follow the DDPM formulation and use an exponential 3x scheduler.
a) Conventional Diffusion
b) Rolling Seffission-Bachhan Inference (ROBI)
One Step Denoising
Iterative Inference
Batched Denoising Step
Rolling-Batche Inference
Diffusion time &
Dilfusion time it
Model calls
Model calls
Simulation state
Prediction step
State updace
Physical time
Physical time
Physical time
Physical time
Figure 2: Overview of a) conventional autoregressive diffusion inference and b) ROBI. a) Conven- tional inference denoises the entire state physical time step once before it shifts to the next time step (see One Step Denoising). The Iterative Inference requires K model calls per time step, where K denotes the number of diffusion steps. b) ROBI parallelizes denoising steps across physical time, processing to K time steps batched, and reconstruct the physical states with the clean sample prediction subsequently (see Batched Denoising Step). This process allows Rolling-Buud Batched Inference after the initial warm-up. reducing the number of model calls to one per time step.
The first denoising step is defined such that oæ R O, which simplifies the v-prediction target to vt -u (cf. Equation (D).. The noisy sample u' e corresponds Gaussian Noise and is uninformative. Hence, for & K the model target converges to [[vo K, k, - as
[ K,,b-')) + 1b.l2 This MSE objective of the first denoising step le = K mirrors the training objective of one-step models (9, i.e., auto-regressive models that predict the solution of the next time directly with single prediction step. Hence, we expect similar accuracy as one-step models during the first diffusion steps. However, the DDPM- based model refines the initial prediction iteratively at each time step, improving the accuracy of the solution further. Note that, due to the noise scheduler,

---

each denoising step focuses on different amplitude and frequency levels of the solution 99. with later denoising steps increasingly paying attention higher frequencies.
Rolling Inffusion-Ratchenr Inference (ROBI. Conventional diffusion inference requires K model
calls, each corresponding to denoising step. per simulation time step 9 Figure 2 a) shows an
example. Thus, inference is roughly K times slower than one-step models [5] We propose Rolling Diffusion-Batcher Inference (ROBI) to accelerate inference in DDPM- -baedd autoregressive simula-
tors. Given velocity prediction V& (a k, ub l) at the denoising step kt reconstruct partially
refined prediction 1/1 V u vT vo.Volu k, ub 1) following [58]
As discussed in Section 3.1 early denoising steps behave similarly to one-step model predictions, while later steps progressively refine higher spatial frequencies (from coarse structures to fine details). After < K denoising steps time t, the intermediate estimate / already captures low-to-mid frequency content that is sufficient to condition the next physical step +++1 to predict a solution within this already refined frequency band.
ROBI exploits this property by starting the denoising of step (+1 soon as step has progressed by m steps. We initialize Kk 1+1 ~N(O,I) and denoise the batch of both time steps tue (t, t + 1]. More generally, after jm denoising steps, we initialize the time step (+j and denoise rolling time window with tne (t...... / j} in parallel. After a short warm-up. fully denoised samples (those with k=0) drop out and the window size becomes constant with w=K/m and Et, -t..,,,,+-1111 Notably, the denoising index k and thus the noise level increase along the window toward later physical times, aligning with the natural growth of forecast uncertainty. This reduces the number of model calls from KT (conventional inference) to K Km steps. For m=1, the number of model calls reduces to K+T, which is effectively the same complexity O(T) as for one-step models, when K < T. We refer to as the denoising stride. Figure b) visualizes the special case m=1 and K=3 diffusion steps. Each model call advances the simulation by one physical step and applies one denoising step to each of the K partially denoised predictions in the window.
single Batched Denoising Step (cf. Figure b) left for m=1 and K=3) of ROBI consists of two consecutive steps. In the Prediction step, the model outputs the velocity va(uf" kw u 1 for each element in the prediction window Et, ..., -+ with the diffusion indices kw E emm..... K}. Subsequently, we perform State Update step. Using the scheduler, we reconstruct upj.... to update po ( ). Both
the conditioning states u,",, and sample the reverse transition u used in the next Prediction Step as inputs. After m such Batched Denoising Steps, we advance in time and drop the fully denoised states (k=0) and initialize a new Gaussian sample (k=K) for the next time index t+w, so the subsequent call evaluates (u kw u(")..
For m=K, ROBI reduces to conventional autoregressive diffusion inference (cf. Figure 2 a)). Thus, the denoising stride m can be considered hyperparameter that trades off prediction accuracy and memory usage with inference speed. Most notably, in State Update, ROBI reconstructs the physical states, while the rolling time window is treated purely as batch dimension in the prediction model. Consequently, the model always predicts Vo (u(,, k) = ve(uğ, ki -p- i.e., conditioned only on the previous reconstructed state during both training and inference, which proves to be more stable and accurate for autoregressive ML-based simulations (5. Furthermore, this preserves the time-shift cquivariance of Markovian systems, fast training convergence and small GPU memory utilization.
In practice, we find predicting residuals Aui, improves accuracy. Let the model denoise a batch of residual states Atj" R Aujj of the time window (t,,.,,,,, -+ and denote the last
fully denoised state as u - We then recover clean states inside the window via a cumulative sum u6 +++j u + El Augti, j E {O,..., w - 1}. To further accelerate inference, we optionally stop denoising early at a truncation step kup and use the partially denoised state únjs, as the final prediction of the time step. This remains effective because early denoising steps already approximate one-step prediction.
3.2 Denoising Diffusion Probabilistic Models (DDPMs) for mesh-based simulations
To fully utilize Denoising Diffusion Probabilistic Models (DDPMs)'s potential for generating rich, multi-frequency solutions, prediction models must handle multi-scale information. Hierarchical Graph Neural Networks (HGNNs) are particularly well-suited for this, their architecture inherently
5

---

learns representations at varying levels of granularity, mirroring the diverse frequency content present in DDPM outputs. Leveraging this idea, we train HGNN predict the discrete denoising velocity V s k; u for mesh-based simulations on the mesh graph 9 EM) takes the current noisy sample uf and is conditioned on the previous clean sample u²,0
Root-node AMG-based Mesh Coarsening, We construct hierarchical mesh graph 9 = go.l. (V0:1, M) ) consisting of L+1 mesh graphs with nodes you and mesh edges g.. by coarsening the fine graph G := G L times. Coarsening is performed with root-node smoothed aggregation (53 implemented in [59]. The solver takes the fine-mesh adjacency (with self-loops) A as its system matrix and creates hierarchy of adjacency A0:L, upsampling (prolongation) U and downsam- pling (restriction) 10:L-1 matrices. In practice, our implementation only requires the fine adjacency A the root-node solver [59], and constructing the graphs of each level from the non-zeros of the returned matrices A The resulting nodes j E V of each level are always a subset of the fine mesh nodes. Unlike bi-stride pooling with Delaunay remeshing [22], this AMG-based approach better preserves mesh geometry by leveraging the strength of connections in the adjacency matrix. Figure in Section visualizes this difference.
We additionally define --1 downsampling edges ELD and upsampling edges gL. which connect nodes between successive levels g' and gi+3, We define these as the connections (non-zero values) of the given up- U0:L-1 and downsampling DU=L-1 matrices, resulting in an extended hierarchy graph n (y0:1, g.:LM U g0:L-1,D U g0:2-1,1). As before, found that the obtained pooling mappings respect the mesh geometry well, as shown in Figure [0 d) in Section
For contact experiments, we add contact edges go.c [22] to the graph hierarchy G In a simulation involving two colliding components, we define a bidirectional edge between the nodes of the first part and the nodes of the second part if their distance is less than the specified contact radius R. The resulting hierarchy given by Gft (VO:J, go.T.M U go.t.C g0.l- (0:
3.3 Algebraic-hierarchice. Message Passing Networks (AMPNs).
Encoder. Let g' be a hierarchical graph as defined above and u the noisy sample at denoising step k of simulation step t. We define node embeddings k; E yo:r, mesh edge embedding ex go. contact edge embeddings cf E E downsampling edge embeddings cb E C0:L-1.D and upsampling edge embeddings e E go:L-1.U, We add relative node distances x!j = x x5. and their norm |xfy to all edge embeddings and the initial distance x²: x² x4 and their norm XXy to mesh edges, down- and upsampling embeddings. Node embeddings include one-hot encoding n, of the node type. Node embeddings at level I = 0 additionally include u' A and task-specific features. All embeddings are projected to the latent dimension via linear layers. We add a Fourier encoding (57) for the denoising step & and the normalized level 1* L/L to inform the AMPN of relative graph depth.
Processor and Decoder. Similar to AMG solvers
54 53 and UNets 660
Algebraic-hierarchical
Message Passing Networks (AMPNs)) use a V-cycle to propagate information between levels. They consist of five core message passing modules: Pre-Process,ng, Downsampling. Solving, Upsampling and Procesrrcssing. as shown in Figure [ Pre-Processing, Solving and Post-Processing modules an Intra-Levve-Message Passing Stack (Intra-MP-Staad
consisting of N message passing steps to update the heterogeneous subgraph g' =0 =(V', _/.C U gl.M) of level l. Given node embeddings k, E VL, contact edge embeddings eç, E g/. and mesh edge embeddings eit E ELM, the message passing update of the level graph at step is defined by
Ca+11 Wn C.P + S0. (k, kj. 0eec eij
kj. e C.)) Wo.g oij (.en(k" kg,e M.n C.++1
M,tt1
6
(2)
a1 Way K. + fō,v ei
M,+ly
The operator + denotes permutation invaatiiinnnnniiainn aeciiiannnggggeggg aggregation, f MLPs and W weight matrices 446
52). Downsampling modules update the subgraph G!D (V++1 U U.! EI.) with an Inter-Level-
Message Passing Stack Cnter-MP-Stache of N message passing steps. The receiver embeddings

---

1 E Vi+1, the sender embeddings kyml E V" and the edge embeddings etj E ELD, Similarly, the upsampling layers update the embeddings of the subgraph gi (V++1 U Vd, E.. of level t. Here, the receiver embeddings are kjfe E V the sender embeddings kanu E yl+1, and the edge embeddings eij E gt.u, A message passing step an Intra-MP-Stace is defined as
W oi + (k, k, kenn, .iy).
k k Wo,v kee. + (6 coly cott)
(3)
Our V-cycle starts level with pre-processing and downsampling each level, repeated until the coarsest level | = L is reached. We then apply multiple message passing steps at level L, which has the largest receptive field. Next, upsample and post-process each level back up / 0. All Pre-Processing, Downsampling, Upsampling, and Dost-Processing modules share weights across levels. A final linear layer decodes the fine-level node embeddings k; 1 to produce the velocity prediction v co ( k; "
Experiments
Datasets. We evaluate our model on the three different datasets, namely BENDINGBEAM, IMPACT- PLATE [22] and DEFORMINGPLATE [5]. We introduce the BENDINGBEAM dataset (Figure 3)), featuring quasi-static, geometrically non-linear deformations of beams with high aspect ratios. The setup challenges models to capture global deformations via broad receptive fields and resolve high spatial frequencies due to locally thin, low-stiffness regions. In IMPACTPLATE (Figure b)), the models must Iearn flexible dynamics with varying material parameters and accurately resolve very localized deformation at the contact point. DEFORMINGPLAS LATE (Figure 3)) considers quasi-static contact simulations induced by scripted actuators that deform 3D plates consisting of nonlinear, hyperelastic material.
a) BendingBeam
ImpactPlate
DeformingPlarr
Figure 3: Example predictions ROBIN on the considered datasets. ROBIN predicts the part defor- mations well as the von Mises stress (color, yellow is high) on all experiments. a) BENDINGBEAM considers global part deformations of beams induced by local forces. b) In IMPACTPLATE, the models have to predict locally large deformations, caused by collision with an accelerated ball. e) The hyperelastic plates in DEFORMINGPLATE are deformed by scripted actuator.
Experimental setup. For all tasks, we target the displacement residual of the node positions with respect to the next time step Aut, x x during denoising. We additionally denoise the von Mises stress of Mises, i.0 directly without residuals to gain further insight into the dynamics of the three experiments. ROBIN uses K = 20 denoising steps and denoising stride of m 1 by default.. The task-specific features are specified Section C listed in Table We measure the prediction error using RMSE, as specified in Section C Section Clalso provides information about additional settings ROBIN, including training details and hyperparameters.
Baselines. We compare our model with three prominent baselines for nonlinear deformation simula- tions, namely MGNs, Hierarchical Contact Mesh Transformers (HCMTs), and Bi-Stride Multi-Scale GNNs (BSMSs). Detailed setups provided in Section D
7

---

Variants. We demonstrate that a single trained ROBIN can easily switch between different rollout modes by varying the denoising stride m and the truncation step krr We therefore evaluate multiple rollout settings (m/linr on the same trained model ROBIN model: the default (1/20), conventional autoregressive diffusion inference (20/20) (i.e., m=20 denoising steps per physical step), and an intermediate variant (5/20) with = 5. To study early stopping, we further reduce the truncation step kir with (1/10), (1/5), (1/3), (1/2) and (1/1).
Ablations. We ablate key components of ROBIN to assess their impact. 10 diffusion steps and 5 diffission steps train with reduced diffusion length K. W/o diffusion trains the AMPN as a one-step autoregressive model with MSE loss. W/o hierarchy disables hierarchical message passing, operating only on the fine mesh G State prediction replaces residual- based prediction with direct denoising of Uf,A instead of Au .. W/o shared layer uses 15 unshared message passing layers. HCMT model
replaces AMPNs by HCMTs as the hierarchical model for ROBIN. Section D provides additional implementation and training details.
Generalization to large meshes. To assess upscaling, we introduce BENDING with meshes average over ten times larger than in BENDINGBEAM. We compare trained ROBIN to training from scratch, showing rapid transfer to substantially larger meshes without architectural changes, enabled by shared blocks in the AMPNs that accommodate varying hierarchy depths.
a pre-
Results
Baselines. Figure a) compares the rollout RMSE of ROBIN against HCMT, MGN, and BSMS. ROBIN yields substantial error reductions for the prominent IMPACTPLATE and DEFORMING- PLATE datasets, and yields even larger improvement on BENDINGBEAM, Figure demonstrates that ROBIN is able to propagate local boundary conditions across the part for accurate predictions of the global part deformation. In addition, ROBIN resolves the non-linear bending curve of the FEM. This requires fusing geometric features across scales, e.g., global part dimensions together with local thin walls, for accurate prediction of the global part stiffness. All baselines struggle reproduce the FEM results, particularly the global deformation modes.
denoising stride / truncation step
ROBIN
MGN
1/10
1/5
1/1
HCMT
BSMS
O
1/20
5/20
20/20
Rollout RMSE (a + o)
Rollout RMSE
10
6 10
4 10
10-2
9
10-2
10a
102
Rollout Time [s]
BB
IP DP
Figure 4: Left: Rollout error measured by the RMSE. of the predicted nodes positions. ROBIN surpasses the accuracy of the baselines HCMT, MGN, and BSMS on all three datasets BENDING- BEAM, IMPACTPLATE and DEFORMINGPLATE Right: Comparison of inference time and error of ROBIN and its variants on BENDINGBEAM. The default variant of ROBIN (1/20) achieves the same accuracy as conventional diffusion inference (20/20), while the inference speed is close to the one step variant (1/1). Reducing the truncation step k. trades accuracy for speed.
Inference speed. Figure 4 b) plots rollout RMSE versus inference time on BENDINGBEAM, The default variant (1/20) of ROBIN is about an order of magnitude faster than conventional autoregressive diffusion inference (20/20), without compromising accuracy. Decreasing the truncation step kn (i.e., fewer diffusion steps per physical step at inference) trades rollout accuracy for speed. Nevertheless, the fastest variant (1/1), which can be seen as one-step model variant of ROBIN, still significantly outperforms the accuracy of the baselines (cf. Figure a)). Most notably, ROBIN (1/20) requires only Rs 58% more time than the one-step variant (1/1), despite performing K=20 denoising steps per time step, highlighting the efficiency of the parallel denoising scheme ROBI. Similar trends hold for IMPACTPLATE and DEFORMINGPLATE (cf. Figure II in Section
Figure 14 in Section
8

---

FEM
b) ROBIN
c) HCMT
d) MGN
e) BSMS
Figure 5:
Comparison of the predicted rollout deformations and von Mises stresses (color, yellow high) on BENDINGBEAM between a) the FEM, b) ROBIN, e) HCMT, d) MGN, and e) BSMS. ROBIN is able accurately reproduce the FEM results. Neither HCMT, MGN nor BSMS are able to resolve global deformation modes, illustrating the importance of the AMPN for global message propagation.
visualizes how high frequency errors accumulate over the rollout if we skip the later denosing steps, demonstrating the importance of reducing the high frequency errors for long rollouts despite low displacement RMSE.
Diffusion truncation. Figure visualizes displacement RMSE and fine-mesh edge- wise gradient RMSE [u, l22ll/x
denoising stride / truncation step
x9)1l2 against ground truth. Early diffu- sion steps primarily remove fow-frequency error (global RMSE drops), whereas later steps reduce high-frequency error (gradi- ent RMSE). Skipping the later denoising steps results in high frequency error accu- mulation and mesh degradation throughout the rollout (cf. Figure 14 in Section E). as observed in one step models (9. With m=1, ROBIN does not increase the gradi- ent error on BENDINGBEAM. We observe similar results on DEFORMINGPLATE and IMPACTPLATE, as illustrated in Figure 15 in Section F
20/20
1/20
1/5
1/2
5/20
1/10
1/3
1/1
RMSE (u +
10
10-2
Displacement
Gradient
Figure 6: Comparison of the predicted rollout displace- ments and displacement gradients on BENDINGBEAM for different truncation steps kn, While the first diffu- sion steps strongly decrease the displacement RMSE, the later steps are important to reduce the local displace- ment gradient RMSE. A fast inference with a low de- noising stride neither increases the displacement RMSE nor the gradient RMSE during a rollout.
Ablations. As shown in Figure 7 ablations confirm the importance of each ROBIN component. Reducing the diffusion length K slightly decreases accuracy across all datasets. However, even with K=5 dif- fusion steps, ROBIN remains significantly more accurate than all baselines. Using non-shared layers significantly degrades performance BENDINGBEEM, likely due reduced ef- fective receptive field. The large error increase for State prediction indicates that residual prediction is crucial for LAGRANGIAN simulations. Across all datasets, ROBIN outperforms the non-hierarchical variant, the non-diffusion variant, and the combination of DDPMs with the strongest hierarchical baseline (HCMT), demonstrating the synergy between DDPMs and AMPNs.
Generalization to large meshes. Fine-tuning the pre-trained ROBIN converges markedly faster and attains substantially lower RMSE than training from scratch, demonstrating mesh-size independence and efficient transfer to deeper AMG hierarchies with twelve times more nodes, without architectural changes. In Figure 8 pretrained predictions visually indistinguishable from FEM in thin, high-stress regions with large bending. The model trained from scratch exhibits negligible deformation due much slower convergence within the same training budget. ROBIN consistently requires 31 8 per case, while the numerical solver averages 108 and can require up to 4248 s, due to problem-dependent nonlinear convergence costs. Table 3 in Section lists the quantitative results.

---

ROBIN
diffusion steps
State prediction
w/o diffusion
10 diffusion sleps
w/o shared layer
w/o hierarchy
HCMT model
Rollout RMSE (/1 +
10-1
10-2
BendingBeam
Impact Plate
DeformingPlate
Figure 7: Rollout RMSE of the displacement predictions. ROBIN remains accurate, for very small number of diffusion steps. Shared layers are crucial for BENDINGBEAM to increase the receptive field. Replacing residual predictions with state predictions, hierarchical architectures with non-hierarchical architectures or with HCMT architectures, and diffusion with non-diffusion architectures significantly decrease the accuracy across all datasets.
a
FEM
b) Pre-trained
From scratch
Figure 8: Comparison of the predicted rollout deformations and von Mises stresses (color, yellow is between a) the FEM, b) fine-tuned ROBIN model, which was
large) on BENDING pre-trained on the small BENDINGBEAM dataset, and, c) ROBIN model trained from scratch for the same number of training iterations. The fine-tuned ROBIN closely matches the FEM deformation and stress distribution, capturing even local high-stress hotspots, whereas the from-scratch model underestimates deformation due to much slower convergence within the same training budget.
Conclusion
We introduced Rolling Diffusion Batched Inference Network (ROBIN), Liffusion-based HGNN that utilizes AMPNs to refine mesh-based predictions across scales. Leveraging the expressiveness of multiscale message passing and the accuracy of diffusion, ROBIN outperforms sfate-of-the-- simulators varied nonlinear solid mechanics tasks in terms of predictive accuracy. These tasks include novel BENDINGBEA dataset that reveals limitations current learned simulators. ROBI, ROBIN's inference scheme, parallelizes diffusion across time steps, reducing inference runtime by up to an order magnitude without sacrificing accuracy. We validated ROBIN on three challenging datasets, including the new BENDINGBEAM benchmark, and demonstrated significant gains in accuracy and efficiency. We discuss the broader impact of this work and our method in Appendix
biffusion-Batched
Limitations and Future Work. ROBIN currently does not possess SO(3) equivariance. Adding this property could improve the accuracy of prientation-sensitive predictions. We focus on DDPM. while other diffusion formulations or denoising schedules could provide valuable insights and further enhance performance. Similarly, our experiments cover nonlinear solid mechanies, and extensions to other domains, such as fluid dynamics, are promising direction. Lastly, ROBIN's combination of fast inference and high accuracy opens opportunities for accelerating multi-stage design and optimization workflows.
10

---

Acknowledgements
This work is part of the DFG AI Research Unit 5339 regarding the combination of physics-based simulation with Al-based methodologies for the fast maturation of manufacturing processes. The financial support by German Research Foundation (DFG, Deutsche Forschungsgemeinschatt) is grate- fully acknowledged. The authors acknowledge support by the state of Baden- Wen---trrrttrgggg through bwHPC, as well as the HoreKa supercomputer funded by the Ministry of Science, Research and the Arts Baden- Weiten--rrtebbbrrgg and by the German Federal Ministry of Education and Research. This work supported by the Helmholtz Association Initiative and Networking Fund on the HAICORE@ KIT partition.
References
[1] Mats G. Larson and Fredrik Bengzon. The Finite Element Method: Theory, Implemen- tation, and Applications, volume 10 of Texts in Computational Science and Engineering. Springer, Berlin, Heidelberg. 2013. ISBN 978-3-642-332864 978-3-642-33287-6 doi: 10.1007/978-3-64--338887
[2] F. Moukalled, L. Mangani, and M. Darwish. The Finite Volume Method in Computational Fluid Dynamics: An Advanced Introduction with OpenFOAM@ and Matlab, volume 113 of Fluid Mechanics and Its Applications. Springer International Publishing, Cham, 2016. ISBN doi: 10. 10.10078----3311 1619--677-6..
978-3-319-16873-9 378-3-319-16874-66
[3] M. Necati Özişik. Helcio R. B. Orlande, Marcelo J. Colaço, and Renato M. Cotta. Finite Difference Methods in Heat Transfer. CRC Press, July 2017. ISBN 778-1-351-34991-8
[4] Alvaro Sanchez- Sanzhez-Gonzalez,, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovee, and Peter Battaglia. Learning to Simulate Complex Physics with Graph Networks. In Proceedings of the 37th International Conference on Machine Learning, pages 8459-8468. PMLR, November 2020.
[5] Tobias Pfaff, Meire Fortunato, Alvaro Mesh-Based Simulation with Graph Networks. In International Conference Learning Representations, October 2020.
and Peter Battaglia. Learning
[6] Kelsey Allen, Tatiana Lopez-Guevara, Kimberly L. Stachenfeld, Alvaro Sanchez Gonzalez, Peter Battaglia, Jessica B. Hamrick, and Tobias Pfaff. Inverse Design for Fluid-Structure Interactions using Graph Network Simulators. Advances in Neural Information Processing Systems, 01113399-13774 December 2022.
[7] Clemens Zimmerling, Christian Poppe, Oliver Stein, and Luise Kärger. Optimisation of manufacturing process parameters for variable component geometries using reinforcement learning. Materials & Design, 21:: 10423,, February 2022. ISSN 02641275. doi: 10.1016/j.
[8] Tobias Würth, Constantin Krauß, Clemens Zimmerling, and Luise Kärger. Physics-informed neural networks for data-free surrogate modelling and engineering optimization An example from composite manufacturing. Materials & Design, 231:1 12034, July 2023. ISSN 02641275.
doi: 10.1016/j mattis. 11204..
[9] Phillip Lippe, Bas Veeling, Paris Perdikaris, Richard Turner, and Johannes Brandstetter. PDE- Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers. Advances in Neural Information Processing Systems,
December 2023.
[10] David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling Diffusion Models.
In Forty-First International Conference Machine Learning, June 2024.
[11] Georg Kohl, Liwei Chen, and Nils Thuerey. Benchmarking Autoregressive Conditional Dif- fusion Models for Turbulent Flow Simulation. In ICML. 2024 AI for Science Workshop, June 2024.

---

[12] Wenzhuo Liu, Mouadh Yagoubi, and Mare Schoenauer. Multi-resolution Graph Neural Net- works for PDE Approximation. In Igor Farkaš, Paolo Masulli, Sebastian Otte, and Stefan Wermter, editors, Artificial Neural Networks and Machine Learning ICANN 202/, pages 151-163, Cham, 2021. Springer International Publishing. ISBN 777-3-033-88365-- doi:
10.1007/978-3-03--6666
[13] Meire Fortunato, Tobias Pfaff, Peter Wirnsberger, Alexander Pritzel, and Peter Battaglia. MultiScale MeshGraphNees,, October 2022.
[14] Mario Lino, Stathi Fotiadis, Anil A. Bharath, and Chris D. Cantwell. Multi-scale rotation- equivariant graph neural networks for unsteady Eulerian fluid dynamies. Physics of Fluids, 34 (8)0881 10, August 2022. ISSN 1070-6631. doi:
[15] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
[16] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in nearal information processing systems, 33:6840-6851, 2020.
[17] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[18] Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2020.
[19] Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation. In International Conference on Learning Representations, 2020.
[20] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 02783649241273666, 2023,
[21] Paul Maria Scheikl, Nicolas Schreiber, Christoph Haas. Niklas Freymuth, Gerhard Neumann, Rudolf Lioutikov, and Franziska Mathis-Ullrie) Movement primitive diffusion: Learning gentle robotic manipulation of deformable objects. IEEE Robotics and Automation Letters, 2024.
[22] Youn- Yeol Yu, Jeongwhan Choi, Woojin Cho, Kookjin Lee, Nayong Kim, Kiseok Chang, ChangSeung Woo, Ilho Kim, Woo Lee, Joon Young Yang, Sooyoung Yoon, and Noseong Park. Learning Flexible Body Collision Dynamics with Hierarchical Contact Mesh Transformer. In The Twelfth International Conference Learning Representations, October 2023.
[23] Junuthula Narasimha Reddy. An Introduction the Finite Element Method, volume McGraw- Hill New York, 2005.
[24] Susanne C. Brenner and I.. Ridgway Scott. The Mathematical Theory of Finite Element Methods, volume 15 of Texts in Applied Mathematics. Springer New York, New York, NY, 2008. ISBN 978-0-387-75933-3 978-0-387--5934--0
doi:
[25] Xiaoxiao Guo, Wei Li, and Francesco lorio. Convolutional Neural Networks for Steady Flow Approximation. In Proceedings of the 22nd ACM SIGKDI) International Conference on Knowledge Discovery and Data Mining, pages 481-490, San Francisco California USA, August 2016. ACM. ISBN 978- 178-1-4503-42332- doi: 10. 1 00.114//999672.29999999
[26] Zichao Long, Yiping Lu, Xianzhong Ma, and Bin Dong. PDE-Net: Learning PDEs from Data. In Proceedings of the 35th International Conference on Machine Learning, pages 3208-3216.
PMLR, July 2018
[27] Johannes Brandstetter, Daniel E. Worrall, and Max Welling. Message Passing Neural PDE Solvers. In International Conference Learning Representations, October 2021.
12

---

[28] Jonas Linkerhägner, Niklas Freymuth, Paul Maria Scheikl, Franziska Mathis-Ullrich, and Gerhard Neumann. Grounding graph network simulators using physical sensor observations. arXiv preprini urXiv:2302.11864 2023.
[29] Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural Ordinary Differential Equations. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[30] Valerii lakovlev, Markus Heinonen, and Harri Lähdesmäki. Learning continuous-time PDEs from sparse data with graph neural networks. In International Conference on Learning Repre- sentations, October 2020.
[31] M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, Fobruurrrrr 2019. ISSN 00219991. doi: 10. jc...00...2000005555 10.045.
[32] Sina Amini Niaki, Ehsan Haghighat, Trevor Campbell, Anoush Poursartip. and Reza Vaziri. Physics-informed neural network for modelling the thermochemical curing process of composite- tool systems during manufacture. Computer Methods in Applied Mechanics and Engineering, 384: 1 13959, October 2021. ISSN 00457825. doi: 10.101 htt.:///.ma.2222 13959.
[33] Luning Sun, Han Gao, Shaowu Pan, and Jian-Xun Wang. Surrogate modeling for fluid flows based on in Applied Mechanics and Engineering, 361:112732, April 2020. ISSN 00457825. doi:
deep learning without simulation data. Computer Methods
10.10 (t01///ccma.209.......
[34] Han Gao, Luning Sun, and Jian-Xun Wang. PhyGeoNNe:: Physics-informed geometry-adaptee convolutional neural networks for solving parameterized steady-state PDEs on irregular domain. Journal of Computational Physics, 428:1 10079, March 2021. ISSN 00219991. doi: 10.1016/j. jcp.2020.1 10079.
[35] Dule Shu, Zijie Li, and Amir Barati Farimani. A physics-informed diffusion model for high- fidelity flow field reconstruction. Journal of Computational Physics, 478: 11 1972, April 2023. ISSN 0021-9991. doi: 10. j..0016/.epp.222311197.
[36] Han Gao, Matthew J. Zahr, and Jian-Xun Wang. Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems. Computer Methods in Applied Mechanics and Engineering. 390: 14502, February 2022. ISSN 0045-7825. doi: 10. 10.1016/j.cma.20000
14502.
[37] Tobias Würth, Niklas Freymuth, Clemens Zimmerling, Gerhard Neumann, and Luise Kärger. Physics-informed MeshGraphNets (PI-MGNs): Neural finite element solvers for non- non-stationary and nonlinear simulations on arbitrary meshes. Computer Methods in Applied Mechanics and Engineering, 429: 1 17102, September 2024. ISSN 0045-7825. doi: 10. 10.10161 mma..//.....22444444. 17102.
[38] Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier Neural Operator for Parametric Partial Differential Equations. In International Conference on Learning Representations October 2020.
[39] Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M. Benson. U-FNO-An enhanced Fourier neural operator-based based deep-learning model for multiphase flow. Advances in Water Resources, 163:104180, May 2022. ISSN 0309-1708. doi: 10.1016/j. advwatres.2022. 104180.
[40] Lu Lu, Pengzhan Jin, Guofei Pang. Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218-22) March 2021. ISSN 2522-5839. doi: 10.1038/
442256-021-003022
[41] Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. GNOT: General Neural Operator Transformer for Operator Learning.
13

---

[42] Steeven Janny, Aurélien Bénéteau, Madiha Nadri, Julie Digne, Nicolas Thome, and Christian Wolf. EAGLE: Large-scale scale Learning Turbulent Fluid Dynamics with Mesh Transformers. In The Eleventh International Conference on Learning Representations, September 2022.
[43] Xu Han, Han Gao, Tobias Pfaff, Jian-Xun Wang, and Li-Ping Liu. Predicting Physics in Mesh-reduced Space with Temporal Attention, May 2022.
[44] Benedikt Alkin, Andreas Fürst, Simon Schmid, Lukas Gruber, Markus Holzleitner, and Johannes Brandstetter. Universal Physics Transformers. CoRR, January 2024.
[45] Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M. Bronstein. Geometric Deep Learning on Graphs and Manifolds Using Mix- ture Model CNNs. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 5115-5124, 2017.
[46] Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzaled Vinicius Zam- baldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. Relational inductive biases, deep learning, and graph networks, October 2018.
[47] Artur Grigorev, Michael J. Black, and Otmar Hilliges. HOOD: Hierarchical Graphs for Gen- eralized Modelling of Clothing Dynamics. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 16965-16974, 2023.
[48] Yadi Cao, Menglei Chai, Minchen Li, and Chenfanfu Jiang. Efficient Learning of Mesh-Based Physical Simulation with Bi-Stride Multi-Scale Graph Neural Network. In Proceedings of the 40th International Conference on Machine Learning, pages 3541-3558. PMLR, July 2023.
[49] Roberto Perera and Vinamra Agrawal. Multiscale graph neural networks with adaptive mesh refinement for accelerating mesh-bases simulations. Computer Methods in Applied Mechanics and Engineering,
September 2024. ISSN 0045-7825. doi: 10.
117152.
[50] Tianyi Li, Alessandra S. Lanotte, Michele Buzzicotti, Fabio Bonaccorso, and Luca Biferale. Multi-Scale Reconstruction of Turbulent Rotating Flows with Generative Diffusion Models. Atmosphere, 15:60, December 2023. doi: 10.3390/atmos1501000
[51] Marten Lienen, David Lüdke, Jan Mansen-Palmuss and Stephan Günnemann. From Zero to Turbulence: Generative Modeling for 3D Flow Simulation. The Twelfth International Conference on Learning Representations, October 2023.
[52] Mario Lino Valenci,, Tobias Pfaff, and Nils Thuerey, Learning Distributions of Complex Fluid Simulations with Diffusion Graph Networks. In The Thirteenth International Conference Learning Representations, October 2024.
[53] Petr Vanek, Jan Mandel, and Marian Brezina. Algebraic multigrid by smoothed aggregation for second and fourth order elliptic problems. Computing, (7943):19-199) 1996.
[54] W. Ruge and K. Stüben. 4. Algebraic Multigrid. In Stephen F. McCormick, editor, Multigrid Methods, pages 73-130. Society for Industrial and Applied Mathematics, January 1987. ISBN 978-1-61197-188. 978- 978-1-61197-100-77
doi: 10.11337/.77811111710777
[55] Salva Rühling Cachay, Bo Zhao, Hailey Joren, and Rose Yu. DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting.
[56] Han Gao, Xu Han, Xiantao Fan, Luning Sun, Li-Ping Liu, Lian Duan, and Jian-Xun Wang. Bayesian conditional diffusion models for versatile spatiotemporal turbulence generation. Com- puter Methods Applied Mechanics and Engineering, 427:1 17023, July 2024. ISSN 0045-7825. doi: 10.1016/j.cma 2024. 117023.
14

---

[57] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, volume 33, pages 6840-6851. Curran Associates, Inc., 2020.
[58] Tim Salimans and Jonathan Ho. Progressive Distillation for Fast Sampling of Diffusion Models. In International Conference on Learning Representations, October 2021.
[59] Nathan Bell, Luke N. Olson, and Jacob Schroder. PyAMG: Algebraic multigrid solvers in python. Journal of Open Source Software, 7(LA-UR-23-2655)) 2022.
[60] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors, Medical Image Computing and Computer-Assisted Intervention MICCAI 2015, pages 234-241 Cham, 2015. Springer International Publishing. ISBN 978-3-
319-24574-4. doi:
[61] Tom Gustafsson and Geordie Drummond Mebain. scikit-fem: A python package for finite element assembly. Journal of Open Source Software, 5(52):2369, 2020.
[62] A. Paszke. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint 11X1v:1912.01703, 2019.
[63] Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization, January 2017.
[64] Stefan Elfwing, Eiji Uchibe, and Kenji Doya. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Networks, 107:3-11, November 2018. ISSN 08936080. doi: www..//..neume.2017......
[65] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthicu Devin, Sanjay Ghemawat, Geoffrey Irving, and Michael Isard. (TensorFlow): A system for (Large-Scale) machine learning. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI /6), pages 265-283, 2016.
[66] D. T. Lee and B. Schachter. Two algorithms for constructing Delaunay triangulation. International Journal of Computer & Information Sciences, 9(3::119-22 12219222,, June 1980. ISSN 0091-7036, 1573-7640. doi: 10. 10.1007/BF009977755
15

---

A Broader Impact
The ML-based simulator, Rolling Diffusion-Batches Inference Network (ROBIN), offers significant advantages for computational modeling and simulation. This is achieved by reducing computational costs while maintaining accuracy. This enables engineers to iterate through significantly more design variations or to quickly evaluate numerous scenarios using the fast model. However, like all powerful computational tools, there a risk of misuse, for instance, weapons development or unsustainable resource exploitation.
B Datasets
Table 1 provides an overview of the considered datasets in this work.
Table 1: Comparison of the datasets BENDINGBEAM, BENDINGBEAMLARGE IMPACTPLATE 22 and DEFORMINGPLATE (5] considered in this work. The column Nonlinearity distinguishes three different types: geometry (Geo), material (Mat), and boundary conditions (BC).
Datasets
Dynamic
Nonlinearity
avg.
Mesh Type
Steps T
Dim
# Nodes
BENDINGBEAM
Quasi-Static
Geo
744
Triangles
400
2D
BENDINGBEAMLARGE
Quasi-Static
Geo
8897
Triangles
100
2D
IMPACTPLATE
Dynamic
Geo, BC
2208
Triangles
52
2D
DEFORMINGPLATE
Quasi-Static
Geo, BC, Mat
1271
Tetrahedrons
400
3D
BENDINGBEAM. This dataset considers the bending of beam parts due to external forces. Bending one of the most basic deformation modes of parts structural mechanics. The dataset is designed as diagnostic benchmark for neural PDE solvers, addressing various potential bottlenecks. The force and handle boundary conditions are very local, only being defined on small subset mesh nodes (cf. Figure 9). However, the resulting deformations affect all nodes.
Figure 9: Visualization of the different node types the BENDING experiment. Blue nodes mark handle boundaries with fixed node positions. The red color indicates nodes where the force boundary condition is applied. Those locally applied forces in combination with the global, geometry- dependent part stiffness mainly determine the global deformation.
Hence, the neural PDE must effectively propagate local information to all nodes of the mesh. Next, the dataset considers beams with large aspect ratios. This results in large graph diameters, which represent the shortest path between the most distant nodes. The mesh resolution is increased at thin walls, which additionally increases the graph diameter. The local boundary conditions have to be transmitted across a large number of nodes, which challenges the ability to propagate messages globally. The geometry and especially the thin locations of the geometry strongly influence the global bending stiffness and deformation the part. Overall, the model has to output accurate solutions across various spatial frequencies.
The solutions are created with scikit-fem [1], iteratively solved using Newton Rapthon until the residual fell below a tolerance of 10- Each simulation is solved for total number of 400 time steps. We create a total number of 1000 simulations for training. 100 for validation and 100 for testing.
To evaluate the generalization capability of ROBIN, we create an additional dataset variant BEND- INGBEAMLARGE, containing meshes more than times larger than those in BENDINGBEAM. with an average of 8897 nodes. One simulation in BENDINGBEAMLARGG contains 100 time steps.
C Setup
Hardware and Compute. We train all models on a single NVIDIA A100 GPU with maximum training time of 48 hours, while most models required approximately 40 hours. In total, trained
16

---

11 models across 3 datasets, each on 5 seeds: ROBIN, MGN, HCMT, BSMS, as well as 7 ablations of ROBIN. That amounts 40 hours 5 x 6600 hours training time. Furthermore, we trained two models on BENDINGBFAMLARGGE on seeds for approximately 40 hours, yielding a total duration of 40 hours x x 2 = 400 hours. For each training, we required comparable amount of time for development and hyperparmmeter tuning. Additionally, we conducted inference experiments to measure the inference speed for 4 models (ROBIN, MGN, HCMT, and BSMS) and 7 ROBIN variants across 5 different seeds. We run inference experiments on 3 datasets, with each experiment taking about 1 hour on average. In total obtain runtime of 1 hour 5 x 3 11 165 hours for the inference experiments.
pre-trained ROBIN model and a model from scratch, each
Training. We implement ROBIN in PyTorch [62] and train with ADAM 663]. We use an exponential learning rate decay. which decreases the learning rate from 1e - 4 to 1e 6 over the training time, including 1000 linearly increasing warm-up steps. We clip gradients such that their L2-norm doesn't exceed 1. We train ROBIN in BENDINGBEAM with 9M samples and in IMPACTPLATEE with 6M samples both with batch size of 16, resulting in 562,500 and 375,000 training iterations. In DEFORMINGPLATE we reduce the batch size to 12 and train for 300,000 iterations with 3.6M samples.
Features. Table 2 provides an overview of the used input and output features for ROBIN. In addition to the default features, we extend the node embeddings of BENDINGBEAM with the force residual A ABCC. which is defined by the boundary condition. In IMPACTPLATE, we add the density Ps and the Young's 's modulus Y; as node features. In DEFORMINGPLATE. we add the scripted displacement residual AuBC of the actuator. We normalize all input features based on the training dataset, setting them to have zero mean and unit variance. We add small amount of training noise 4 5 of 10-5 xx to the node positions xt, where we scale the noise level with the standard deviation of the
features xx- For IMPACTPLATE we noise the input history Aut (-1 x prevent overfitting on the history.
x (-1 with 10-3 0x to
Table 2: Node k; and edge embeddings cij for the different datasets, depending the node V and edge sets E.
Inputs Inputs VIL
Inputs gu.L.c
Inputs gDiL.
Outputs
Datasets
Inputs 1
gLLL.
yu.M
X x
BENDINGBEAM
n; Au;, k. A/BC
m
x
x
va,(Au(,))
nie Ant k. Autoo Pi, Yi
x
IMPACTPLATE
n
x xl..
Vi,((Au),)
DEFORMINGPLATE
n. Aut AUBc
x
Hierarchical Graph. Since the relative motion of the components in the considered experiments is not too large, we define the contact edges based on the initial mesh configuration and keep them fixed to maintain a constant graph. In DEFORMINGPLATE we set the contact radius to Rt = 0. connecting actuator nodes with plate nodes. In IMPACTPLATE we connect ball nodes and plate nodes with a radius of R 1.2. In all three experiments, create L 2 coarse layers to obtain 3 mesh levels.
Message Passing Networks. We use 3 Pre- and 3 Post processcns layers, Up- and 2 Downsampling layers and 5 Solving layers, which yields a total number of 15 learnable layers. We add a layer norm before each MI. and use two linear layers, a hidden size of 128 and Sigmoid Linear Unit (SiLU) 64 activation function. A max aggregation is used in all message passing layers.
gngebraic-hierarchine
Denoising Diffusion Probabilistic Models. We use K = 20 denoising steps and denoising stride of m = 5 for ROBIN by default. The 3 variances of the DDPM scheduler are geometrically spaced for training and inference, starting from a minimum noise variance of le - 4 (for 31) and going up to 1.0 (for 3x).
Metrics. To compare the rollout accuracy, we follow 22 and define the Root Mean Squared
Euclidean distance error RMSE = / VI/(N.N,) EN Σ 1 (ūg uzy)?, where the prediction ü.j
17

---

and the ground truth Way have N; nodes and N, features. We then calculate the mean over all time steps, the mean over the dataset and finally the mean j and standard deviation over the 5 seeds.
D Baselines, Ablations and Variants
implementaiion of the authors for the baselines for training HCMT and MGN with an exponential learning rate decay from 1e-4 to 1e-6, a batch size of and a hidden size of 128. We use
Baselines. We use the official TensorFlow
165
HCM1 [22 and MGN S. We use ADAM 63
15 tressage-passing steps for MGN on all datasets, as well as total of 15 Hierarchical Mesh Transformer (HMT) and Contact Mesh Transformer (CMT) layers for HCMT.
On BENDINGBEAM, we train HCMT for 4M training iterations with a training noise 4 of 0.001. We maximize the receptive field and set the number of mesh levels to 5, the maximum at which at least five nodes remain available across all meshes in the dataset, required for Delaunay remeshing [66]. This results in 9 HMTs. Since BENDINGBEAM a contact-free task, we replaced the dual-branch CMT by 6 single-branch Mesh Transformer layers that only attend to mesh edges instead of mesh and contact edges. We use the same architecture and hyperparameter for HCMT on IMPACTPLATTE and DRFORMINGPLATE as proposed by the authors [2 and train it for 3M steps and 2M steps, respectively.
We follow the authors' implementation and add world edges to the mesh graph of MGN instead of contact edges to increase the receptive field of the non-hierarchical architecture. MGN is trained for 3M iterations on BENDINGBEAM and uses a training noise of 0.001 with world edge radius of R = 0.13. On IMPACTPLATE we train MGN for 3M steps and use world edge radius of R 0.03 and a training noise of 0.003. We train MGN on DEFORMING authors' proposed settings [5] To prevent out-of-memory errors in edge cases on DEFORMINGPLLATE,, we restrict the number of world edges to 200,000 by selecting those with the smallest node distances.
for 1.5M steps and use the
For BSMS 48
the official PyTorch 62) implementattor of the authors. We follow the authors and use the maximum number of hierarchy levels possible to maximize the receptive field. We train BSMS on BENDINGBEAM with a batch size of 12 and 6M training samples, i.e. 500K training iterations, We use training noise of 0.001 and 5 mesh levels. On IMPACTPLATI we train BSMS with batch size of and 5M samples (625K training iterations), a training noise of 0.003, a contact radius of R = 0.4, and 7 hierarchy levels. We train BSMS on DEFORMINGPLATE with batch size of 8 and 6M samples (750K training iterations). a training noise of 0.003, a contact radius of R- = 0.03, and 6 hierarchy levels.
Ablations. The 10 diffusion steps and 5 diffusion steps ablation use the same settings as ROBIN, despite the reduced number of diffusion steps K. For the w/o hierarchy ablation we use the fine mesh graph G instead of the hierarchical graph gu:L, replace our AMPN by a single Intra-MP-Stack with 15 learnable message passing steps and remove the positional level encoding. In addition, we follow MGN and replace contact edges with world edges to increase the receptive field. To stay within the training budget, we reduce the number of training samples to 1.2M for BENDINGBEAM and to SM for IMPACTPLATE. For DEFORMINGPLATE we reduce the batch size to 1 and the training samples to 0.6M and also restrict the number of world edges to 200,000 as for MGN. The w/o diffusion ablation trains the AMPN with an MSE loss to predict directly the displacement residual Aut, and uses a step autoregressive rollout, such as HCMT, MGN, and BSMS. We use the same training noise settings as the baselines to stabilize the rollouts. The w/o shared layer ablation uses a total number of 15 non-shared learnable message passing layers distributed as follows: 1 Pre-Processing and Post- Proceescsssing layer per level, 1 Up- and Downsampling layer between each level, and 5 Solving layers. The faster predictions allow an increase in the number of training samples to 11M for BENDINGBEAM, to SM for IMPACTPLATE. and to 4.6M for DEFORMINGPLAYE the
For the HCMT model ablation, we replace the AMPNs with HCMTs. More specifically, same mesh hierarchy and the same model architecture as HCMTs. Everything else remains the same in ROBIN, including DDPMs and ROBI.
18

---

E Results
AMG-based mesh coarsening. Root-node AMG coarsening [3 preserves mesh geometry and connectivity (cf. Figure[TOp. The coarse mesh remains well-aligned with thin geometrical features, and smoothed transfer operators yield wider, algebraically informed receptive fields compared to bi-stride pooling. This fidelity is crucial for predicting geometrically nonlinear deformations.
a) Fine mesh
b) coosseid and Delaunay-remesiie
c) AMG-based coarsening
d) A.MG-based down- Nown-/upsampp /upsampling graphs.
Figure 10: Comparison of AMG-based mesh coarsening to Bi-stride-coarsening and Delaunay- remeshing (BSDL). a) the original, fine mesh. b) the mesh after two BSDL coarsening steps. e) the mesh after one AMG coarsening step. This mesh has approximately as many nodes as the mesh in (top) and level 2 (bottom). Bright blue
b). d) up- and downsampling edges from level indicates coarse nodes of level 1 (top) or 2 (bottom), respectively.
Inference speed. Each point in Figure corresponds rollout setting of ROBIN on IMPACT- PLATE and DEFORMINGPLATE. As for BENDINGBEAM in Section decreasing the truncation step reduces wall time while truncation step of k already sufficient obtain a higher accuracy than all baselines across all datasets (cf. Table The ROBIN default (1/20) is significantly faster than conventional diffusion inference and even slightly more accurate on IMPACTPLATE, which we attribute to anchoring low-frequency components and reducing drift.
denoising stride truncation step
denoising stride
truncation step
1/10
1/5
1/2
1/1
1/10
1/5
1/2
1/1
1/20
6/20
20/20
1/20
5/20
20/20
Rollout R.ME
Rollout RMSE
2.2 x 10
10 2
10-2
1.8 x 10
1.6 x 10
1.4 x 10-2
10-3
1.2 x 10
10'
10'
10a
Rollout Time [s]
Rollout Time [s]
(a) IMPACTPLATE.
(b) DEFORMINGPLATE.
Figure 11: RMSE rollout error and inference time of different inference variants of ROBIN on a) IMPACTPLATE and b) DEFORMINGPLATE. ROBIN, i.c., the variant (1/20), is most accurate on IMPACTPLANE and on par on DEFORMINGPLATE with the slower variants (5/20) and conventional inference (20/20). Reducing the truncation step kn increases speed while decreasing accuracy. The one step variant (1/1) achieves the largest speed-up on DEFORMINGPLATE, but also loses the most accuracy there.
19

---

Baselines. Figure 12 and Figure 13 visualize the rollout displacement and von Mises stress predic- tion of ROBIN, HCMT. MGN, and BSMS IMPACTPLATE and DEFORMINGPLATE, respectively.
a) FEM
b) ROBIN
HCMT
d) MGN
BSMS
Figure 12: Comparison of the rollout deformation prediction and von Mises stress prediction (color, yellow is high) on IMPACTPLATE to the ground truth of the FEM. ROBIN most accurately resolves the deformation the contact surface and the resulting stress. The deformation prediction of HCMT and BSMS are close to the FEM prediction, though stress is underestimated. MGN predicts accurately the global modes but exhibits local disturbances.
20

---

a) FEM
b) ROBIN
c) HCMT
d) MGN
BSMS
Figure 13: Rollout deformation and von Mises stress prediction (color, yellow is high) on DE- FORMINGPLATE of ROBIN, the baselines and the FEM. All models accurately reproduce the part deformation. HCMT slightly overestimates and BSMS slightly underestimates the stress at the thin wall between the hole and the boundary.
Diffusion truncation. The rollouts in Figure across truncation steps hr illustrate coarse-to- fine frequency behavior of ROBIN. Early steps capture the global, low-frequency shape, whereas additional steps sharpen high-frequency details, e.g., high stresses in thin geometrical features. Even with early truncation, ROBIN maintains coherent global modes. Longer schedules primarily refine local features and minimize the accumulation high-frequency errors, while retaining the global deformation pattern.
Figure 15| visualize the rollout displacement RMSE and displacement gradient RMSE of different ROBIN variants on DEFORMINGPLATE and IMPACTPLATE, As for BENDINGBEAM, early denoising steps critical for reducing global displacement error. Later diffusion steps focus on high-frequency solution components. On IMPACT PLATE, the gradient RMSE shows modest increase while ROBIN attains the lowest displacement RMSE overall. We hypothesize that partially denoising states (small m) stabilizes low-frequee wow-frequency components and reduces drift. In contrast, fully denoising (larger m) suppresses short-term, high- gighffreuuency error accumulation. However, this trade-off does not affect other datasets with significantly longer rollouts.
3 lists the quantitative results of experiments considered in this work and used for the visualizations.
Quantitative results. Table
21

---

a) Rollout time step t 100
Truncation stop
1.00
b) Rollout time step / = 200
Truncation step
Rollout tirue 200
c) Rollout time step t 400
Truncation step
400
20
Figure 14: Effect of diffusion truncation rollouts. Figures a), b), and c) show predicted defor- mations and von Mises stress prediction (color, yellow is high) at f = 100, t = 200, and t = 400, respectively. In each figure. rows correspond to truncation steps har = 3, 5. 10, 20. Using low truncation step kú increases inference speed and enables robust predictions the global deformation modes. However, it also causes local mesh degradation due to the accumulation of figh-frequency errors, as observed in MSE trained one step models (9].
denoising stride truncation step
denoising stride / truncation step
20/20
1/20
1/5
1/2
20/20
1/20
1/5
1/2
5/20
1/10
1/3
1/1
5/20
1/10
1/3
1/1
RMSE (p + σ
RMSE (1 +
10
10
10-2
Displacement
Gradient
Displacement
Gradient
(a) IMPACTPLATE.
(b) DEFORMINGPLATE..
Figure 15: Comparison of the rollout displacement RMSE and displacement gradient RMSE for different ROBIN setting (denoising stride m / truncation step kn) on a) IMPACTPLATE and b) DEFORMINGPI.NN Early diffusion steps substantially reduce the displacement RMSE and later steps the local displacement gradient RMSE. The fast inference setting (1/20) remains accurate and yields slightly lower displacement RMSE on IMPACTPLATE. but also slightly higher gradient RMSE.
22

---

Table 3: Quantitative results on BENDINGBEAM, IMPACTPLATE, DEFORMINGPLATE. and the large-mesh dataset BENDINGBEAMLARGE We report displacement RMSE [10 [10-3], gradient RMSE of the displacement field [10 [10- and rollout wall-clock time [s]. Values are mean std over the test set across 5 seeds (solver timings excluded). The row Numerical solving gives the average runtime of the high-fidelity solver (maximum parentheses). Baselines contrasts different models, Variants sweep the ROBI settings (denoising stride truncation step) and additionally report Gradient RMSE, Ablations remove ROBIN components, and Generalization to large meshes evaluates the upscaling capabilities of ROBIN.
BENDINGBEAM
IMPACTPLATE
DEFORMING PLATE
Numerical solving
Time [s]
Time [s]
Time [S]
46.2 (max. 186.1)
742.6 22
1157.2
Baselines
RMSE [10 [10-]]
Time [s]
RMSE [10 [10""]
Time [s]
RMSE [10 [10-")
Time [S]
ROBIN
29.00 1
15.03 1
12.33 1
4,94
4,98
61.60
HCMT
121.53 1.87
25.82 1: 0.28
19.57 1: 0.38
4.33 0.05
8.04 0.13
31.57 1: 0.14
+ 70.87
21.40 0.15
2.00
8.76
24.42
BSMS
141.98 7.96
4.70 0.04
63.52 + 32.36
2.16
13.59 = 5.21
13.28 = 0.07
Varlants
(denoising
stride/ /
truncation step)
RMSE [10 31
Time [s]
RMSE [10 3]
Time [s]
RMSE [10 3]
Time [s]
20/20
28.90 1 1.91
162.43
0.89
14.43 1 1.93
22.61 1: 0.36
4,96 1: 0.36
190.10
1.33
13.87 +:
4.83
0.35
-
1/20
29.00 + 1.00
15.03 + 0.04
12.33 = 0.96
4,94
0.01
0,33
61.60 = 0.29
1/10
30.35 1: 2.41
11.03
1: 0.10
12.38 1: 0.92
2.00
5.18 0.33
33.71 - 0.10
1/5
35.81
3.30
9.88 0.19
13.01
0.01
0.33
20.24 0.04
1/3
42.83 + 4,54
9.57 0.11
14.38
+ 1.40
L.55
6.42 0.23
14,85 = 0.11
9.47
7,26
1/1
64.23 1 11.86
9.49
1 0.12
0.12
21.46
1: 2.12
1,32 122.1.00 0.01
10.79
= 0.52
7,11 1: 0.04
Gradient RMSE [10
Gradient RMSE [10
[10->]
Gradient RMSB [10-
20/20
12.75 1: 0.70
1.92 0.07
28.98 1: 1.56
5/20
12.74 1: 0.65
1.89 0.08
28.83
1.57
1/20
12.77 + 0.72
2.55 + 0.33
29.10 - 1.50
1/10
37.58
0.30
1.76
1/5
94.00 + 20.90
3.68 + 0.37
73.13 + 5.64
0.56
22.33
1/2
200.44 + 128.29
8.61 + 1.02
150.33 = 48.48
1/1
324.05 285.65
23.41 1:
157.30 8.34
Ablations
RMSE [10 *|
RMSE [10 *]
RMSE [10 *]
ROBIN
29.00 +: 1.00
12.33 - 0.96
4.98 0.33
10 diff. steps 5 diff. steps shared layer State prodiction wio hierarchy
33.28
14,58
3,86
41.35 1: 6.27
15.60 - 1.74
6.40 0.56
66.05 + 7.36
12.88 +
5.65 0.26
596.38 + 70.93
130.50 = 30,80
137.61 = 30,54
122.38 1.52
57.02 + 3.48
12.04 1 0.24
wio diffusinn
43.83 1.16
99.47 47.30
9.78 2.12
HCMT model
111.05 1.15
29.18 = 1.43
14.86 = 0.42
BENDING ARRG
Numerical solving
Time [s]
108. 30 (max, 4248.01)
Generalization
large meshes
RMSE [10- [10-3]
Time [s]
trainre
77.59 1 4.97
30.94 1 0.63
12.14
1:
23

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in the abstract and introduction are fully supported by our method section, as well in the qualitative and quantitative results in the experiments section. The appendix provides more detailed results where required.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The conclusion discusses current limitations of the approach, including the scope of the paper and assumptions made.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
24

---

Answer:
[NA]
Justification:
The paper does not introduce any new theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We detail our method in Section and provide additional information in the appendix where required. We detail our experimental setup and datasets used in the experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
25

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We do not provide data and code at the time of submission, but will open-source both after acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We specify high-level training and test details in Section and provide further details in Appendices D and C
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report mean and standard deviation across five seeds for all main experi- ments. When evaluating the runtime-error tradeoff, we directly report information for all seeds without aggregation, explicitly showing the performance difference between runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
26

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide information on the compute resources for all experiments in
Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps:/neurips... cc http://www.pa..coooo/ot/cctcttca
Answer: [Yes]
Justification: We have read the NeurIPS Code of Ethics. We made sure that research complies to the Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., s if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We include discussion on broader impact in Appendix A
Guidelines:
The answer NA means that there is no societal impact of the work performed.
27

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: We do not use pretrained language models, image generators or similar high- risk models in our approach, and do not scrape datasets. We still discuss potential cases for miss-use uusss-ue our learned simulator in Appendix A
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We use two publicly available datasets, namely ImpactPlate and Deforming- Plate. We credit the original papers of both explicitly.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
28

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: At time of submission, we do not release new assets. After acceptance, we will open-source BendingBeam dataset, and provide appropriate documentation alongside it.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
29

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hatt:p//eearrpppcc//ooofeeeeeee/ /ct/////oeeeeeccc///2////.. for what should or should not be described.
30


Reviews:
[{"review_id": "wA8D4JHzeS", "reviewer": "Reviewer_bZkb", "summary": "The authors proposed a novel method, Rolling Diffusion Batched Inference (ROBI), that can accelerate inference in DDPM-based simulation, and then they combined it with a Hierarchical Graph Neural Network that combines multiscale message passing with ROBI to provide fast, accurate diffusion-based simulations.", "questions": "1. How is the computation cost compared to HGNN? Does it take longer or shorter to train？\n2. How does ROBI learn the parameter compared to conventional diffusion?", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The strengths of this paper are:\n1. The paper has a clear model structure and mathematical formula which reinforce the theoretical soundness and reliability of the proposed model.\n2. The model's novelty is good, although many people are using the Diffusion model, the authors improve it here.\n3. The paper features a detailed model structure that is easy to follow and understand, and the experiment is well-designed, clear, and straightforward.", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the thorough and favorable review, especially the recognition of our meticulous mathematical formulation and well-designed experimental setup. Below, we will briefly address the questions and concerns raised.\n\n> How is the computation cost compared to HGNN? Does it take longer or shorter to train？\n> \n\nWe agree that it is important to report the computation cost against the baselines. Previously, we only presented ROBIN runtimes for various diffusion inference settings in Figure 5. We report the rollout time   and RMSE of ROBIN and the baselines below. We also added an option to truncate the denoising process to speed up ROBIN’s inference further. In this setting, we stop after a chosen number of denoising steps, treating the reconstructed sample as the prediction and skipping the remaining steps. We report on variants that stop after the first (ROBIN - one step), second (ROBIN - two step), and the fifth (ROBIN - five step) steps.\n\n|  | **BendingBeam** |  | **DeformingPlate** |  | **ImpactPlate** |  |\n| --- | --- | --- | --- | --- | --- | --- |\n|  | Rollout time [s] | Rollout RMSE [$10^{-3}$] | Rollout time [s] | Rollout RMSE [$10^{-3}$] | Rollout time [s] | Rollout RMSE [$10^{-3}$] |\n| ROBIN - one step | 9.488 +/- 0.117 | 64.232 +/- 11.862 | 7.113 +/- 0.042 | 10.787 +/- 0.521 | 1.324 +/- 0.006 | 21.462 +/- 2.115 |\n| ROBIN - two step | 9.475 +/- 0.105 | 48.936 +/- 7.248  | 12.588 +/- 0.127 | 7.263 +/- 0.179 | 1.424 +/- 0.005 | 15.788 +/- 1.714 |\n| ROBIN - five step | 9.879 +/- 0.188 | 35.812 +/- 3.299 | 20.243 +/- 0.038 | 5.729 +/- 0.299 | 1.907 +/- 0.011 | 13.011 +/- 0.977 |\n| ROBIN | 15.026 +/- 0.040 | 29.002 +/- 1.997 | 61.601 +/- 0.291 | 4.978 +/- 0.332 | 4.943 +/- 0.010 | 12.330 +/- 0.958 |\n| MGN | 21.398 +/- 0.153 | 189.515 +/- 70.865 | 24.418 +/- 0.217 | 8.761 +/- 0.293 | 2.927 +/- 0.031 | 54.068 +/- 5.878 |\n| HCMT | 25.818 +/- 0.280 | 121.526 +/- 1.865 | 31.572 +/- 0.144 | 8.035 +/- 0.133 | 4.331 +/- 0.050 | 19.571 +/- 0.382 |\n\nA key advantage of ROBIN is that our AMPNs are trained as one step autoregressive models that are conditioned only on the previous time step. We observed no additional training time compared to conventional autoregressive models. All methods used a fixed two-day training budget.\n\nFurthermore, we conducted an upscaling experiment that demonstrated ROBIN’s architecture is independent of mesh size. This means that, for larger mesh sizes, the number of layers in our AMG-based hierarchy increases, and the AMPNs can handle them because of the shared model blocks. We fine-tuned the pre-trained ROBIN models using the new BendingBeamLarge dataset. While the BendingBeam dataset contains meshes with an average of 750 nodes, the large dataset contains an average of 9K nodes, with a maximum of 16K nodes. We trained both the pre-trained and untrained models for 750K training iterations with a batch size of $1$. We fine-tuned the pre-trained models with a learning rate of $10^{-6}$, and we used a learning rate decay ranging from $10^{-4}$ to $10^{-5}$ for the untrained models. \n\n|  | **BendingBeamLarge** |  |\n| --- | --- | --- |\n| ROBIN variant | Rollout time [s] | Rollout RMSE [$10^{-3}$] |\n| pre-trained | 30.936 +/- 0.627 | 78.929 +/- 4.971 |\n| untrained | 30.945 +/- 0.580 | 215.500 +/- 12.135 |\n\nPretrained models converge quickly on meshes with about twelve times more nodes, which highlights the ability of ROBIN to upscale and its training efficiency in fine tuning.\n\n> How does ROBI learn the parameter compared to conventional diffusion?\n> \n\nWe thank the reviewer for the insightful question. ROBI is an inference rollout scheme, and its training is identical to that of conventional diffusion-based inference. It does not have any additional parameters that need to be learned. \n\nROBI intuitively uses the insight that each diffusion step progressively predicts higher-frequency features to already start denoising future time steps before the current step has been fully denoised. Although this process does not reduce the number of required diffusion steps, it enables the parallel denoising of multiple steps. Figure 2 provides a schematic overview, and Figure 5 evaluates this effect. Empirically, we find that ROBI retains the accuracy of conventional diffusion inference while substantially speeding up the process on accelerated hardware. We will add a brief paragraph clarifying this relationship to the paper.\n\nWe want to thank the reviewer again for the helpful feedback. We hope that our clarifications and new results address the concerns raised. We encourage the reviewer to reach out to us during the discussion if there are further questions."}, {"review_id": "AbwfDtLckP", "reviewer": "Reviewer_GHwE", "summary": "Graph based simulators are widely used for the mesh based data modalities, however these models struggle with long range dependencies and correlations. Furthermore autoregressive rollouts leads to error accumulation which introduces another source of error. This paper introduces ROBIN a diffusion based learned simulator to address these issues. A rolling diffusion denoising scheme is introduced to accelerate the overall generation process by parallelizing the diffusion across time. Furthermore a hierarchical graph neural network built on multigrid coarsening is introduced to allow for multiscale message passing.", "questions": "1) Can authors provide a clearer picture of the novelty and comparison with existing autoregressive diffusion methods? It is not clear how the rollout denoising differs from others!\n\n2) If it is possible to put the dependence of the denoising between steps into equation it would become much more clear to the reader. Can authors formulate such equation (Even if it is a small inline equation)?\n\n3) Can authors clarify why a parallel generation is not possible in this concept? This way the complexity simply becomes O(T)!\n\n4) Why did the authors focus on a very limited area? Is it possible to expand this to other PDEs? This will help understanding the generalizability.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strength**\n1) The concept of a rollout diffusion inference and autoregressive diffusion is rapidly growing popularity for world models and other scenarios where actions and inputs are continuously introduced to the model. The rollout diffusion idea of the paper is interesting to see in the context of solid mechanics. \n\n2) The multi-resolution message passing opens doors to mixing simulations at different scales as well. This is something that can be explored by other researchers to have a generations informed from variety of scales. \n\n\n**Weakness**\n1) A major concern is the time parallel denoising scheme for inference. Authors mention that for generation of T frames (timesteps) with K denoising steps the time complexity will be O(KT). However the first diffusion models introduced for spatiotemporal data were not trained or used at inference with rollouts. These models (and most current video generative models) denoise all frames (timesteps) in parallel with an inflated UNet so the actual complexity would be O(K). It seems the paper uses the argument of efficiency throughout to develop the the rollout style denoising scheme to achieve O(K-m + mT).\n\n2) Paper provides the arguments to how they get to O(K-m + mT) in \"Rolling Diffusion-Batched Inference (ROBI)\" section however their claim in introduction is that they reduce the inference time from O(KT) to O(T)\n\n3) Novelty of the ROBI is unclear. How does ROBI differ from the current autoregressive diffusion frameworks. \n\n4) Provided experiments are limited. Given the claim of a new denoising scheme is introduced authors potentially should provide comparisons with other autoregressive diffusion frameworks (mostly in image or video generation). Otherwise there is a wide range of PDE problems that are more commonly used for benchmarking that can be used to compare methods with baselines. \n\n5) Baselines are old and and limited. There has been major researches focusing on PDE generations in recent years however the paper only compares to two baselines one of which is older than 5 years. MGN is commonly is not used as a standalone baseline and it helps complete an overall picture when also having other baselines.", "was_revised": true, "final_justification": "Authors provided further comparison with other frameworks demonstrating improved rollout. Furthermore the denoising scheme is novel in this context. I believe the experiments are done on a very specific subset of the PDEs, and generalizability is still of concern however given the novelties I believe the reasons to accept the paper are marginally more than the reasons to reject.", "rebuttal": "We thank the reviewer for their careful reading and constructive criticism. Below, we clarify our contributions and explain how they differ from existing work.\n\n> Can authors provide a clearer picture of the novelty and comparison with existing autoregressive diffusion methods? \n> \n\nROBIN is designed for deterministic physical simulations rather than perceptual video generation. It combines a diffusion model with hierarchical graph neural networks on unstructured meshes. This allows it to capture different solution frequencies and propagate information across large spatial distances and mesh resolutions. CNN-based video models cannot do this. In addition, ROBIN outputs residual predictions rather than absolute states, thereby improving accuracy and stability.\n\nROBIN advances the simulation from time $t−1$ to time $t,$ conditioned only on the previous clean state. This preserves causality and time shift equivariance while reducing memory usage and training time.  Sequence-based diffusion models that denoise entire histories often require learning warm-up and cool-down phases [1]. Conditioning on multiple past steps has been found to reduce the accuracy of learned simulators [2,3].\n\nOur model is trained using a v-prediction target, residual predictions and an exponential diffusion scheduler. It requires only 5 diffusion steps to surpass the baseline accuracy. During the rebuttal, we evaluated inference truncation. We stop after a chosen number of denoising steps and use the reconstructed sample as the prediction, which decreases inference time. The table below reports (over 5 seeds) on variants that stop denoising after the second denoising step (ROBIN - two step), along with the full ROBIN setting and the baselines. We added Bi-stride multi-scale GNNs (BSMS-GNNs) [4] as a third baseline on BendingBeam and will include results for DeformingPlate and ImpactPlate in the camera-ready version.\n\n|  | **BendingBeam** |  | **DeformingPlate** |  | **ImpactPlate** |  |\n| --- | --- | --- | --- | --- | --- | --- |\n|  | Rollout time [s] | Rollout RMSE [$10^{-3}$] | Rollout time [s] | Rollout RMSE [$10^{-3}$] | Rollout time [s] | Rollout RMSE [$10^{-3}$] |\n| ROBIN - two step | 9.475 +/- 0.105 | 48.936 +/- 7.248  | 12.588 +/- 0.127 | 7.263 +/- 0.179 | 1.424 +/- 0.005 | 15.788 +/- 1.714 |\n| ROBIN | 15.026 +/- 0.040 | 29.002 +/- 1.997 | 61.601 +/- 0.291 | 4.978 +/- 0.332 | 4.943 +/- 0.010 | 12.330 +/- 0.958 |\n| MGN | 21.398 +/- 0.153 | 189.515 +/- 70.865 | 24.418 +/- 0.217 | 8.761 +/- 0.293 | 2.927 +/- 0.031 | 54.068 +/- 5.878 |\n| HCMT | 25.818 +/- 0.280 | 121.526 +/- 1.865 | 31.572 +/- 0.144 | 8.035 +/- 0.133 | 4.331 +/- 0.050 | 19.571 +/- 0.382 |\n| BSMS-GNNs | 4.896 +/- 0.124 | 141.979 +/- 7.960 | - | - | - | - |\n\nROBIN outperforms state-of-the-art baselines with only two diffusion steps, whereas video diffusion models typically require many more. While extending ROBI to video models is an interesting future direction, our models are explicitly designed for simulation. We do not claim to have made any contributions toward overcoming the limitations of current image or video generation frameworks.\n\n> There has been major researches focusing on PDE generations in recent years however the paper only compares to two baselines one of which is older than 5 years.\n> \n\nAs noted above, we added BSMS‑GNNs [4] as another strong baseline. HCMT currently achieves the highest accuracy on benchmarks such as DeformingPlate, and MeshGraphNet (MGN), despite being five years old, remains competitive. This demonstrates the importance of ROBIN in surpassing the accuracy limits of solid-mechanics learned simulators.\n\n> Authors mention that for generation of T frames (timesteps) with K denoising steps the time complexity will be O(KT). […] Can authors clarify why a parallel generation is not possible in this concept?\n> \n\nROBIN generates trajectories autoregressively, predicting each step based on the previous one. Forecasting large time steps or an entire rollout at once reduces the accuracy of physical simulators [2,5]. The most accurate solid‑mechanics simulators, such as HCMT, are autoregressive. \n\nSince physical simulations are Markovian, conditioning only on the previous step is sufficient. Jointly denoising all time steps ignores the evolving state and violates causality. Even using more than one past step can reduce accuracy [2,3]. Additionally, parallel generation is less desirable because sequence models must retain many time steps and large meshes in memory, treating entire trajectories as single training samples. Our one step training uses single transitions, which improves memory and data efficiency while enabling large-mesh experiments.\n\nWe demonstrate that scalability on BendingBeamLarge, which contains 200 training, 20 validation and 20 test simulations of 100 time step and beam meshes with 5K-16K nodes. We fine-tuned a pre-trained model and compare it with a model trained from scratch (over 5 seeds). Both used 750K iterations and a batch size of 1. The pre-trained model is trained with a learning rate of $10^{-6}$ and the untrained with an exponential decay from $10^{-4}$ to $10^{-5}$.  \n\n|  | **BendingBeamLarge** |  |\n| --- | --- | --- |\n| ROBIN variant | Rollout time [s] | Rollout RMSE [$10^{-3}$] |\n| pre-trained | 30.936 +/- 0.627 | 78.929 +/- 4.971 |\n| untrained | 30.945 +/- 0.580 | 215.500 +/- 12.135 |\n\nPretraining substantially improves accuracy and enables adaptation to meshes with up to 16K nodes. \n\n> Paper provides the arguments to how they get to O(K-m + mT) in \"Rolling Diffusion-Batched Inference (ROBI)\" section however their claim in introduction is that they reduce the inference time from O(KT) to O(T)\n> \n\nROBI reduces inference costs from $O(KT)$ to $O(K-m + mT)$. Our experiments set $m=1$, yielding $O(K + T)$. Since time‑dependent simulations typically have many more time steps $T$ than the $K=5–20$ diffusion steps , this simplifies to  $O(T)$. We will clarify this assumption in the camera‑ready version.\n\n> If it is possible to put the dependence of the denoising between steps into equation it would become much more clear to the reader.\n> \n\nGiven a prediction horizon $h=K$, a batch of previous clean states $\\mathbf{u}^{t_j-1}\\_{0}$ and noise samples  $\\Delta\\mathbf{u}^{t_j}\\_{k_j}$ for the time steps $t_j = t+j$ and $j\\in\\{0,...,h-1\\}$ with increasing diffusion steps $k_j=1+j$, ROBIN outputs the diffusion velocities $\\mathbf{v}\\_\\theta(\\Delta\\mathbf{u}^{t_j}\\_{k_j}, k_j, \\mathbf{u}^{t_j-1}\\_0)$. \nWe approximate the clean samples $\\Delta\\tilde{\\mathbf{u}}^{t_j}\\_{0} \\approx \\Delta\\tilde{\\mathbf{u}}^{t_j}\\_{0|k_j} = \\sqrt{\\bar{\\alpha}\\_{k_j}} \\Delta\\mathbf{u}^{t_j}\\_{{k_j}} - \\sqrt{1-\\bar{\\alpha}\\_{k_j}} \\mathbf{v}\\_\\theta(\\Delta\\mathbf{u}^{t_j}\\_{{k_j}}, {k_j}, \\mathbf{u}^{t_j-1}\\_0)$  and the current noisy samples $\\\\Delta\\\\mathbf{u}^{t\\_j}\\_{k\\_j-1}=\\\\frac{\\\\sqrt{\\\\bar{\\\\alpha}\\_{k\\_j-1}}{\\\\beta}\\_{k\\_j}}{1-\\\\bar{\\\\alpha}\\_{k\\_j}}\\\\Delta\\\\tilde{\\\\mathbf{u}}^{t\\_j}\\_{0\\\\mid k\\_j} +\n\\\\frac{\\\\sqrt{\\\\alpha\\_{k\\_j}}(1-\\\\bar{\\\\alpha}\\_{k\\_j-1})}{1-\\\\bar{\\\\alpha}\\_{k\\_j}} \\\\Delta\\\\mathbf{u}^{t\\_j}\\_{k\\_j}  \n $ $ + \\\\sigma\\_{k\\_j} \\\\boldsymbol{\\\\varepsilon}^{t\\_j} $ with $ \\\\boldsymbol{\\\\varepsilon}^{t\\_j}\\\\sim\\\\mathcal{N}(0,I) $. Next, we can reconstruct the states of each time step using a cumulative sum over those approximated clean samples $\\\\tilde{\\\\mathbf{u}}^{{t_j}}\\_{0} = \\\\mathbf{u}^{t-1}\\_{0} + \\\\sum\\_{i=0}^j \\\\Delta \\\\tilde{\\\\mathbf{u}}^{t+i}\\_{0}$ with the last fully-denoised sample $\\\\mathbf{u}^{t-1}\\_{0}$. Finally, we can move one physical time step further, such that  $t_j = t+1+j$. Therefore, we remove the denoised time step $t$ from the batch, initialize the sample of the new time step ${t+h}$ as Gaussian noise $\\\\Delta\\\\mathbf{u}^{t+h}\\_{K} \\\\sim \\\\mathcal{N}(\\\\mathbf{0}, \\\\mathbf{I})$ and predict the next batch $\\\\mathbf{v}_\\\\theta(\\\\Delta\\\\mathbf{u}^{{t\\_j}}\\_{k\\_j}, {k\\_j}, \\\\tilde{\\\\mathbf{u}}^{{t\\_j}-1}\\_{0} = \\\\mathbf{u}^{t}\\_{0} + \\\\sum\\_{i=1}^j \\Delta \\\\tilde{\\\\mathbf{u}}^{t+i}\\_{0})$. Note that the proposed state reconstruction is done before the model call and allows the model to treat all time steps as a batch, where each prediction is only conditioned on the previous reconstructed state, e.g., $\\\\mathbf{v}\\_\\\\theta(\\\\Delta\\\\mathbf{u}^{t+2}\\_2, 2, \\\\tilde{\\\\mathbf{u}}^{t+1}\\_{0})$ for time step $j=1$.\n\n> Why did the authors focus on a very limited area? Is it possible to expand this to other PDEs?\n> \n\nThe present work focuses on solid mechanics, a significant engineering discipline encompassing manufacturing, structural mechanics, and lightweight design, where accurate learned simulation remains under-explored. Solid mechanics exhibit global dependencies and high solution frequencies, which present a challenge for learned simulators. However, ROBIN can handle both by effectively combining diffusion with hierarchical GNNs. Although this study is limited to solid mechanics, the method could be applied to other PDEs, including fluid dynamics. Diffusion and hierarchical CNN‑based simulators improve image‑based fluid simulations [1,2]. We plan to apply ROBIN to mesh-based fluid simulations in future work.\n\nWe thank the reviewer for the suggestions and hope that we have addressed all concerns. We will include the new results in the revised paper and welcome further questions during the discussion phase.\n\n[1] David Ruhe, et al. Rolling Diffusion Models. ICML, 2024.\n\n[2] Phillip Lippe, et al. PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers. NeurIPS, 2023.\n\n[3] Tobias Pfaff, et al. Learning Mesh-Based Simulation with Graph Networks. ICLR, 2020.\n\n[4] Yadi Cao, et al. Efficient learning of mesh-based physical simulation with bi-stride multi-scale graph neural network. PMLR, 2023.\n\n[5] Zongyi Li, et al. Learning Chaotic Dynamics in Dissipative Systems. NeurIPS, 2022.\n\n[6] Kiwhan Song, et al. History-Guided Video Diffusion. *arXiv*, 2025."}, {"review_id": "q3ObjoL60k", "reviewer": "Reviewer_WXuT", "summary": "This work proposes an auto-regressive hierarchical graph neural network and diffusion model based framework for modelling the dynamics of systems in solid-mechanics from a Lagrangian frame. The paper extends the work of the paper PDE-Refiner Lippe et al. 2023 with several key extensions. They extend onto problems on unstructured meshes with Lagrangian dynamics using a hierarchical GNN. The mesh hierarchy utilises a novel AMG based approach from classical FE meshing, which creates visually more balanced and appealing coarse meshes compared to recent ML baselines. The authors also introduce a strategy Rolling Diffusion Batched Inference (ROBI), that allows accelerated roll-out at inference time by initialising the denoising process of future states conditioned by \"early-denoising\" of preceding states. Experiments are performed on 3 dynamical problems from solid mechanics, baselined against 2 SOTA models MGN and HCMT. A sensitivity analysis of the ROBI procedure wrt to $m$ the denoising stride and $K$ the denoising steps and a model ablation study of key architecture components are also performed.", "questions": "No", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "This paper would be a straight accept for me if the quality of the write up was better but I had to spend quite some time to extract the details. In particular the clarity of the write up in sections 3.1 and 3.2 regarding ROBI and the HGNN could be improved, I list my concerns below to see they can be improved.\t\t\n\t\t\nIn section 3.1:\t\t\nRegarding the v-prediction target, the explanation was not clear. My eventual understanding is when k=K (first denoising steps) this is where we sample pure noise which is input to the model but also the target is closest to the 1 step direct target. Conversely when k~0 (latter denoising steps) the target is the remaining noise on the sample. So when the SNR is the highest the target is noise and when the SNR is the lowest the target is the signal. I found this quite counter intuitive and perhaps it can be clarified/reworded. Also, \"One step models\" are not defined, this confused me for a long time until I read PDE-Refiner paper.\n\n- There is some abiguity if there are also neural nets used to predict $\\mu$ and $\\Sigma$. But are these different to the main denoiser that predicts the denoising velocity and if so how are they trained?\n- There are no definitions of t_B/k_B/t_b/k_b\t\n- It was not immeadiate to derive the formula on line 148, it could be pointed out this comes from substituting $\\epsilon$ from the noising step into 1.\n- Clarifications on figure 3, what is meant by \"pooling\", as pooling is usually associated as a step within coarsening. In the caption does \"1 (top)\" refer to line c) ?\t\t\n\nIn the ROBI section, the writing would benefit from clarifying/reasserting that the network has 2 inputs, the t-1 condition and the current t sample at denoise level k, as this is non-standard in diffusion models, AR is a niche case. I now understand this is because the model takes in 2 arguments, previous step as condition and current noisey sample. It is the \"early-denoising\" of the preceding t-1 conditioning that gives a \"trade-off\" against accuracy when m<<K as it is utilised from a less developed state. Similarly, The sentence that starts \"Using these properties, we begin\", I thought for a long time the N(0,1) was a typo, given the prior sentence is motivating \"sufficient to condition the next step\".\n\nIn section 3.2:\t\t\nThe description of root-node AMG coarsening is also unclear, I would not be able to implement it from the description. Neither the rote-node or smoothing aspects from \"algebraic root-node-based smooth aggregation\" are explained. The description of upsampling/downsampling are not well explained \"Those matrices are constructed by smoothing the sparse initial aggregation mapping using the adjecency matrix\" I think I understand there is some kind of graph diffusion rewiring but providing more details would again strengthen the paper.\t\t\n\nI have some small evaluation concerns. No numerical figures are reported just bar charts that are hard to read and interpret. For the rollout in Figure 5b, I understand we compare this is a sensitivity study for different $m$ and $K$ but it is a slight red flag that ML baseline and even classic data generation times are not provided. Especially since this is a diffusion model where inference time is a known challenge.", "was_revised": true, "final_justification": "The provided clarifications and model inference timings have addressed my concerns.", "rebuttal": "> This paper would be a straight accept for me if the quality of the write up was better but I had to spend quite some time to extract the details. In particular the clarity of the write up in sections 3.1 and 3.2 […].\n> \n\nWe thank the reviewer for the positive feedback and for identifying areas that can be improved in our write-up. We will revise sections 3.1 and 3.2 for the final version. Below, we address the reviewer’s questions and clarify several ambiguities.\n\n> Regarding the v-prediction target, the explanation was not clear. […] I found this quite counter intuitive and perhaps it can be clarified/reworded.\n> \n\nThe v-prediction $\\mathbf{v}_k^t = \\sqrt{\\bar{\\alpha}_k} \\mathbf{\\epsilon}^t - \\sqrt{1-\\bar{\\alpha}_k} \\mathbf{u}^t_0$ indeed behaves in the opposite way to the SNR. At early denoising steps (low SNR, $\\bar{\\alpha}_k \\approx0$), the target behaves like the clean signal $\\mathbf{u}^t_0$. At later steps (high SNR, $\\bar{\\alpha}_k \\approx1$), the target behaves like the residual noise **$\\mathbf{\\epsilon}^t$.** Although this may seem counterintuitive, it simplifies the learning process. When the sample is good (high SNR) correcting the small residual is easier than regenerating the entire signal. When the sample is random (low SNR), the input contains little useful structure, so directly predicting the signal is easier than estimating the error. This reparameterization also keeps target magnitudes more uniform across timesteps.\n\n> Also, \"One step models\" are not defined, this confused me for a long time until I read PDE-Refiner paper. […] There are no definitions of t_B/k_B/t_b/k_b.\n> \n\nWe will introduce one-step models as autoregressive models that are trained to predict the solution for the next time step using the solution from the previous step. We will define $t_B$ and $k_B$ more clearly and use them consistently. \n\n> There is some abiguity if there are also neural nets used to predict $\\mu$ and $\\Sigma$. But are these different to the main denoiser that predicts the denoising velocity and if so how are they trained?\n> \n\nIn Denosing Diffusion Probalistic Models [1] we train a model to predict either the clean sample $u_{0}^t$, the noise $\\epsilon^t$ or, as in our case, the velocity $v^t$.  The Gaussian mean $\\mu_{\\theta}$ is than computed analytically from this output. The covariance matrix $\\Sigma_{\\theta}$ can also be learned by the same network, but we assume isotropic covariance $\\Sigma = \\sigma_k^2 \\mathbf{I}$.  \n\n> It was not immeadiate to derive the formula on line 148, it could be pointed out this comes from substituting from the noising step into 1.\n> \n\nGiven a clean sample $u_0^t$ and noise $\\epsilon^t$, the forward diffusion process produces at step $k$ a noisy sample $u_k^t=\\sqrt{\\bar\\alpha_k}\\,u_0^t+\\sqrt{1-\\bar\\alpha_k}\\,\\epsilon^t$ [1]. We eliminate $\\epsilon^t$ by substituting the v-prediction definition from Equation (1) $v_k^t=\\sqrt{\\bar\\alpha_k}\\,\\epsilon^t-\\sqrt{1-\\bar\\alpha_k}\\,u_0^t$ and solve for the clean sample $u_0^t=\\sqrt{\\bar\\alpha_k}\\,u_k^t-\\sqrt{1-\\bar\\alpha_k}\\,v_k^t$ [1]. Given a model prediction $v_\\theta(u_k^t,k,u_{0}^{t-1})$, we can reconstruct a clean sample $u_{0|k}^t=\\sqrt{\\bar\\alpha_k}\\,u_k^t-\\sqrt{1-\\bar\\alpha_k}\\,v_\\theta(u_k^t,k,u_{0}^{t-1})$ using this equation. \n\n> Clarifications on figure 3, what is meant by \"pooling\", as pooling is usually associated as a step within coarsening. In the caption does \"1 (top)\" refer to line c) ?\n> \n\nIn our AMG-based hierarchy, we create coarse meshes and build up- and downsampling graphs that connect nodes across levels. Figure 3 d) shows the edges between levels 0 and 1 at the top and between levels 1 and 2 at the bottom. All the nodes at the top correspond to level 0 (Figure a), while the subset of nodes marked in bright blue corresponds to level 1 (Figure c). Similarly, all the nodes in the bottom part of Figure 3d correspond to the nodes in Figure 3c. Since \"pooling\" is misleading, Figure 3 d) will be renamed \"AMG-based down- and upsampling graphs.”\n\n> I understand we compare this is a sensitivity study for different and but it is a slight red flag that ML baseline and even classic data generation times are not provided.\n> \n\nThe table below shows a comparison of Rollout RMSE between the baselines.\n\n|  | **BendingBeam** |  | **DeformingPlate** |  | **ImpactPlate** |  |\n| --- | --- | --- | --- | --- | --- | --- |\n|  | Rollout time [s] | Rollout RMSE [$10^{-3}$] | Rollout time [s] | Rollout RMSE [$10^{-3}$] | Rollout time [s] | Rollout RMSE [$10^{-3}$] |\n| MGN | 21.398 +/- 0.153 | 189.515 +/- 70.865 | 24.418 +/- 0.217 | 8.761 +/- 0.293 | 2.927 +/- 0.031 | 54.068 +/- 5.878 |\n| HCMT | 25.818 +/- 0.280 | 121.526 +/- 1.865 | 31.572 +/- 0.144 | 8.035 +/- 0.133 | 4.331 +/- 0.050 | 19.571 +/- 0.382 |\n| ROBIN | 15.026 +/- 0.040 | 29.002 +/- 1.997 | 61.601 +/- 0.291 | 4.978 +/- 0.332 | 4.943 +/- 0.010 | 12.330 +/- 0.958 |\n\nThe numerical solver required 46.2 s (max. 186.1 s) on average on BendingBeam, 1157.2s on DeformingPlate [2] and 742.6 s on ImpactPlate [3].\n\nWe also evaluated ROBIN on BendingBeamLarge (200 training, 20 validation and 20 test simulations), containing simulations of 100 time steps on beam meshes with 5–16K nodes. On average, the solver required 108.3 s (max. 4248.0 s) to complete. We compare the fine-tuning of a pre-trained model of BendingBeam to training from scratch. Both are trained for 750K iterations and a batch size of 1, while fine-tuning uses a learning rate of $10^{-6}$ and training from scratch an exponential decay from $10^{-4}$ to $10^{-5}$.  \n\n|  | **BendingBeamLarge** |  |\n| --- | --- | --- |\n| ROBIN variant | Rollout time [s] | Rollout RMSE [$10^{-3}$] |\n| pre-trained | 30.936 +/- 0.627 | 78.929 +/- 4.971 |\n| untrained | 30.945 +/- 0.580 | 215.500 +/- 12.135 |\n\nWe extended ROBI to include the option of truncating the denoising process after $K_I$ steps and using the partially denoised state as the final prediction. The table below shows for 5 seeds how different truncation levels $K_I$ compare to conventional inference and ROBIN (5 seeds). All evaluations are based on the same trained models.\n\n|  |  |  |  | **BendingBeam** |  | **DeformingPlate** |  | **ImpactPlate** |  |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| Inference variant |  $K$ | $K_I$ |  $m$ | Rollout time [s] | Rollout RMSE [$10^{-3}$] | Rollout time [s] | Rollout RMSE [$10^{-3}$] | Rollout time [s] | Rollout RMSE [$10^{-3}$] |\n| Conventional Inference | 20 | 20 | 20 | 162.426 +/- 0.891 | 28.896 +/- 1.906 | 190.100 +/- 1.333 | 4.960 +/- 0.355 | 22.607 +/- 0.358 | 14.431 +/- 1.933 |\n| ROBIN | 20 | 20 | 1 | 15.026 +/- 0.040 | 29.002 +/- 1.997 | 61.601 +/- 0.291 | 4.978 +/- 0.332 | 4.943 +/- 0.010 | 12.330 +/- 0.958 |\n| ROBIN - ten step | 20 | 10 | 1 | 11.030 +/- 0.099 | 30.349 +/- 2.409 | 33.705 +/- 0.098 | 5.181 +/- 0.347 | 2.896 +/- 0.009 | 12.379 +/- 0.923 |\n| ROBIN - five step | 20 | 5 | 1 | 9.879 +/- 0.188 | 35.812 +/- 3.299 | 20.243 +/- 0.038 | 5.729 +/- 0.299 | 1.907 +/- 0.011 | 13.011 +/- 0.977 |\n| ROBIN - two step | 20 | 2 | 1 | 9.475 +/- 0.105 | 48.936 +/- 7.248  | 12.588 +/- 0.127 | 7.263 +/- 0.179 | 1.424 +/- 0.005 | 15.788 +/- 1.714 |\n| ROBIN - one step | 20 | 1 | 1 | 9.488 +/- 0.117 | 64.232 +/- 11.862 | 7.113 +/- 0.042 | 10.787 +/- 0.521 | 1.324 +/- 0.006 | 21.462 +/- 2.115 |\n\n\n\n> In the ROBI section, the writing would benefit from clarifying/reasserting that the network has 2 inputs, the t-1 condition and the current t sample at denoise level k.\n> \n\nIn the revision, we will state explicitly that at level $k$ during inference the model takes the current noisy sample $u_{k}^{t}$ and the partially denoised state of $\\\\tilde{u}\\_{0|k}^{t-1}$ as input to predict the denoising velocity $v_\\\\theta(u\\_k^t,k,\\\\tilde{u}\\_{0|k}^{t-1})$. We will clarify that the refinement level of the reconstructed sample $\\\\tilde{u}_\\{0|k+1-m}^{t-1}$ increases with $m$.\n\n> The description of root-node AMG coarsening is also unclear […] but providing more details would again strengthen the paper.\n> \n\nWe use PyAMG’s ````rootnode_solver```` [4] to create the AMG hierarchy. The solver takes a sparse square matrix as input and returns a hierarchical cycle. We pass the fine-mesh adjacency matrix as the system matrix, and the solver yields coarsened adjacency matrices, corresponding coarse nodes, and up- and downsampling matrices. The nonzeros in these matrices define the edges of our up- and downsampling graphs.  Thus, the implementation requires only the fine-mesh adjacency, the ````rootnode_solver````  with its default settings, and graph construction based on the returned matrices (i.e., non-zero values).\n\nThe ````rootnode_solver```` itself aggregates nodes based on connectivity within the provided matrix and selects one root per aggregate. These roots become the next-level coarse nodes, and each fine node connects to its aggregate's root in the up- and downsampling matrices. These operators are then smoothed using the adjacency, effectively enlarging each coarse node’s receptive field beyond its aggregate. The implementation will be included in the code publication after the paper is accepted.\n\nWe thank the reviewer for their comments and for pointing out the ambiguities. We will incorporate the revsions into the camera-ready version, and we encourage the reviewer to contact us during the discussion.\n\n[1] Jonathan Ho, et al. Denoising diffusion probabilistic models. NeurIPS, 2020.\n\n[2] Tobias Pfaff, et al. Learning Mesh-Based Simulation with Graph Networks. ICLR, 2020.\n\n[3] Youn-Yeol Yu, et al. Learning Flexible Body Collision Dynamics with Hierarchical Contact Mesh Transformer. ICLR, 2023.\n\n[4] Nathan Bell, et al. PyAMG: Algebraic multigrid solvers in python. Journal of Open Source Software, 2022"}, {"review_id": "ruaT3Ja5vQ", "reviewer": "Reviewer_V9Hd", "summary": "The paper introduces a method for learned, GNN-based simulation targeted at solid mechanics applications. It introduces a novel rolling diffusion method and hierarchical message passing scheme which allows fast & accurate predictions.", "questions": "No major open questions. Overall I think this a paper worth accepting as this is an interesting approach which shows progress on a challenging problem. I would however encourage the authors to think about more thoroughly investigating the effect of some of the choices made in the paper, either as follow-up work if this paper is accepted, or a major revision of this paper if it is not.\nE.g. I'd love to see a proper review of different ways to perform HGNN, and how this affects different physical domains, from smooth simulations with long-range effects such as the ones studied here, to local chaotic CFD simulations. Similarly, a deep dive into the various choices in diffusion & denoising and the effect this has on consistency and frequency distribution could be very interesting.", "limitations": "yes, except for the points mentioned above.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Quasistatic elasticity simulations are challenging for GNN simulators; yet the presented approach does seem to be quite effective at them\n- All the absolutely necessary ablations are there (even though comparisons to other HGNN or diffusion approaches would have been nice)\n- This is a nice demonstration that rolling diffusion can be a useful approach for learned simulators\n- Overall, the paper is well written and easy to follow\n\nWeaknesses:\n- The main issue I see is that the paper splits its attention too much between rolling diffusion, HGNNs and applications in solid mechanics. The paper does the minimum diligence in all of these areas, but doesn't deliver any fundamental insights in either that might transfer to other work.\nE.g., there are so many HGNN papers with slightly different choices of graph construction and message propagation, and this paper adds yet another option without properly evaluating it against other options. Similarly, I feel the effect of conditioning on not fully denoised previous timesteps is underexplored. There is one ablation on this with downstream performance, but it would have been much more interesting to develop an understanding to what this does to simulation rollouts; e.g. does this affect coherence of high-frequency details?\n- The choice of experimental domain is a bit odd. Quasistatic elasticity is a good domain for studying hierarchical message passing, but less so for diffusion-- these simulations are very deterministic and smooth. So diffusion a) seems much less important here but also b) the tradeoffs in conditioning on not fully resolved diffusion might look very different in a less smooth domain. So it would have been good to also include a domain with more chaotic dynamics on multiple scales (e.g. CFD, weather, fracture,...).\n- (Minor) The description of ROBI and corresponding figure could be improved; it's a bit of a notation-fest, and the figure is hard to parse. It took me a few passes to understand which step is conditioned on what exactly in diffusion & simulation time.", "was_revised": true, "final_justification": "I think this is a promising paper, and I'm in favor of accepting it to NeurIPS. The authors addresses all the issues raised in review, and performed additional experiments which will further strengthen the paper.", "rebuttal": "We appreciate the positive comments of the author on the effectiveness of our approach for quasistatic elasticity, the inclusion of necessary ablations and the clarity of the paper. We address the concerns raised below.\n\n> The choice of experimental domain is a bit odd. Quasistatic elasticity is a good domain for studying hierarchical message passing, but less so for diffusion-- these simulations are very deterministic and smooth.\n> \n\nIt might seem counterintuitive, but we found that preserving smoothness is often difficult for HGNNs, which makes diffusion attractive in this domain. Models trained on an MSE loss must accurately capture global deformations, and even slight shifts or rotations can increase error. HGNNs often accumulate errors from high frequencies that are less penalized, which leads to local mesh distortions and drifting out of the training distribution over time. Non-smooth solutions result in significant deviations in the displacement gradient, which directly impacts stress.\n\nTo analyze this behavior, we added a truncation parameter to the ROBIN algorithm. This allows us to stop the denoising process early. For example, we can stop after the first $K_I = 5$ diffusion steps (instead of denoising all $K = 20$ steps) and treat the reconstructed clean sample as our ground truth. Regarding the rollout RMSE, we have evaluated ROBIN for different truncation valuess $K_I$. For each edge $(i,j)$ of the fine mesh, we have also evaluated the gradient  $||(\\\\mathbf{u}\\_{i} - \\\\mathbf{u}\\_{j})||_2 / ||(\\\\mathbf{x}\\_{i} - \\\\mathbf{x}\\_{j})||_2$ along that edge and calculated the RMSE. \n\n|  |  | **BendingBeam** |  | **DeformingPlate** |  | **ImpactPlate** |  |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n|  | $K_I$ | Rollout RMSE [$10^{-3}$] | Rollout gradient RMSE [$10^{-3}$] | Rollout RMSE [$10^{-3}$] | Rollout gradient RMSE [$10^{-3}$] | Rollout RMSE [$10^{-3}$] | Rollout gradient RMSE [$10^{-3}$] |\n| ROBIN - one step | 1 | 64.232 +/- 11.862 | 324.047 +/- 285.654 | 10.787 +/- 0.521 | 157.279 +/- 8.338 | 21.462 +/- 2.115 | 23.414 +/- 4.853 |\n| ROBIN - two step | 2 | 48.936 +/- 7.248  | 200.437 +/- 128.285 | 7.263 +/- 0.179 | 150.325 +/- 48.476 | 15.788 +/- 1.714 | 8.609 +/- 1.019 |\n| ROBIN - three step | 3 | 42.831 +/- 4.536 | 147.966 +/- 65.129 | 6.422 +/- 0.225 | 110.958 +/- 22.329 | 14.576 +/- 1.396 | 6.066 +/- 0.560 |\n| ROBIN - five step | 5 | 35.812 +/- 3.299 | 93.999 +/-  20.897 | 5.729 +/- 0.299 | 73.134 +/- 5.636 | 13.011 +/- 0.977 | 3.677 +/- 0.371 |\n| ROBIN - ten step | 10 | 30.349 +/- 2.409 | 37.582 +/- 4.711 | 5.181 +/- 0.347 | 41.587 +/- 1.758 | 12.379 +/- 0.923 | 2.610 +/- 0.316 |\n| ROBIN | 20 | 29.002 +/- 1.997 | 12.765 +/- 0.716 | 4.960 +/- 0.355 | 29.098 +/- 1.494 | 12.330 +/- 0.958 | 2.553 +/- 0.334 |\n\nThe initial diffusion steps concentrate on global solution frequencies, significantly reducing the global RMSE. Later steps focus on high frequencies, reducing local gradient error. Over time, these high-frequency errors can lead to mesh distortions, as observed in MSE-trained HGNNs.\n\n> Similarly, I feel the effect of conditioning on not fully denoised previous timesteps is underexplored. […] e.g. does this affect coherence of high-frequency details? […] Similarly, a deep dive into the various choices in diffusion & enoising and the effect this has on consistency and frequency distribution could be very interesting.\n> \n\nWe agree that investigating how conditioning partially denoised states affects the high-frequency component would be helpful. The table below shows how the denoising stride $m$ affects the rollout RMSE and rollout gradient RMSE for all the datasets.\n\n|  |  | **BendingBeam** |  | **DeformingPlate** |  | **ImpactPlate** |  |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Inference variants | $m$ | Rollout RMSE [$10^{-3}$] | Rollout gradient RMSE [$10^{-3}$] | Rollout RMSE [$10^{-3}$] | Rollout gradient RMSE [$10^{-3}$] | Rollout RMSE [$10^{-3}$] | Rollout gradient RMSE [$10^{-3}$] |\n| ROBIN | 1 | 29.002 +/- 1.997 | 12.765 +/- 0.716 | 4.960 +/- 0.355 | 29.098 +/- 1.494 | 12.330 +/- 0.958 | 2.553 +/- 0.334 |\n| Modest denoising stride | 5 | 28.987 +/- 1.932 | 12.741 +/- 0.653 | 4.916 +/- 0.346 | 28.828 +/- 1.566 | 13.865 +/- 1.900 | 1.894 +/- 0.084 |\n| Conventional Inference | 20 | 28.896 +/- 1.906 | 12.747 +/- 0.696 | 4.960 +/- 0.355 | 28.977 +/- 1.556 | 14.431 +/- 1.933 | 1.916 +/- 0.074 |\n\nROBIN with $m=1$, in which the previous physical time step is always denoised only one diffusion step further,  does not increase gradient error on BendingBeam or DeformingPlate. However, there is a modest increase on ImpactPlate, and ROBIN achieves the lowest RMSE overall. We hypothesize that partially denoising states keeps low-frequency components anchored and reduces drift. In contrast, fully denoising states  (larger $m$) reduces short-term, high-frequency error accumulation. However, this phenomenon does not seem to affect long rollouts. We plan to investigate this further in future work, as well as how it is influenced by alternative diffusion backbones and schedulers.\n\n> […] there are so many HGNN papers with slightly different choices of graph construction and message propagation, and this paper adds yet another option without properly evaluating it against other options.\n> \n\nWe acknowledge the many HGNNs in the literature and emphasize that our primary contribution is the synergy of diffusion with our AMPNs, not a new HGNN. Our AMPNs follow the architecture of algebraic multigrid methods to solve for frequencies at different scales. For example, we use an explicit solver layer at the coarse level to solve for global frequencies and shared intra-level message passing stacks for high frequencies w.r.t. the mesh scale of the level. We combine this approach with diffusion to specifically address all frequencies during training and inference. Our ablation studies highlight the synergy between the two. We lose accuracy on all tasks if we remove one of them, but we significantly outperform the baselines when they are used together.\n\nTo further demonstrate the benefits of combining our AMPNs with diffusion, we conducted an ablation study on the Bending Beam dataset, replacing our AMPNs with the HCMT architecture. Additionally, we compared our results to those of another hierarchical GNN baseline: the Bi-stride multiscale-Graph Neural Networks (BSMS-GNNs) [1]. \n\n|  | **BendingBeam** |\n| --- | --- |\n|  | Rollout RMSE [$10^{-3}$] |\n| ROBIN | 28.896 +/- 1.906 |\n| ROBIN - HCMT | 111.049 +/- 1.146 |\n| HCMT | 121.526 +/- 1.865 |\n| BSMS-GNNs | 141.979 +/- 7.960 |\n| MGN | 189.515 +/- 70.865 |\n\nAlthough integrating diffusion into HCMT improves RMSE compared to HCMT alone, ROBIN significantly outperforms both. BSMS-GNNs are nearly as accurate as the HCMT baseline, but significantly more accurate than non-hierarchical MeshGraphNets. Only the combination of diffusion and AMPNs can accurately resolve the wide frequency spectrum of BendingBeam.\n\nTo further demonstrate the advantages of the AMPN architecture, we have fine-tuned ROBIN on the BendingBeamLarge dataset. This dataset contains simulations spanning 100 time steps and meshes with up to 16K nodes. Although ROBIN was trained on BendingBeam with 3 hierarchies (~750 nodes), the shared blocks of the AMPNs enable direct application of ROBIN to BendingBeamLarge meshes with 4 levels. We fine-tuned ROBIN for 750K iterations with a batch size of 1 and a learning rate of $10^{-6}$. Similarly, we have trained an untrained ROBIN with the same settings, but with a learning rate decay ranging from $10^{-4}$ to $10^{-5}$. \n\n|  | **BendingBeamLarge** |  |\n| --- | --- | --- |\n| ROBIN variant | Rollout time [s] | Rollout RMSE [$10^{-3}$] |\n| pre-trained | 30.936 +/- 0.627 | 78.929 +/- 4.971 |\n| untrained | 30.945 +/- 0.580 | 215.500 +/- 12.135 |\n\nAs expected, the pre-trained ROBIN model predicts solutions with a much lower RMSE than an untrained model. The pre-trained model can be quickly adapted to a large-scale dataset without architectural changes, which highlights ROBIN’s generalizability to different mesh sizes.\n\n> (Minor) The description of ROBI and corresponding figure could be improved; […] It took me a few passes to understand which step is conditioned on what exactly in diffusion & simulation time.\n> \n\nWe thank the reviewer for pointing that out. We will clarify the description of ROBI, as well as Figure 2. In particular, we will clarify the conditioning process on the partially denoised, reconstructed state $\\tilde{u}_{0|k}^{t-1}$ and explain how the reconstruction process of that state depends on previous time steps.\n\nWe appreciate the positive and constructive feedback from the reviewer and will make the required updates in the final version. We are pleased to address any additional inquiries during the discussion phase.\n\n[1] Yadi Cao, et al. Efficient learning of mesh-based physical simulation with bi-stride multi-scale graph neural network. In International conference on machine learning. PMLR, 2023."}]

Author rebuttal:

Meta-review: This paper proposes an improved diffusion model based on HGNN to simulate solid mechanics, which can improve both the quality and the speed.

Strengths:
1. The method is novel and has many interesting ideas, even it is based on some existing work.
2. The results are quite convincing and cover a good range of solid mechanics.

Weakness:
1. The initial draft lacks some detailed discussions.

Rebuttal:
Both the authors and reviewers actively participated in discussion. The authors provided extensive new results. The reviewers were very positive of the rebuttal, and raised the points. 

Justification of recommendation:
A solid paper with novel ideas. Very strong rebuttal. Please include the new results and address the final comments in the final version.
Decision: Accept (spotlight)