SlowGuess/NeurIPS_2023_OCR huggingface dataset:
These tables show that CORAL remains competitive and robust on up-sampled inputs. Other baselines can also predict on denser grids, except for MP-PDE, which over-fitted the training grid.
C.2 Learning a Dynamics on Different Grids
To extend our work, we propose to study how robust is CORAL to changes in grids. In our classical setting, we keep the same grid for all trajectories in the training set and evaluate it on a new grid for the test set. Instead, here, both in train and test sets, each trajectory  $i$  has its own grid  $\mathcal{X}_i$ . Thus, we evaluate CORAL's capability to generalize to grids. We present the results in Table 8. Overall, coordinate-based methods generalize better over grids compared to operator based and discrete methods like DeepONet and MP-PDE which show better or equivalent performance when trained only on one grid. CORAL's performance is increased when trained on different grids; one possible reason is that CORAL overfits the training grid used for all trajectories in our classical setting.
Table 8: Learning dynamics on different grids - Test results in the extrapolation setting. Metrics in MSE.
Xtr↓ Xte	dataset →	Navier-Stokes	Shallow-Water
In-t	Out-t	In-t	Out-t
π = 20%	DeepONet	5.22E-1	5.00E-1	1.11E-2	1.12E-2
MP-PDE	6.11E-1	6.10E-1	6.80E-3	1.87E-2
irregular grid	DINO	1.30E-3	1.01E-2	4.12E-4	3.05E-3
CORAL	3.21E-4	3.03E-3	1.15E-4	7.75E-4
π = 5%	DeepONet	4.11E-1	4.38E-1	1.11E-2	1.12E-2
MP-PDE	8.15E-1	1.10E-1	1.22E-2	4.29E-2
irregular grid	DINO	1.26E-3	1.04E-2	3.89E-3	7.41E-3
CORAL	9.82E-4	9.71E-3	2.22e-3	4.89e-3
Table 9: Training time comparison - Expressed in days (d) or hours (h) on several datasets.
Model	Cylinder	Navier-Stokes	Shallow-Water	Elasticity	NACA
CORAL (INR)	6h	1d	5d	4h	2d
CORAL (Process)	1h	4h	1h	1h	1h
NodeMLP	0.5h	-	-	-	-
GraphSAGE	1d	-	-	-	-
MP-PDE	7h	19h	21h	-	-
DINO	-	8h	2d	-	-
DeepONet	-	6h	5h	-	-
FNO	-	8h	6h	0.5h	0.5h
UNet	-	-	-	0.5h	0.5h
Geo-FNO	-	-	-	1.0h	1.0h
Factorized-FNO	-	-	-	1.0h	1.0h
C.3 Training Time
In Table 9, we present the training time for CORAL and different baselines for comparison. Since, the training of CORAL is separated in 2 steps, we show in line "INR" the training time for INR fitting and in line "Process" the second step to train the forecast model. We see that the longest part of the training procedure in CORAL is the fitting of the INR. MP-PDE is the slowest baseline to train, due to the KNN graph creation. DeepONet and FNO are the fastest baselines to train because they only need a forward pass.
C.4 Inference Time
In this section, we evaluate the inference time of CORAL and other baselines w.r.t. the input grid size. We study the impact of the training grid size (different models trained with  $5\%$ ,  $20\%$  and  $100\%$  of the grid) (Figure 6a) and the time needed for a model trained  $(5\%)$  on a given grid to make computation on finer grid size resolution (evaluation grid size) (Figure 6b).
On the graphs presented in Figure 6, we observe that except for the operator baselines, CORAL is also competitive in terms of inference time. MP-PDE inference time increases strongly when inference grid gets denser. The DINo model, which is the only to propose the same properties as CORAL, is much slower when both inference and training grid size evolve. This difference is mainly explained by the number of steps needed to optimize DINo codes. Indeed, at test time DINO's adaptation requires  $100\mathrm{x}$  more optimization steps. MPPDE's computation is slow due to the KNN graph creation and slower message passing. DeepONet and FNO are faster due to forward computation only. CORAL's encoding/decoding is relatively resolution-insensitive and performed in parallel across all sensors. Process operates on a fixed dimension independent of the resolution. FNO is fast due to FFT but
(a) Average inference time (in seconds) of the implemented baselines to unroll a trajectory until  $T = 40$  on Navier-Stokes. For irregular grids, FNO is performed following linear interpolation.
Figure 6
(b) Inference time w.r.t. evaluation grid size (same models).
cannot be used on irregular grids. For these experiments, we used linear interpolation which slowed the inference time.
C.5 Propagation of Errors Through Time
In Figures 7a to 7c, we show the evolution of errors as the extrapolation horizon evolves. First, we observe that all baselines propagate error through time, since the trajectories are computed using an auto-regressive approach. Except for the  $100\%$ , DeepONet had difficulties to handle the dynamic. It has on all settings the highest error. Then, we observe that for MP-PDE and FNO, the error increases quickly at the beginning of the trajectories. This means that these two models are rapidly propagating error. Finally, both DINo and CORAL have slower increase of the error during In-t and Out-t periods. However, we clearly see on the graphs that DINo has more difficulties than CORAL to make predictions out-range. Indeed, while CORAL's error augmentation remains constant as long as the time evolves, DINo has a clear increase.
C.6 Benchmarking INRs for CORAL
We provide some additional experiments for dynamics modeling with CORAL, but with diffrent INRs: MFN (Fathony et al., 2021), BACON (Lindell et al., 2022) and FourierFeatures (Tancik et al., 2020). Experiments have been done on Navier-Stokes on irregular grids sampled from grids of size  $128 \times 128$ . All training trajectories share the same grid and are evaluated on a new grid for test trajectories. Results are reported in Table 10. Note that we used the same learning hyper-parameters for the baselines than those used for SIREN in CORAL. SIREN seems to produce the best codes for dynamics modeling, both for in-range and out-range prediction.
Table 10: CORAL results with different INRs. - Test results in the extrapolation setting on Navier-Stokes dataset. Metrics in MSE.
Xtr↓ Xte	INR	In-t	Out-t
π = 20%	SIREN	5.76e-4	2.57e-3
MFN	2.21e-3	5.17e-3
irregular grid	BACON	2.90e-2	3.32e-2
FourierFeatures	1.70e-3	5.67e-3
π = 5%	SIREN	1.81e-3	4.15e-3
MFN	9.97e-1	9.58e-1
irregular grid	BACON	1.06	8.06e-1
FourierFeatures	3.60e-1	3.62e-1
Errors wrt time
(a) Evolution of errors over time and across test samples for a model trained on  $100\%$  of the grid.
Errors wrt time
(b) Evolution of errors over time and across test samples for a model trained on  $20\%$  of the grid.
Errors wrt time
(c) Evolution of errors over time and across test samples for a model trained on  $5\%$  of the grid.
Figure 7: Errors along a given trajectory.
C.7 Impact of 2nd order meta-learning
We provide in Figure 8 the evolution of the reconstruction error through the training epochs for Navier-Stokes with first-order and second-order meta-learning. The first order method is able to correctly train until it falls into an unstable regime for the common parameters. The second order method is much more stable and achieves a  $10\mathrm{x}$  reduction in MSE.
INR - training loss with 1st order meta-learning.
Figure 8: Training of the modulated INR - Comparison on Navier-Stokes over three independent runs of first order and second order meta-learning. We use the same number of inner-steps.
INR - training loss with 2nd order meta-learning.
C.8 Key hyper parameter analysis
Table 11 presents a hyperparameter study on reconstruction and forecasting tasks for Navier-Stokes dataset. Three hyperparameters—initial weight  $w_{0}$ , latent dimension  $d_{z}$ , and width—are varied to observe their impact. We can notice that:
-  $w_{0} = 30$  slightly improves the reconstruction on the test set.
-  $d_{z} = 64$  yields a better forecasting In-t performance.
- width = 256 significantly improves the model's performance across nearly all metrics.
Table 11: Hyper parameter study - Reconstruction and forecasting results on Navier-Stokes dataset. Metrics in MSE. Reconstruction are reported on the Train (In-t) and on the Test (In-t + Out-t).
Param ↓	Value ↓	Reconstruction	Forecasting
Train	Test	In-t	Out-t
w0	20	3,62e-5	6,86e-5	2,78e-4	1,88e-3
30	3,66e-5	5,85e-5	4,03e-4	2,28e-3
dz	64	3,94e-5	1,11e-4	1,22e-4	1,42e-3
256	2,73e-5	8,03e-5	1,63e-4	2,12e-3
width	64	1,50e-4	2,87e-4	2,84e-4	2,39e-3
256	1,60e-5	6,41e-5	1,23e-4	2,04e-3
CORAL baseline	-	1.05e-4	1.21e-4	1.86e-4	1.02e-3
D Supplementary results for geometry-aware inference
D.1 Inverse Design for NACA-airfoil
Once trained on NACA-Euler, CORAL can be used for the inverse design of a NACA airfoil. We consider an airfoil's shape parameterized by seven spline nodes and wish to minimize drag and maximize lift. We optimize the design parameters in an end-to-end manner. The spline nodes create
the input mesh, which CORAL maps to the output velocity field. This velocity field is integrated to compute the drag and the lift, and the loss objective is the squared drag over lift ratio. As can be seen in Figure 9, iterative optimization results in an asymmetric airfoil shape, enhancing progressively the lift coefficient in line with physical expectations. At the end of the optimization we reach a drag value of 0.042 and lift value of 0.322.
(a) Step  $= 0$
(b) Step  $= 1000$
(c) Step  $= 3000$
(d) Step  $= 5000$
Figure 9: Design optimization of a NACA-Airfoil.
E Qualitative results
In this section, we show different visualization of the predictions made by CORAL on the three considered tasks in this paper.
E.1 Initial Value Problem
We provide in Figure 10 and Figure 11 visualizations of the inferred values of CORAL on Cylinder and Airfoil.
E.2 Dynamics modeling
We provide in Figure 13 and Figure 12 visualization of the predicted trajectories of CORAL on Navier-Stokes and Shallow-Water.
Figure 10: CORAL prediction on Cylinder
Figure 11: CORAL prediction on Airfoil
Figure 12: Prediction MSE per frame for CORAL on Navier-Stokes with its corresponding training grid  $\mathcal{X}$ . Each row corresponds to a different sampling rate and the last row is the ground truth. The predicted trajectory is predicted from  $t = 0$  to  $t = T'$ . In our setting,  $T = 19$  and  $T' = 39$ .
Figure 13: Prediction MSE per frame for CORAL on Shallow-Water with its corresponding training grid  $\mathcal{X}$ . Each row corresponds to a different sampling rate and the last row is the ground truth. The predicted trajectory is predicted from  $t = 0$  to  $t = T'$ . In our setting,  $T = 19$  and  $T' = 39$ .
E.3 Geometry-aware inference
We provide in Figure 14, Figure 15, Figure 16 visualization of the predicted values of CORAL on NACA-Euler, Pipe and Elasticity.
Figure 14: CORAL predictions on NACA-Euler
Figure 15: CORAL predictions on Pipe
Figure 16: CORAL predictions on Elasticity