DeepLag: Discovering Deep Lagrangian Dynamics for Intuitive Fluid Prediction
Authors: Qilong Ma
Haixu Wu
Lanxiang Xing
Shangchen Miao
Mingsheng Long
Venue: NeurIPS 2024 poster (2024)
Keywords: Deep learning
Fluid prediction
Lagrangian perspective
Primary area: machine_learning_for_physical_sciences
TL;DR: We propose DeepLag to tackle the intricate fluid dynamics by guiding the Eulerian fluid prediction with learned dynamics of tracked Lagrangian particles.

Abstract: Accurately predicting the future fluid is vital to extensive areas such as meteorology, oceanology, and aerodynamics. However, since the fluid is usually observed from the Eulerian perspective, its moving and intricate dynamics are seriously obscured and confounded in static grids, bringing thorny challenges to the prediction. This paper introduces a new Lagrangian-Eulerian combined paradigm to tackle the tanglesome fluid dynamics. Instead of solely predicting the future based on Eulerian observations, we propose DeepLag to discover hidden Lagrangian dynamics within the fluid by tracking the movements of adaptively sampled key particles. Further, DeepLag presents a new paradigm for fluid prediction, where the Lagrangian movement of the tracked particles is inferred from Eulerian observations, and their accumulated Lagrangian dynamics information is incorporated into global Eulerian evolving features to guide future prediction respectively. Tracking key particles not only provides a transparent and interpretable clue for fluid dynamics but also makes our model free from modeling complex correlations among massive grids for better efficiency. Experimentally, DeepLag excels in three challenging fluid prediction tasks covering 2D and 3D, simulated and real-world fluids. Code is available at this repository: https://github.com/thuml/DeepLag.

DeepLag: Discovering Deep Lagrangian Dynamics for Intuitive Fluid Prediction
Qilong Ma, Haixu Wu Xing, Shangchen Miao, Mingsheng Long School of Software, BNRist, Tsinghua University, China
(mq122, wuhx23, 1abxx2,xxx2222 x1x22, 11xx22,msc22) msc21) Smails. tainghusnngha.nenn edu. en, mingshenggtsingguu..c edu.
Abstract
Accurately predicting the future fluid is vital to extensive areas such as meteorology. oceanology, and aerodynamics. However, since the fluid is usually observed from the Eulerian perspective, its moving and intricate dynamics are seriously obscured and confounded in static grids, bringing thorny challenges to the prediction. This paper introduces a new Lagrangian-Eulerian combined paradigm to tackle the tanglesome fluid dynamics. Instead of solely predicting the future based on Eulerian observations, propose DeepLag to discover hidden Lagrangian dynamics within the fluid by tracking the movements of adaptively sampled key particles. Further, DeepLag presents
paradigm for fluid prediction, where the Lagrangian movement the tracked particles is inferred from Eulerian observations, and their accumulated Lagrangian dynamics information is incorporated into global Eulerian evolving features to guide future prediction respectively. Tracking key particles not only provides transparent and interpretable clue for fluid dynamics but also makes our model free from modeling complex correlations among massive grids for better efficiency. Experimentally. DeepLag excels in three challenging fluid prediction tasks covering 2D and 3D, simulated and real-world fluids. Code is available this repository: https://githu: cwm/too/tiiate hemm/hhum//eeppag
Introduction
Fluids, characterized by molecular structure that offers no resistance to external shear forces, easily deform even under minimal stress, leading highly complex and often chaotic dynamics [TO]. Consequently, the solvability of fundamental theorems in fluid mechanics, such as Navier-Stokes equations, is constrained to only a limited subset of flows due their inherent complexity and intricate multiphysics interactions (38]. In practical applications, computational fluid dynamics (CFD) widely employed to predict fluid behavior through numerical simulations, but hindered by significant computational costs. Accurately forecasting future fluid dynamics remains a formidable challenge. Recently, deep learning models (8] 28 [2] have shown great promise for fluid prediction due to their exceptional non-linear modeling capabilities. These models, trained CFD simulations or real-world data, can serve as efficient surrogate models, dramatically accelerating inference.
A booming direction for deep fluid prediction is learning deep models to solve partial differential equations (PDEs) [45]. However, most of these methods [48 29 22 21] attempt capture fluid dynamics from the Eulerian perspective, which means modeling spatiotemporal correlations among massive grids unchanging over time. From this perspective,
complicated moving dynamics in fluids could be seriously obscured and confounded in static grids, bringing challenges in both computational efficiency and learning difficulties for accurately predicting future fluids.
In parallel to the Eulerian method, we notice another major approach for elucidating fluid dynamics, the Lagrangian method [13, also known as the particle tracking method. This method primarily
"Equal Contribution
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Learned Trajectory
T 2
T+ 4
T+ 6
T+ 8
T+ 10
Figure 1: Comparison between Lagrangian (left) and Eulerian (right) perspectives. The left depicts the learned trajectories of Lagrangian particles overlaid on the mean state, while the right displays the positions of tracked particles in successive Eulerian frames. Fluid motion is more visibly represented through the dynamic Lagrangian view compared to the density variations in static Eulerian grids.
focuses on tracing individual fluid particles by modeling the temporal evolution of their position and velocity. Unlike the Eulerian methods, which observe fluid flow at fixed spatial locations, the Lagrangian approach describes the fluid dynamics through the moving trajectory of individual fluid particles, offering a more natural and neat representation of fluid dynamics with inherent advantages in capturing intricate flow dynamics. Moreover, it allows larger inference time step tha Eulerian methods while complying with the find that fluid dynamics is much more visually apparent in Lagrangian trajectories
- condition (6] that guarantees stability.
In Figure on the left compared to the density changes observed on static Eulerian grids on the right.
Building on the two perspectives mentioned earlier, we propose DeepLag a qulerian-LLagangiinn Recurrent Network. Our aim to integrate Lagrangian tracking into the deep model, thereby enhancing the dynamics modeling in Eulerian fluid prediction. To achieve this, present the EuLag Block, a powerful module that accomplishes Lagrangian tracking and Eulerian predicting at various scales. By leveraging the pross-attention mechanism, the EuLag Block assimilates tracked Lagrangian particle dynamics into the Eulerian field, guiding fluid prediction. It also forecasts the trajectory and dynamics of Lagrangian particles with the aid of Eulerian features. This unique Beglerian-Lagrangee design harnesses the dynamics information captured by Lagrangian trajectories and the fluid-structure features learned in the Eulerian grid. In experiments, DeepLag consistently outperforms existing models, demonstrating state-of-the-art performance across three representative datasets, covering 2D and 3D fluid various scales. Our contributions are follows:
Going beyond learning fluid dynamics at static grids, we propose DeepLag featuring the Lagraninn Recurrent Network, which integrates both Eulerian and Lagrangian frameworks from fluid dynamics within pure deep learning framework concisely.
Inspired by Lagrangian mechanies, we present EuLag Block, which can accurately track particle movements and interactively utilize the Eulerian features and dynamic information in fluid prediction, enabling better dynamics modeling paradigm.
DeepLag achieves consistent state-of-the art on three representative fluid prediction datasets with superior trade-offs for performance and efficiency, exhibiting favorable practicability.
2 Preliminaries
2.1 Eulerian and Lagrangian Methods
Eulerian and Lagrangian descriptions are two fundamental perspectives for modeling fluid motion. The Fulerian view, commonly used in practical applications [23]. observes fluid at fixed points and records physical quantities, such density, future fluid can be predicted by integrating velocity along the temporal dimension and interpolating the results to observed grid points [49). In contrast, the Lagrangian view focuses on the trajectory of individual particles, tracking a particular particle from initial position So by its displacement d d= d( 1) at time This approach reveals the intricate evolution of the fluid by following particle trajectories, making it convenient for describing complex phenomena like vortices, turbulence, and interface motions 40 Two perspectives are constitutionally equivalent, bridged by the velocity:
function of position and time,
v(s. f). Thus,
v (d(sn, t),t) ) (so. t).
(1)

---

Furthermore, the material derivative Dq that describes the change rate of a physical quantity q of a Dr fluid parcel can be written as the sum of the two terms reflecting the spatial and temporal influence on [1] which represent the derivatives on Fulerian domain and Lagrangian convective respectively:
Dq DE
Qq a Domain derivative
+
u Vg
(2)
Convective derivative
This connection inspires us to incorporate Lagrangian descriptions into dynamics learning with Eule- rian data, enabling a more straightforward decomposition of complex spatiotemporal dependencies.
While traditional particle-based (or mixed-representation) solvers demonstrate superior accuracy and adaptability in inferring small-scale phenomena and dealing with nonlinear and irregular boundary conditions, they require computing the acceleration of each particle through physical equations, followed by sequential updates of their velocity and position [31]. This pointwise modeling approach often demands a significant number of points to fully characterize the dynamics of the entire field to meet accuracy requirements. Moreover, the irregular arrangement of particles and difficulty in parallelization result higher computational costs and challenges with particle interpolation and gridding [12]. This renders particle-base based solvers suboptimal compared to Eulerian solvers, particularly in high-dimensional spaces and large-scale simulations. However, the proposed Deepl.ag leverages the strengths of both solvers, eschewing equations and instead utilizing Fulerian information to assist particle tracking directly. This greatly alleviates the pressure on Lagrangian representation, requiring significantly fewer representative particles to aggregate the dynamics of the entire field.
2.2 Neural Fluid Prediction
As computational fluid dynamics (CFD) methods often require hours even days for simulations 41. deep models have been explored as efficient surrogate models that can provide sear-instantaneous predictions. These neural fluid prediction models approximate the solutions of governing fluid equations differently and can be categorized into three mainstream paradigms as in Figure ((a-c).
Classical MI. methods As depicted in Figure aa), these methods either replace part of numerical method with a neural surrogate [39] or encode multivariate fields into a single-variable latent state z 26 (55, on which they model an ODE governing the state function z, T + Rdd representing the first-order time derivative, through neural networks. However, the absence of physical meaning and guidance for latent states in evolution leads to error accumulation and generally short forecasting horizon. A detailed comparison between DeepLag and these models is provided in Appendix
B
Physics-Informed Neural Networks (PINNs) This branch of methods in Figure bb) adopts deep models to learn the mapping from coordinates to solutions and formalize PDE constraints along with initial and boundary conditions as the loss functions [48] 29 46, 47). Though this paradigm can explicitly approximate the PDE solution, they usually require exact formalization for coefficients and conditions, limiting their generality and applicability to real-world fluids that are usually partially observed 35 Plus, the Fulerian input disables them from handling Lagrangian descriptions.
Neural Operators Recently, a new paradigm, illustrated in Figure ((c), has emerged where deep models learn the neural operators between input and target functions, e.g., past observations to future fluid predictions. Since DeepONet (22), various neural operators have significantly advanced fluid prediction, which directly approximates mappings between equation parameters and solutions. For Eulerian grid data, models based on U-Net (32) and ResNet [15] architectures have been proposed as well as variants addressing issues like generalizing to unseen domains [44) irregular mesh I1. and uncertainty quantification [51]. Transformer- based models [43] enhance modeling capabilities and efficiency by exploiting techniques like Galerkin attention B. incorporating ensemble information from the grid [14], applying low-rank decomposition to the attention mechanism 220]. and leveraging spectral methods in the latent space [52]. Additionally, FNO (2]) learns mappings in the frequency domain, and MP-PDE (2 utilizes the peessag--passing mechanism. For Lagrangian fluid particle data, some CNN-based methods [34
30 27
42
model particle interactions through redesigned
basic modules, while GNN-based methods 33
25 update particle positions using the Encode-
Process- Decode paradigm. Despite the progress made by these methods, they are limited to one description and do not scamlessly combine Eulerian and Lagrangian views.
3

---

DeepLag
Following the convention neural fluid prediction [21]. we formalize the fluid prediction problem as learning future fluid given past observation, as shown in Figure 2 Given bounded open subset of d-dimensional Euclidean space C IRd and a Eulerian space C R with o observed physical quantities, letting u, (x) C 2 and ux+1(x) U represent the Eulerian fluid field observation at two consecutive time steps on finite coordinate set x C D, we aim at fitting the mapping (x) > u++1 Concretely, provided initial P step observations Up {ui,..... up). the fluid prediction process can be written as the following autoregressive paradigm:
U, (u--rt1---.. U1)
Fa
U,+1,
(3)
where 2 P and Fø represents the learned mapping between Ug and the predicted field u,+1-
Inspired by the material derivative in Eq. (2). we present DeepLag as Fulerian-Llarrangia Recurrent Network, which utilizes the Eul.ag Block learn Fulerian features and Lagrangian dynamics interactively at various scales to address the complex spatiotemporal correlations in fluid prediction. Specifically, we capture the temporal evolving features at fixed points from the Eulerian perspective and the spatial dynamic information of essential particles from the Lagrangian perspective through their movement. By integrating Lagrangian pivotal dynamic information into Eulerian features, fully model the spatiotemporal evolution of the fluid field over time and motion. Moreover, DeepLag can obtain critical trajectories within fluid dynamics with high computational efficiency by incorporating high figh-dimensional Eulerian space into
Lagrangian space.
(n) Classienl MI. methoda
PINNs
(e) Neural Operators
Rnooder
OID
apaces
Decoder
Impul
Solve
Solution
Inpus Cunction
Approsimat Mapping
Ourput Fonction
U
Za
dt
(et),)
fo(
u(x.
(o)
vÊx. (")
(d) Deepl.ag
Inpur field
Outpat field Mgas
Eulerian
Extract Dynamies
Obeervatio
Space
Dynamics Sampling
EuLag Block
(Mc11: Pui pb Pe.u..P..b)) P..
Hiddln
Lagrangian Space
Sample
Dynomics
Pr.b; position and dynamies Pe+1, Dett
Figure 2: Three types of neural fluid prediction models (a-c) and overview of DeepLag (d). The EuLag Block accumulates the previous dynamics each time and scale to guide the Eulerian field update and then evolves the particle movement and dynamics conditioned on the updated field.
3.1 Overall Framework
It is widely acknowledged that fluids exhibit different motion characteristics across varying scales 3 52 In order capture the intrinsic dynamics information at different scales, we track the trajectories of the key particles on L scales separately and propose Buleriian-Lagraneee realize the interaction between Eulerian and Lagrangian information. For clarity, we omit the scale index / for primary physical quantities in this subsection, where / E ... LL..
Recurrent Network to
Initializing Lagrangian particles To better capture complex dynamics in the fluid field, we sample by importance of the points from Eulerian perspective observations to determine initial positions for Lagrangian tracking. This is done by our dynamics sampling module. For the first predicting step = P. given observations u, (xxxx)) Di, < k Ni) Nexo on all N points in observation domain D, C Rd of each scale, we extract their spatial dynamics by convolutional network. We then calculate the probability matrix N via softmax along spatial dimension:
S Softmax (ConNNet(u,)).

---

where ConvNet(-) consists of convolutional layer and linear layer, with activation function in-between. We then sample M, Lagrangian tracking particles by the probability matrix at each scale:
P. Sample( (xxx
(x*) N S),
(5)
where P: (p-,, E DI
R M, represents the set of sampled M, particles.
Eulerian fluid prediction with Lagrangian dynamics For the 1-th timestep, we adopt learnable embedding layer with multiscale downsampling Down(.) to encode past observations or predictions
U² (u-p++.-... uz} to obtain the Eulerian representations u/ E R RNNxCi at each scale.
We integrate the EuLag Block to fuse the Lagrangian dynamics at each scale and direct the evolution of Eulerian features. This interaction simultaneously enables Eulerian features to guide the progression of Lagrangian dynamics. For the /-th scale, we track Mj key particles over time, with p. [Pl, E
Ad, denoting their learned particle Di M representing their positions and h, { J dynamics. As shown in Figure d), the positions and dynamics of the Mj particles at the /-th scale
are learned autoregressively using the EuLag Block, which can be written as
u4+1: {p++i), (b++1} } = EuLag(uu, {p:}. (hi)),
(6)
where the scale index I is omitted for notation simplicity. The EuLag Block learns to optimally leverage the complementary strengths of Eulerian and Lagrangian representations, facilitating mutual refinement between these two perspectives towards an exceeding performance. More details about the specific implementation of the EuLag Block are elaborated in the following subsection 3.2
After evolving into new Eulerian features u,+1, Lagrangian particle position P++1 and dynamics h the 1-th scale, we further aggregate U4++1 with the predicted Eulerian field coarser scale by upsampling Up(-). Eventually, the full-resolution prediction U,+1 at step t + is decoded from u with projection layer. We unfold the implementation of the overall architecture in Appendix A.2
Falerian Rvolution
Lagrangian Tracking
Euler Space
th Scale Iapula
Lagrange Space
K
Interpolate
Local Dynaniles
Gilcaal
Aggregate
Dynamics
0 Peet
Pr
LagToBu
EaToLog
Figure 3: Overview of the EuLag Block, which accumulates previous dynamics information to guide Eulerian evolution for predicting particle movement. Scale index is omitted for simplicity.
3.2 EuLag Block
As stated in Eq. (6). we adopt a recurrent network to interactively exploit the information from two fluid descriptions, which consist of two main components: Lagrangian-guiin guided feature evolving and interian-conditionee particle tracking. The following quantities are all in the /-th scale.
Lagrangian-guidee feature evolving Classical theories [ and numerical algorithms that fluid predictions can be solved by identifying the origin of the fluid parcel and interpolating the dependent variable from nearby grid points. However, without specifying certain PDEs, we cannot explicitly determine the former position of the particle on each Eulerian observed point. Thus, we first adaptively synthesize the Lagrangian dynamics of the tracked particles guide the evolution of Eulerian features using a cross-attention mechanism. Formally, we adopt Lagrangian- toe--t---uuua -ottgiiin-tt-uueriian cross-attention, where the Eulerian field u, serves as queries, and Lagrangian dynamics concatenated with particle position hellp, TR. (CC+t)) is used keys and values:
31 show
W (Wou (W K by hullp::) Wy h
Lag TaggToFu-ae ToFu--Atn(uu Ip. h, hellp:) Softmax
(7)
VC, +
where Wq. QQ W K and W y stand for linear projections. We wrap the attention in Transformer block with residual connections to implement the Lagrangian-dynamicc manamics-guiden Eulerian evolution process:
u++ u, + LagToEu(u, h,| Pp. h (p:).
(8)
5

---

Sulerian-conditione particle tracking Traditional Lagrangian methods rely on interactions among vast quantities of particles to estimate the future fluid fields. However, the high computational cost hinders the application in deep surrogate models. In other data-driven approaches, the sparse sampling of particles computational-frinne other particles. Considering the equivalence of the Eulerian and Lagrangian representations indicated by the material derivative in Eq. (.. we propose to learn particle movements based on the Eulerian conditions. Concretely, we utilize another paglerian-toLaagrnngee
but cannot directly derive Lagrangian dynamies for
cross-attention, where the evolved dense Eulerian features are used to navigate the Lagrangian dynamics of sparse particles:
(We by hullp:) (Wk))+1)
SuTooaagAAttnb//Ipe 11 Mp+1) = Softmax
W Wyu
(9)
VCI
where we use a different set of W q. W and W'. Similarly, the Transformer block EuToLag wrapping this attention produces the change of forecasted global Lagrangian dynamics S and movement of tracking particles op,, which leads to the next step by residual connections:
hglobul, || hel/p: + EuToLag (he/)pp llp, M4+1, Ue+1).
(10)
To better model the dynamic evolution of the particles, we gather local Lagrangian dynamic by employing bilinear interpolation to evolved Eulerian features U,+1 on new particle position P++1. then use linear function to aggregate with global dynamics information hglobal,t411
h Aggregate (nnrrgaatt((teeplaaeeeeee
P++1). Aglobal,t+1).
(11)
Additionally, particles could move out of the observation domain as the input field may have an open boundary. We check the updated position of tracking particles and resample from the latest probability matrix S to substitute the ones that exit, ensuring the validity of the Lagrangian information.
Overall, the Eul Block can fully utilize the complementary advantages of Eulerian and Lagrangian perspectives in describing fluid dynamics, thereby being better suited for fluid prediction. For more implementation details of the EuLag Block, please refer to Appendix A.3
4 Experiments
We evaluated DeepLag on three challenging benchmarks, including simulated and real-world sce- narios, covering both 2D and 3D, well as single and multi-physics fluids. Following the previous convention [21). we train DeepLag and the baselines for each task to predict ten future timesteps in autoregressive fashion given ten past observations. Detailed benchmark information is listed in Table ! We provide an elaborate analysis of the efficiency, parameter count, and performance Additionally, more detailed visualizations, besides in later this section, are provided in Appendix H Furthermore, the trained models are engaged to perform 100 frames extrapolation examine their long-term stability, whose results are in Appendix]] []]
difference section 44 and Appendix E
Baselines To demonstrate the effectiveness of our model, we compare DeepLag with seven baselines all benchmarks, including the clas- sical multiscale model U-Net [32) and advanced neural operators for Savicr-Stokes equations: FNO 21, Galerkin Transformer 3 Vortex for 2D image 7 GNOT [14] LSM 122 and Fact-
Table Summary of the benchmarks. #Var refers to the number of observed physics quantities in fluid. #Space is the spatial resolution.
Datasets
Type
#Var
#Dim
#Space
Bounded N-$
Simulation
1
2D
128
128
Ocean Current
Real World
5
2D
180 x 300
Former 20) U-Net has been widely used in fluid modeling, which can model the multiscale
3D Smoke
Simulation
4
3D
323
property precisely. LSM [22] and FactFormer 200 are previous state-of-the-art neural operators.
Metrics For all three tasks, we follow the convention in neural fluid prediction 21 52 and report relative L2 as the main metric. Implementations of the metrics are included in Appendix A.4
Implementations Aligned with convention and the baselines, DeepLag is trained with relative L2 as the loss function on all benchmarks. We use the Adam [8] optimizer with an initial learning rate of 5 10-1 and StepLR learning rate scheduler. The batch size is set to 5, and the training process stopped after 100 epochs. All experiments are implemented in Py PyTorc 24 and conducted on a single NVIDIA A.00 GPU. Training curves are shown in Appendix

---

Ground Truth (T=
DeepLag (Our))
U-Net
LSM
Timewiee Relalive L2
Dobce GIArT
Roletive 1.2
Prediction Reroe
Prodict Stop
Figure 4: Showcases (left) and timewise relative L2 (right) on Bounded Navier-Stokes dataset. Both predictions (upper row) and absolute error maps (lower row) are plotted for intuitive comparison.
4.1 Bounded Navier-Stokes
Setups In real-world applications, handling complex boundary conditions in predicting fluid dynamics indispensable. Thus, we experiment with newly generated Bounded Navier-Stokes, which simulates scenario where some colored dye flows from left to right through a 2D pipe with several fixed pillars as obstacles inside. Details about this benchmark can found in Appendix C.T
Quantitive results As shown in Ta- ble p DeepLag achieves the best per- formance on Bounded Navier-Stokes, demonstrating its advanced ability han- dle complex boundary conditions. In comparison to the previous best model, DeepLag achieves a significant 13.8%
Table 2: Performance comparison on Bounded Navier- Stokes. Relative L2 of 10 frames and 30 frames prediction are recorded. Promotion represents the relative promo- tion DeepLag w.r.t the second-best (underlined). "NaN" refers to the instability during rollout.
Relative L2 (4)
(Relative I.2: 0.0618 v.s. 0.0544) and 2.7% (Relative L.2: 0.1020 v.s. 0.0993) relative promotion on short and long roll- out. The timewise error curves of all the models are also included Figure We can find that DeepLag presents slower error growth and excels long-term forecasting. This result may stem from the iagrangian-uuide which can accurately capture the dynam- ics information over time, further verify- ing the effectiveness of our design.
Model
Frames
30 Frames
U-Net 32
0.0618
0.1038
FNO
0.1041
0.1282
Galerkin Transformer
3
0.1084
0.1369
Vortex 7
0.1999
NaN
GNOT [4
0,1388
0.1793
LSM [52
0.0643
0. 0020
FactFormer
0.0733
0.1
fluid prediction,
DeepLag (Ours)
0.0543
0.0993
Promotion
13.8%
2.7%
Showcases To intuitively present the forecasting skills of different models, we also provide show- case comparisons in Figure[4and particle movements predicted by Deepl ag in Appendix K We can find that DeepLag can precisely illustrate the vortex in the center of the figure and give a reasonable motion mode of the Kármán vortex phenomenon formed behind the upper left pillar. As for U-Net and LSM, although they successfully predicted the position of the center vortex, the error map shows that they failed to predict the density field as accurately deteriorates on this benchmark. This may be because it is based on spatial factorization, which is unsuitable for irregularly placed boundary conditions. These results further highlight the benefits of Eulerian Balerian-Larrangian co-design, which can simultaneously help with dynamic and density prediction.
DeepLag. In addition, FactFormer
4.2
Ocean Current
Setups
Predicting large-scale ocean currents, especially in regions near tectonic plate boundaries prone to disasters such as tsunamis due to intense terrestrial activities, plays a crucial role in various domains. Hence, we also explore this challenging real-world scenario our experiments. More details about the source and settings of this benchmark can be found in Appendix C.2

---

DeepLag (Ours)
U-Net
FactFormer
Ratimated Partiele Tenjectory (T 10 20)
IT 20
Prediction
Figure 5: Showcase comparison and visualization of Lagrangian trajectories learned by DeepLag on Ocean Current. Notably, potential temperatures predicted by different models are plotted. Error maps of predictions normalized to (-4, 4, 4) for a better view.
Quantitive results We report relative L2 for the Ocean Current dataset in Table where DeepLag still achieves the best with 8.3% relative promotion w.r.t. the second-best model. Even in 30 days of forecasting, the number is 12.8%. These results show that DeepLag performs well in real-world, large-scale fluids, which usually involve more inherent stochasticity than simulated data. Moreover, provide the ACC metric and timewise curve in Appendix F
Showcases To provide an intuitive compar- ison, we plotted different model predictions in Figure 5 In comparison to other models, Deeplag exhibits the most minor prediction error. It accurately predicts the location of the high-temperaturre region the south area and provides a clear depiction of the Kuroshio pattern 337 bounded by the red box.
Table 3: Performance comparison on Ocean Current. We report the relative L2 of the short-term and long- term rollouts with their relative promotions.
Relative L2 (L)
Model
10 Days
30 Days
U-Net (32
0.0185
0.0297
FNO 21
0.0246
0.0420
Learned trajectory visualization To reveal the effect of learning Lagrangian trajectories, visualize tracked particles in Figure 5 We observe that all the particles move from west east, consistent with the Pacific circulation. Additionally, the tracked particles show dis- tinct moving patterns, confirming their ability to represent complex dynamics. The move-
Galerkin Transformer
0.0323
0.0515
Vortex 7
0.9548
NaN
GNOT 114
0,0206
0.0336
LSM [52
0.0182
0.0290
FactFormer
0.0183
0.029
Deeplaag (Ours)
0.0168
0.0257
Promotion
8.3%
12.8%
ment of upper particles matches the sinuous trajectory of the Kuroshio current, demonstrating the capability of DeepLag to provide interpretable evidence for prediction results. Visualizing the tracking points in Lagrangian space instills confidence in the reliability and interpretability of the predictions made by our model, which can provide valuable and intuitive insights for real-world fluid dynamics.
4.3
3D Smoke
Setups 3D fluid prediction has been a long- standing challenge due to the tanglesome dy- namies involved. Therefore, we generated this benchmark to describe scenario in which smoke flows under the influence buoyancy in a three- dimensional bounding box. For more details, please refer to Appendix
Table Performance comparison on the 3D Smoke dataset. Relative L2 with relative promo- tion w.r.t. the second-bes best model recorded.
Model
Relative L2 (1)
U-Net 32
0.0508
FNO 21
0.0635
Galerkin Transformer S
0.1066
Quantitive results Table 4 shows that DeepLag still performs best in 3D fluid. Note that in
GNOT [4
0.2100
LSM [2
0,0527
this benchmark, the canonical deep model U-
FactFormer 20
0.0793
Net degenerates seriously, indicating that pure Eulerian multiscale framework insufficient model complex dynamics. We also noticed that the Transformer-basee neural operators, such GNOT and Galerkin Transformer, failed in this task. This may be because both of them are based on the linear attention mechanism [19] 54, which may depreciate under massive tokens. These comparisons further highlight the capability of DeepLag to handle high-dimension fluid.
Deepl.gg (Ours)
0.0378
Promotion
34.4%

---

Ground
Deeplag
D-Nel
FactFurner
Ground
Deeplag
LSM
FacTFormer
Prodiction
Pradiction
Daner
Figure 6: Showcases comparison of the whole space (left part) and a cross- cross-section (right part) on the 3D Smoke dataset. For better visualization, we present the absolute value of the prediction error and normalize the whole space error into (0, 0.12). As for the cross-section visualization, we choose the aOy plane the middle 3D fluid and normalize error maps to (-0.5, 0.5).
Showcases In Figure 6 we compared the prediction results on the 3D Smoke dataset. DeepLag demonstrates superior performance in capturing both the convection and diffusion of smoke within the bounding box. In contrast, the predictions made by U-Net tend to average value across various surfaces, resulting in blurred details, which also indicates its deficiency dynamies modeling. Similarly, L.SM and FactFormer exhibit more pronounced errors, particularly around the smoke boundaries, where complex wave interactions often occur. By comparison, our model significantly reduces both the overall and cross-sectional error, excelling in the prediction of fine-grained subtle flow patterns and maintaining high accuracy even in challenging regions with intricate dynamics.
4.4 Model Analysis
Ablations To verify the effectiveness of detailed designs in Deepl.ag, we conduct exhaustive ablations in T3 In our original design, we track 512 particles (around 3% of Fulerian grids) in total on 4 hierarchical spatial scales with latent dimension 64 in the Lagrangian space.
The experiments indicate that further increasing the number of particles, scales, or latent dimensions yields marginal performance improvements. There- fore, we opt for these values to strike a balance between efficiency and performance. In addition, we can conclude that all components proposed in this paper are indispensable. Especially, the lack of interaction between the Fulerian and Lagrangian space will cause a severe drop in accuracy, highlight- ing the dual cross-attention that exploits Lagrangian dynamics has a positive influence on the evolution of Eulerian features. Besides, rather than uniformly sampling particles, sampling from a learnable prob- ability matrix also provides upgrade (refer to Appendix G for visual results). When swapping the
Efficlency Compartson on 30 Smoke
0.24
contmmttion
0.15
6.12
Selare Toraforrer
Pestorm
Deeatre
0.5
Hunning Time Sellert
Figure 7: Efficiency comparison among all the models. Running time and Relative 1.2 are evaluated on the 3D Smoke benchmark.
Table Ablations of (a) module removing and (b) hyperparameter sensitivity on Bounded Navier- Stokes, where (a) includes w/o Lag LogToEu: removing Lagrangian-guidde feature evolving, w/o EuToLag: removing Eulerian-comdittooned particle tracking and w/o Learnable Sampling: removing learnable particle sampling strategy, and (b) includes #Particle (M1): adjusting the number of tracking particles of the first layer, #Scale (L): adjusting the number of spatial scales and #Latent (Cz): adjusting the number of latent dimension the first layer. We use the control variable method. when one hyperparameter is changed, the values of the others are all original (ori). Ablations of (c) attention swapping on Bounded Navier-Stokes (2D) and 3D Smoke (3D), where the order of EuToLag and Lag ToEu blocks is swapped. Original and Swapped Relative L2 are reported.
(a) Module Removing
(b) Hyperparameter Sensitivity
(c) Attention Swapping
Design
Relative L2
12 (1) #Scalc Relatie 1.2 #Latent
(1)
Data Original (L) Swapped
DeepLag
0,0543
0:0789
0.0455
258
0.053
2
0.0658
32
0.0594
0.0378
0.0788
weo LagToBu(-)
0,0566
512(o))
0J5443
4(oe) 5
0.0543
G4/ori)
0J0543
wio ToLeg(()
0/0577
768
0.5477
0.0554
128
0.0514
wis cantalle Sampling
0,0552

---

Table Experiments of (a) model efficiency alignment on 3D Smoke and (b) nigh-resolution data on Bounded Navier-Stokes. To compare with an aligned efficiency. we add more convolutional layers into the standard U-Net and increase the latent dimension. the high-resolution simulation, we trained new DeepLag model on newly generated 256x256 Bounded Navier-Stokes dataset. Model parameters (#Param), GPU memory (Mem) and running time per epoch (Time) are reported.
(a) Model Efficiency Alignment
(b) High-resolution Data
Model
|#Param
Mem
Time
Relative 1.2 (4)
Resolution
Mem
Time
Relative L. (4)
U-Net-scale
336M
13672MB 601s/ep
NaN 0.0378
128 x 128 5420MB
1150s/ep
0.0543
DeepLag
19M
12112MB 845s/ep
256 x 256
13916MB 1300s/ep
0.0514
positions of the EuToLag and LagTooEu blocks, the minimal performance change in both 2D and 3D benchmarks suggests the equivalence of the two perspectives, and the flow of information between them is bidirectional and insensitive to the order, underscoring the robustness of our approach. The above results provide solid support to our motivation in tracking essential particles and utilizing Eulerian-Lagrangiin recurrence, further confirming the merits of our model.
Efficiency analysis We also include the efficiency comparison in Figure 7 DeepLag hits favorable balance between efficiency and performance by simultaneously considering performance, model parameters, and running time, demonstrating superior performance with significantly less memory than GNOT and LSM, thereby minimizing storage complexity. Standard U-Net has a large number of parameters, and Transformers have quadratic memory, so in large-scall scale or complex fluid prediction scenarios, using linear complexity attention mechanisms like ours is necessary. This explains why, although U-Net and I.SM are good at Bounded Havier-Stokes, they will worsen when it comes to complex fluid dynamics, such as the Ocean Current and the 3D Smoke benchmark.
Comparison under aligned efficiency As aforementioned, U-Net also presents competitive perfor- mance and efficiency some cases. To carry out fair comparison between baselines and DeepLag, we examine them on 3D Smoke, which scales up U-Net to have rumning time similar to DeepLag as in Table a)). The results show that too many parameters overwhelm small baselines at the lower left corner in Figure 7 like U-Net, indicating that they have a shortcoming in scalability.
Generalization analysis To verify the generalizing ability of DeepLag larger and new domains, we ran tests on high-resolution (HR) data as in Table 6 b) and unseen boundary conditions (BC) as in Appendix [ respectively. The HR simulation on the 256 256 Bounded Navier-Stokes shows that the finer the data are, the more accurate our DeepLaag is, which still has a 17% promotion w.r.t U-Net on relative 1.2 (DeepLag: 0.051, U-Net: 0.060). The comparison of GPU memory and running time per epoch also confirms that the increase of the space complexity is sublinear and the time complexity is minor, underscoring the scalability of DeepLag. For BCs, we ran a zero-shot test with the old model checkpoint on newly generated Bounded N-S dataset, which has obstacles of different numbers, positions, and sizes. DeepLag still has a 7% promotion w.r.t the best baseline, U-Net, on relative L2 (DeepLag: 0.203, U-Net: 0.217). The visual comparison between the two models further shows that DeepLag adaptively generalizes well on unknown and more complex domains.
Conclusions and Limitations
To tackle intricate fluid dynamics, this paper presents Deepl.ag by introducing the Lagrangian dynamics into Fulerian fluid, which can provide clear and neat dynamics information for prediction. A EuLag Block is presented in the Eulerian-Lagranginn Recurrent Network to utilize the complementary advantages of Eulcrian and Lagrangian perspectives, which brings better particle tracking and Eulerian fluid prediction. DeepLag excels in complex fluid prediction with an average improvement of nearly 10% across three carefully selected complex benchmarks, even in 3D fluid, and can also provide interpretable evidence by plotting learned Lagrangian trajectories. However, the number of tracking particles in DeepLag is appointed as a fixed hyperparameter that needs to be adjusted for specific scenarios, and missing Lagrangian supervision in the data leads to lack of judgment on learned Lagrangian dynamics, which leave space for future exploration. That said, results show that the performance always tends to incline as we simply scale up. Therefore, the few hyperparameters for tuning depend more the limitation of computing resources rather than blind searching.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the National Natural Science Foundation of China (U2342217 and 62021002), the BNRist Project, and the National Engineering Research Center for Big Data Software.
References
G. K. Batchelor. An Introduction Fluid Dynamics. Cambridge Mathematical Library. Cambridge University Press, 2000.
[]] Johannes Brandstetter, Daniel Worrall, and Max Welling. Message passing neural PDE solvers, In International Conference on Learning Representations, 2022.
[3] Shuhao Cao. Choose transformer: Fourier or galerkin. Conference on Neural Information Processing Systems, 2021.
[4] Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[5] CMEMS and MDS. Global ocean physics reanalysis. DOL: 10.48670/moi-00021 (Accessed 23 September 2023), 2023.
[6] R. Courant, K. Friedrichs, and Lewy. partial difference equations of mathematical physics. IBM Journal of Research and Development, 11(2)215--34, 1967.
[7] Yitong Deng, Hong-Xima Yu, Jiajun Wu, and Bo Zhu. Learning vortex dynamics for fluid inference and prediction. In ICLR, 2023.
[8] Mahesh Dissanayake and Nhan Phan-Thien, Neural-network-bassee approximations for solving partial differential equations. Communications in Numerical Methods in Engineering. 1994.
[9] L.C. Evans. Partial Differential Equations. Graduate studies in mathematics. American Mathematical Society, 2010.
[10] Joel Ferziger, Milovan Perić, and Robert Street. Computational Methods for Fluid Dynamics. Springer Nature Switzerland, 2020.
[11] Han Gao, Luning Sun, and Jian-Xun Wang. Phygeonet: Physics-informed geometry-alaptive convolutional neural networks for solving parameterized steady-state pdes on irregular domain. Journal of Computational Physics, 428:1 10079, March 2021.
[12] Robert A. Gingold and Joseph John Monaghan. Smoothed particle hydrodynamics: Theory and application to non-spherical stars. Monthly Notices of the Royal Astronomical Society. 181:375-383 1977.
[13] K.C. Gupta. Classical Mechanics of Particles and Rigid Bodies. Wiley, 1988.
[14] Zhongkai Hao, Chengyang Ying, Zhengyi Wang, Hang Su, Yinpeng Dong, Songming Liu, Ze Cheng, Jun Zhu, and Jian Song. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, 2023.
[15] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.
[16] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv: Learning. 2016.
[17] Zichao Jiang, Junyang Jiang, Qinghe Yao, and Gengehao Yang. A neural network-based pde solving algorithm with high precision. Nature News, Mar 2023.
[18] Diederik Kingma and Jimmy Ba. Adam: method stochastic optimization. International Conference on Learning Representations, 2015.
[19] Nikita Kitaev, Lukasz Kaiser, and Auselm Levskaya. Reformer: The efficient transformer. International Conference on Learning Representations, 2020.
[20] Zijie Li, Dule Shu, and Amir Barati Farimani. Scalable transformer for pde surrogate modeling. In Conference Neural Information Processing Systems, 2023.

---

[21] Zongyi Li, Nikola Borislavov Kovachki. Kamyar Azizzadenesheli, Burigede liu. Kaushik Bhattacharya. Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021.
[22] Lu Lu, Pengzhan Jin, Guofei Pang. Zhongqiang Zhang, and George Em Kamiadakis. Learning nonlinear operators via deeponet based the universal approximation theorem of operators. Nature Machine Intelligence, 2021.
[23] Faith A Morrison. An introduction to fluid mechanics. Cambridge University Press, 2013.
[24] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, perfforrrr deep learning library. Advances Neural Information Processing Systems, 2019.
[25] Tobias Pfaff, Meire Fortunato, Alvaro Gonzaez- and Peter Battaglia. Learning mesh-based simulation with graph networks. In International Conference on Learning Representations, 2021.
[26] Gavin D. Portwood, Peetak P. Mitra, Mateus Dias Ribeiro, Tan Minh Nguyen, Balasubramanya T. Nadiga, Juan A. Saenz, Michael Chertkov, Animesh Garg. Anima Anandkumar, Andreas Dengel, Richard Baraniuk, and David Schmidt. Turbulence forecasting via neural ode, 2019.
[27] Md Ashiqur Rahman, Zachary E Ross, and Kamyar Azizzadenesheli. U-no: U-shaped neural operators. TMLR, 2023.
[28] Maziar Raissi. Paris Perdikaris, and George Em Karniadakis. Physies informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint, 2017.
[29] Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019.
[30] Bogdan Raonic, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bartolucci, Rima Alaifari, Siddhartha Mishra, and Emmanuel de Bezenac. Convolutional neural operators for robust and accurate learning PDEs. In shirty-seventh Conference on Neural Information Processing Systems, 2023.
[31] André Robert. A stable numerical integration scheme for the primitive meteorological equations. Amosphere-Oceun, 1961)335-4,, 1981
[32] Olaf Ronneberger. Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In The Medical Image Computing and Computer Assisted Intervention Society, 2015.
[33] Alvaro Gonzaez- Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter W. Battaglia. Learning to simulate complex physics with graph networks. In International Conference Machine Learning, 2020.
[34] Connor Schenck and Dieter Fox. Spnets: Differentiable fluid dynamics for deep neural networks, 2018.
[35] Fabio Sim, Eka Budiarto, and Rusman Rusyadi. Comparison and analysis of neural solver methods for diffcrential equations in physical systems. ELKHA: Jurnal Teknik Elektro, 2021.
[36] takah29, houkensjtu, and zemora. 2d incompressible fluid solver implemented taichi, 2023.
[37] TY Tang, JH Tai, and YJ Yang. The flow pattern north of taiwan and the migration of the kuroshio. Continental Shelf Research, 2000.
[38] Roger Temam. Navier-Stokes equations: theory and numerical analysis. American Mathematical Soc., 2001.
[39] Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. Accelerating eulerian fluid simulation with convolutional networks. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, ICML' 17, page
JMLR.org. 2017
[40] Eleuterio Toro. Riemann Solvers and Numerical Methods for Fluid Dynamics: A Practical Introduction. 2009.
[41] Nobuyuki Umetani and Bernd Bickel. Learning three- dineessiinnio flow for interactive aerodynamic design. ACM Transactions Graphics (TOG), 2018.
[42] Benjamin Ummenhofer, Lukas Prantl, Nils Thuerey, and Vladlen Koltun. Lagrangian fluid simulation with continuous convolutions. In International Conference Learning Representations, 2020.
12

---

[43] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez. E ukasz Kaiser, and Illia Polosukhin. Attention all you need. In NeurIPS, 2017.
[44] Nils Wandel, Michael Weinmann, Michael Neidlin, and Reinhard Klein. Spline-pinn: Approaching pdes without data using fast, innsors-i hermite-spline cnns, 2022.
[45] Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, al. Scientific discovery in the age of artificial intelligence. Nature, 2023.
[46] Sifan Wang, Yujun Teng. and Paris Perdikaris. Understanding and mitigating gradient pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing. 2020.
[47] Sifan Wang. Xinling Yu, and Paris Perdikaris. When and why pinns fail train: A neural tangent kernel perspective. Journal Computational PhysicsF. 2020.
[48] E Weinan and Ting Yu. The deep ritz method: A deep learning- numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 2017.
[49] F.M. White. Fluid Mechanics. McGraw-Hill series in mechanical engineering. 2011.
[50] R Wille. Karman vortex streets. Advances in Applied Mechanics, 1960.
[51] Nick Winovich, Karthik Ramani, and Guang Lin. Convpde-uq: Convolutional neural networks with quantified uncertainty for heterogeneous elliptic partial differential equations on varied domains. J. Compur. Phys., 394(C):2332279, oct 2019.
[52] Haixu Wu, Tengge Hu, Huakun Luo, Jianmin Wang, and Mingsheng Long. Solving nigh-dimensione pdes with latent spectral models. In International Conference on Machine Learning, 2023.
[53] Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Yan Liu. On layer normalization in the transformer architecture, 2020.
[54] Yunyang Xiong, Zhanpeng Zeng. Rudrasis Chakraborty. Mingxing Tan, Glenn M. Fung. Yin Li, and Vikas Singh. Nyströmformer: pyström-hased algorithm for approximating self-attention. Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
[55] Cagatay Yildiz, Markus Heinonen, and Harri Lahdesmaki. Ode2vae: Deep generative second order odes with bayesian neural networks. H. Wallach, H. Larochelle, A. Beygelzimer, E. d'Alché-Buc, Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32, Curran Associates, Inc., 2019.
13

---

A Implementation Details
This section provides the implementation details of DeepLag, including the configurations of model hyperparamelers and the concrete design of modules.
A.1
Hyperparameters
Detailed model configurations of DeepLag are listed in Table 7 Zero Zero-padding padding is only used in the Ocean Current dataset to ensure the exact division in downsampling.
Table 7: Model configurations for Deeplag.
Model Designs
Hyperparameters
Values
Number of observation steps
10
Number of scales L
4
Eulerian-Lage Lagraiinn
Sample Points at each scale (M, Mr.)
(512, 128, 32, 8)
Recurrent Network
Downsample Ratio [Pe+al
0.5
Channels of each scale [Ci, Cr.}
(64, 128, 256, 256)
Paddings for Ocean Current dataset
(12, 20)
Heads in Cross Aetes-nttnnion Channels per head in Cross-Attention
EuLag Block
64
A.2
Sampling and multiscale architecture in overall framework
A.2.1
Sampling to initialize Lagrangian particles
At the first predicting step, the position of Lagrangian particles to track is initialized by the dynamic sampling module consisting of dynamics extracting and sampling.
The ConvNet(() to extract dynamics Given the Eulerian input field {u (x) JCD, at the I-th scale, the ConvNet(.) operation in Eq. 4 is to extract the local dynamics around each Eulerian observation point with Conv(), BatchNorm() and ReLU(( layers, which can be formalized follows:
ConvNet(u,) = Conv ReLU (BatchNorm ( (Conv(fu) (Con)((((()))))))))) 1 from 1 to L. (12)
Here, the output channel of the outermost Conv is 1.
The Sample(-) to select key particles Given the probability distribution matrix Sx(x)xCD, and
the number of particles to sample Mi at the (-th scale, we choose the particles for further tracking by Multinomial() without replacement:
Sample(S)) = Multinomial(S). Mi), from 1 to L.
(13)
A.2.2 Multiscale architecture
Multiscale modeling utilized in Deeplagg as represented in Figure 2 d), where we need to maintain multiscale deep Eulerian features follows:
Encoder Given Eulerian fluid observation (x))x) at the t-th time step. (U(=PP1):001)(x)>>>)))))))))))))
previous time steps and the 0-1 boundary-geometry mask (m(x))xcD where 1 indicates the border and the unreachable area (like pillars Bounded Navier-Stokes), the Encode() operation is to project original fluid properties in physical domain to deep representations with linear layer and position embedding, which can be formalized as follows:
n Linear Cooncat (x)
PosEmmeediing.
(14)
For position embedding, we concatenate two additional channels to the input (three for 3D Smoke), representing normalized (z, y) coordinates (or (z, y, 2) for 3D Smoke).
14

---

Decoder Given evolved Eulerian deep representations in the finest scale (x)) at the + 1)-th time step, the Decode() operation to project deep representations back to predicted fluid properties with two linear layers and a GeLU activation [16], which can be formalized as follows:
Linear
[GLLU ( [Lincar( ( (f
(x)) (x) (x))x))))
(15)
Downsample Given Eulerian deep representations (EED, at the (-th scale, the Down() operation is to concentrate local information of deep representations into a smaller feature map at the (I [+ 1)-th scale with MaxPooling( and Conv() layers, which can be formalized as follows:
++
Conv MMxxPooling ((xx()))x<D))) from 1 to (L 1).
(16)
Upsample Given the evolved Eulerian deep representations [ (x))xCD, at the 1-th scale and {ult1 (x)+x) at the (1 + 1) scale, respectively, the Up() operation to fuse information on corresponding position between two adjacent scales of deep representations into a feature map the 1-th scale with Interpolate() operation and Conv() layers, which can be formalized as follows:
Concat ([ [Inteepplatte (1u77)()))<cc)))) ( (xfr)()))))) from (1- (1-1))m (17)
1/41-
Conv
A.3 EuLag Block
The LagTooEu()) process Hereafter we denote by h, |Pr for the concatenated representations. As we stated in subsection 3.2 the Lagrangian-guide Eulerian feature evolving process, short as LagTo BagToFu(-)). aggregates information from Lagrangian description to guide the update of Eulerian field with single Transformer layer at each scale:
u441 = u + M, (p!)
= u + PFN(LagToEn hillp!. h((l)))) + LogToouu-Atn((e h/llp'. h!llp!)), (18)
where LagnoouuAtttnu,,
pp,, h(]lp() ) is described as Eq. (7. 1 {1,2,.... ........................ L}. Moreover, use the pre-normalization [33 technique for numerical stability.
u/, h.
The EuToLag(.) process Similar to above, we acquire the new particle position p and the global dynamics h global,t41 in the Fulerian-condittonee particle tracking by Fulerian-LLagrangian
cross-attention:
h 1 || = h//l P + FuToTTaghh//lli "'41, w/-1)
h² p + +EaToLag-AAtt(h/ Ip., u/- 1
aatteooooaaaa--thhhaha Ip u w/+1))
(19)
where E 21, - LJ. Then, we interpolate from u/41 by p'41 to be the local dynamics bf and Aggregate it with the global dynamics h bilobal.e41 using a Linear layer:
h
Aggregate (Interpolate(ue+1. Pr-1), hglobal.r+1)
(20)
Linear Concat (h (b Interpolate p())
A.4 Metrics
Relative 1,2 We use the relative I.2 as the primary metric for all three tasks. Compared to MSE, Relative L2 is less influenced by outliers and is more robust. For given n steps 2D predictions R HxWxXXX and their corresponding ground truth x of the
IR. xx www. xn or 3D predictions
same size, the relative L2 can be expressed as:
||
&
Relative L2
(21)
x
where || 1112 represents the L2 norm.
15

---

B Comparison Between DeepLag and Classical ML Methods
DeepLag differs significantly from classical ML approaches like Neural ODE 44 the comparison Table
as highlighted in
Table 8: Comparison between DeepLag and classical ML model
Feature
DeepLag
Neural ODE
ODE Specification
Not required
Explicityy specified
Computational Paradigm
Data-driven
Model-driven
Attention Mechanism
Utilizes attention
Typically not utilized
Integration of Lagrangian Dynamics Input-Output Mapping
Integrated
Operates solcly in Euler space Simple, single-variable ODE
Complex, high-dimensional PDE
While Neural ODE methods explicitly specify ODEs, fluid dynamics governed by multi-variable PDEs, rendering the ODE formulation inadequate. Moreover, DeepLag is data-driven and does require explicit ODE specification. Notably, we are the first to employ attention mechanisms for computing Lagrangian dynamics, which closely resemble operators in both deep learning and numerical Lagrangian methods. Additionally, Deeplaag integrates both Fulerian and Lagrangian frameworks, whereas ODF-based methods operate solely in the Fulerian space. Finally, Deepl lag models the complex mapping of high-dimensional spatiotemporal PDEs, which significantly more intricate than the simpler processes modeled by single-variable (either temporal or spatial) ODEs.
C More Details about the Benchmarks
C.1 Bounded Navier-Stokes
Here we provide some details about the benchmark Bounded Navier-Stokes. Just as its name implies, the motion of dye is simulated from the Navier-Stokes equation incompressible fluid. 2000 sequences with spatial resolution of 128 x 128 are generated for training and 200 new sequences are used for the test. We supplement important details indicating the difficulty of the Bounded Navier-Stokes dataset follows:
About the Reynolds number The Reynolds number the dataset 256. At this Reynolds number, attached vortices dissipate and form a boundary layer separation. The downstream flow field behind the cylinder becomes unsteady, with vortices shedding periodically on both sides the cylinder's rear edge, resulting in the well-known phenomenon of Kármán vortex street [50). Additionally, due to the presence multiple cylinders within the flow field and obstruction from downstream cylinders, more complex flow phenomena than flow around cylinder occur, challenging the model's capacity.
Differences of data sequences Our data generation method involves running simulations for over 10a steps after setting the initial conditions of the flow field. We then randomly select starting time step and extract several frames as an example, which are further randomly divided into training and testing sets. The positions of the cylinders samples, which can simulate a scenario like bridge pillars in a torrential river. Due to the highly unsteady nature of the flow field, the flow patterns observed by the model appear significantly different.
fixed, but the initial condition varies in different
Numerical method used in data generation We utilized the Finite Difference Method (MAC Method) with CIP (Constrained Interpolation Profile) as the Advection Scheme for numerical simu- lations, implemented by 36). This high-order interpolation-constie numerical dissipation, enhancing the accuracy and reliability of numerical simulations.
format effectively reduces
C.2
Ocean Current
Some important details related to the Ocean Current benchmark are attached here. Learning ocean current patterns from data and providing long-term forecasts are of great significance for disaster prevention and mitigation, which motivates to focus this problem.
16

---

The procedures to make this dataset are as follows. First, we downloaded daily sea reanalysis data [5] from 2011 2020 provided by the ECMWF and selected five basic variables on the sea surface to construct the dataset, including velocity, salinity, potential temperature, and height above the geoid, which are necessary to identify the ocean state. Then, we crop 180 x 300 sub-area on the North Pacific from the global record, corresponding to a 375km 2725kmm2mmm region. In total, this dataset consists of 3,653 frames, where the first 3000 frames are used for training, and the last 600 frames used for testing. The training task is to predict the future current of 10 days based on the past 10 days' observation, after which we performed 30 days of inference with the trained model to examine the long-term stability of DeepLag.
C.3 3D Smoke
To verify our model effectiveness in this complex setting of the high-dimensional tanglesome molecular interaction, we also generate a 3D fluid dataset for the experiment. This benchmark consists of a scenario where smoke flows under the influence of buoyancy in a three-dimensional bounding box. This process is governed by the incompressible Navier-Stoke equation and the advection equation fluid. 1000 sequences are generated for training, and 200 new samples are used for testing. Each case is in the resolution of 32
D Training Curves
Thansnn Cure
Twirie
P00
3:33
Caledd Traclace
OXTT
Lid
1.50
Noc
3.
0.00
513
Figure 8: Training curve comparison among all the models on Bounded Navier-Stokes dataset, Ocean Current, and 3D Smoke dataset.
We provide training curves on Bounded Navier-Stokes, Ocean Current, and 3D Smoke datasets in Figure 8 We can observe that DeepLag presents favorable training robustness and converges the fastest on the Bounded Navier-Stokes dataset.
E Analysis on the Parameter Count and Performance Difference
In Table in our paper, we present three different datasets with variations in data type, number of variables, number of dimensions, and spatial resolution. Specifically, the 2D fluid and 3D fluid represent entirely different dynamic systems, so it is normal for different baselines to perform inconsistently across different benchmarks. In the following results, patch_sizz of all models are set to 1 for fair comparison. For instance, U-Net performs well on datasets with distinct mulriscale attributes (such as the Bounded Navier-Stokes dataset), while transformer- based methods excel on datasets with broad practical ranges (like the Ocean Current dataset) or high dimensions (such the 3D Smoke dataset), where attention mechanisms can effectively model global features. Hence, our model's 's ability to handle multiscale and global modeling simultaneously highlights the challenge of achieving consistent state-of of-he----eeheeeeee the--htet performance. Below are the parameter statistics for DeepLag and all baseline models across each benchmark, along with some experiments conducted to ensure parity in the parameter count for each model.
E.1 Analysis on Bounded Navier-Stokes benchmark
The parameter quantities for each model on the Bounded Navier-Stoker benchmark are shown in Table 9 When attempting to further increase the parameter count of smaller models for fair comparison, we encountered CUDA Out-of-memory errors. Specifically, these errors occurred when we increased the Dynamic_nen sarccc__eeeaaa_a___eeeeeeeeeee __ne__layerrreete_l of Vortex from to 50, resulting in parameter count of 171,761, but encountered CUDA Out of Memory issues.
17

---

Table 9: Model parameter summary for Bounded Navier-Stokes dataset.
Model
#Parameter
UNet
17,311,489
FNO
4,744.513
Galerkin-Transe Transfomer
3,917,600
Vortex 7
96,321
GNOT [14
3,736,713
LSM 52
19,188,033
FactFormer
20
5,477,185
DeepLag (Ous)
10,895,771
E.2 Analysis on Ocean Current benchmark
Table 10: Model parameter summary for Ocean Current dataset.
Model
#Parameter
UNet 32
17,314,565
FNO 21
4.747,589
Galerkin Transformer
3
3,941,156
GNOT 14
993,101
Vortex 7
96,321
LSM 52
19,191,109
FactFormer 20
5,482,821
DeepLag (Ours)
13,782,431
The parameter quantities for each model the Ocean Current dataset are summarized in Table 10 When attempting to further increase the parameter count smaller models for fair comparison, we encountered CUDA Out-of -ut-of-memory errors. Specifically, these errors occurred when increased the Dynamic, pel.nee__al_ag of Vortex from to 50, resulting in a parameter count 171,761, but encountered CUDA Out of Memory issues. Similarly, for GNOT, increasing the latent_ __dim from 64 to 96 resulted parameter count 1,659, 179, yet this configuration encountered CUDA Out of Memory errors.
E.3
Analysis on 3D Smoke benchmark
Table 11: Model parameter summary for 3D Smoke dataset.
Model
#Parameter
UNet 32
51,892,292
FNO 21
56,651, 6,0,
Galerkin
Balerkin-Transfore Transformer
3
28,867,88
GNOT 14
1,450,768
LSM
52
25,937, 732
FactFormer
20
1,840,004
DeepLag (Ours)
19,526.827
The parameter quantities for each model the 3D Smoke dataset shown in Table I1 When exper- imenting with increasing the parameter count certain models for fair comparison, we encountered CUDA Out-of-memon errors. Specifically, these errors occurred when we increased the latent_da of GNOT from 64 to 96, resulting in a parameter count 1,450,76s 768 for GNOT, which encountered CUDA Out of Memory issues. Similarly, when adjusting the encoder. transformer. layer of FactFormer from 3 to 13, the parameter count reached 7,382,084, yet this configuration encountered CUDA Out of Memory errors.
Figure our paper, along with the provided tables above, illustrate our rigorous comparison and efforts to standardize model parameters for a fair comparison. However, due to excessive GPU
18

---

Table 12: ACC results on the Ocean Current dataset and the curve of timewise ACC, Both ACC averaged from 10 prediction steps and ACC of the last prediction frame are recorded, A higher ACC value indicates better performance. Relative promotion is also calculated.
MODEL
AVG. ACC (1)
LAST. ACC (1)
Tinewise ACC (top
0.69
U-NET 32
0.0916
0.0888
0.00
Fachoonne
FNO 21
0.0883
0.0853
Decplag
0.45
GALERKIN TRANSFORMER 3
0.0896
0,0848
0,60
GNOT [14
0.4725
0,4557
0473
LSM [52]
0.2378
0,2305
0.470
FACTFORMER 20
0,4789
0,4633
DEEPLAG (OURS)
0,4820
0.4694
PROMOTION
0.6%
1.3%
Predict Step
memory consumption of intermediate results some baseline models, we could not conduct direct performance comparisons under matched parameter conditions. The ability of our model to efficiently utilize GPU memory resources is a valuable aspect of practical applications.
F ACC Metric on Ocean Current
Latitude-weighter Anomaly Correlation Coefficient In meteorology, directly calculating the correlation between predictions and ground truth may obtain misleadingly high values because of the seasonal variations. To subtract the climate average from both the forecast and the ground truth, we utilize the Anomaly Correlation Coefficient to verify the forecast and observations. Moreover, since the observation grids are equally spaced in longitude, and the size of the different grids related to the latitude, we calculate the latitude- -atihtde-eiightd Anomaly Correlation Coefficient, which can be formalized as:
Ens (o)
ACC(u,()
(22)
Eis (at();) (81 x Eig Lat(d:) (xT).)
where v represents certain observed variable, &i is the prediction of ground truth position i j and forecast time x' = x x represents the difference between x and the climatology x, that is, the long-term mean of observations in the dataset. Lat(Ø;) = Ni X
cos where
0o8
04
NLa 180 and de is the latitude of the i-th row of output.
ACC result on the Ocean Current benchmark Notably, DeepLag also excels in the ACC metric, shown in Table [2] which can better quantify the model prediction skill. As shown in the timewise ACC curve, DeepLag consistently achieves the highest ACC and holds more significant advantages in long-term prediction. Since ACC is calculated with long- long-term climate statistics, further improvements become increasingly challenging as it increases, which highlights the value of DeepLag.
G Visual Results for Learnable Sampling
1520
20
Figure 9: Visualization of the pointwise variance of vorticity (middle) and the sampling probability distribution (right) learned by DeepLag. We plot the log(S) here for a better view.
To demonstrate the effectiveness of our learnable probability, we visualize its distribution with respect to the pointwise variance of vorticity, which is directly proportional to the local complexity of
19

---

fluid dynamics, in Figure 9 It is evident that our sampling module tends to prioritize focusing on and sampling particles within regions having higher dynamical complexity, such as the wake flow and Karman vortex (50) near the domain borders and behind the pillar. 'The showcase in Figure T also demonstrates that the tracked particle can well present the dynamics of a certain area. This observation underscores that our design is very flexible at adapting to various complex boundary conditions and can effectively guide the model to track the most crucial particles, enhancing its performance in capturing the fine details in the wake zone where turbulence and vortex form.
H More Showcases
As supplement to the main text, we provide more showcases here for comparison (Figure 10|5]
Ground Truth
IT = 20)
DeepLag (uurs)
U-Net
FNO
Galerkin Transformer
GNOT
LSM
FactFormer
Prediction Keror
Figure 10: Showcases of the Bounded Navier-Stokes dataset.
T-11 11
T-12
T-
7-14
T- T-16
T-16 16
T- 12
T-15 18
T-19
T-20
Grund freth
Pretited
Einx
Figure 11: Showcases of DeepLag the Bounded Navier- dataset.
Ground Truth (T' 20)
DeepLag (Ours)
U-Net
FNO
GNOT
LSM
PactPormer
Prediction
Figure 12: Showeases of the 3D Smoke dataset.
20

---

Treth
Fald
Map
Figure 13: Showcases of DeepLag on the 3D Smoke dataset.
Ground Troth IT 200
Decplang (Ours)
U-Nee
PND
Prodiction Error
Galerkin Trunsdemorr
GNOT
TSM
FactPurmer
Figure 14: Showcases the Ocean Current dataset.
T-
T-
T+
Omeed
Vold
Map
Predicee
Foee
Figure 15: Showcases of DeepLag on the Ocean Current dataset.
21

---

I Result of Long-term Prediction
1.1 Reason for predicting 10 steps
Predicting the last 10 steps from the first 10 steps input is convention (refer to FNO where = le 5. T = 20 is the setring, and later several baselines using the NS dataset also followed this setting). We did this to follow the convention and for ease of comparison. Additionally, the Ocean Current dataset has a one-day interval between every two frames, and predicting 10 days ahead is already long horizon.
1.2 Results of long-term rollout
We conducted experiments for long-term (extrapolation beyond hundreds of frames) predictions. Accurately, we utilize trained 10-frame prediction models perform 100-frame prediction. Our reason for not directly training a model an autoregressive paradigm to predict the next 100 frames due to insufficient memory capacity and the issue of gradient explosion vanishing. The results are as follows. The best result is bold and the second is underlined. The reason we did not run the 3D Smoke dataset is that it very challenging load 110 frames of large 3D data once, which overwhelms our machine. Extra video results are in supplementary materials, which effectively illustrate the performance and consistency of our model's predictions.
I.2.1 Bounded Navier-Stokes
Quantitive results As depicted in T3 the Deeplag lag model still outperforms the strong base- lines the Bounded Havier-Stokes dataset in predicting 100 frames into the future. With Relative L2 of 0.1493, DeepLag achieves superior performance compared to all baselines. Additionally, the performance trends of all models visually illustrated in Figure 20 revealing DeepLag 's ability to maintain lower error growth rates over time, particularly in long-term predictions. This out- come suggests that the Lagrangian particle- purticle-based approach adopted by DeepLag effectively captures dynamic information, contributing to its robust forecasting capability in fluid dynamics modeling.
Table 13: Performance comparison for predicting 100 frames the Bounded Navier-Stoke dataset. Relative L, is recorded. For clarity, the best result is in bold and the second best is underlined. Promotion represents the relative promotion of our model w.r.t the second best model.
Model
Relative L2 (1)
U-Net
32
0.1529
FNO 21
0.4244
Dalerkin-Transfommee
0.3010
GNOT [4]
0,2489
LSM [52
0.1511
FactFormer
20
0.1691
Deepl.ag (Ours)
0.1493
promotion
1.2%
Showcases To visually evaluate the predictive capabilities of models on long-term, present a showcase of last frame comparisons and time-wise prediction with error map in Figure T6 illustrating the long-term rollout performance on the Bounded Navier-Stokes dataset. Notably, DeepLag demonstrates remarkable accuracy in capturing complex flow phenomena, accurately predicting the formation and evolution of vortices, particularly the Kármán vortex street behind the upper left pillar. In contrast, U-Net and LSM exhibit moderate success in predicting the central vortex but struggle with accurately reproducing the density distribution of the flow field, indicated by the error maps. FactFormer, however, shows subpar performance on this benchmark, likely due its reliance on spatial factorization, which may not effectively handle irregular boundary conditions. These findings underscore the advantages of our which enables simultaneous prediction of dynamics and density, contributing to more accurate and comprehensive fluid modeling and forecasting capabilities.
co-design approach,
22

---

Ground Truth (T+ 100)
DeepLag (Ours)
U-Net
LSM
Facter
19)
Prediction Error
Figure 16: Showcases comparison between the most competitive models of long-term rollout on the Bounded Navier-Stokes benchmark.
T+0
T+10
T+20
T+30
T+40
T+50
T+60
T+70
T+80
T+90
T+100
Ground Truth
Predict- ion
Error Map
Figure 17: Timewise showcases of DeepLag of long- term rollout the Bounded Navier-Stoker benchmark.
T+0
T+10
T+20
T+30
T+10
T+50
T+60
T+70
T+80
T+90
T+100
Ground Truth
Prediet
Error Map
Figure 18: Timewise showcases FactFormer of the long-term rollout on the Bounded Navier-Stokes benchmark.
23

---

T+0
T+10
T+22
T+30
T+40
T+50
T+60
T+70
T+80
T+90
T+100
Ground Truth
Predict ion
Error Map
Figure 19: Timewise showcases of LSM of the long-term rollout on the Bounded Navier-Stokes benchmark.
T+0
T+10
T+20
T+30
T+40
T+50
T+60
T+70
T+80
T+90
T+100
Ground Truth
Prediet- ion
Error Map
Figure 20: Timewise showcases of U-Net of long-term rollout the Bounded Navier-Stokes benchmark.
I,2.2
Ocean Current
Quantitive results We present comparison of results for the Ocean Current dataset in Table 14 which includes relative L2, Last frame ACC, and Average ACC metrics. DeepLag maintains its superiority, achieving the lowest relative L2 among all models, with an 8.7% promotion compared to the second-bes model. These findings highlight DeepLag's robust performance predicting real- world large-scale fluid dynamics, which often exhibit inherent stochasticity. Furthermore, Deeplag outperforms other models in ACC metrics, indicating its superior predictive capability. This is further conroborated by the timewise ACC curve, where DeepLag consistently demonstrates the highest ACC values, particularly in long-term predictions.
Showcases To visually assess the long-term forecasting performance of each model, we showcase the last frame predictions along with their errors, and the time-wise prediction errors for each model Visually,
DeepLag predictions closely resemble the Ground Truth compared to other models, demonstrating robust long-term extrapolation capabilities and accurate capture of the Kuroshio pattern [37]. can be observed that FactFormer and LSM exhibit relatively large errors. while GNOT tends to average and loses fine texture details. However, DeepLag does not suffer from these issues.
in Figure 21
25 22
I.3 Examination on the turbulent kinetic energy spectrum
In the field of fluid mechanics, simulation results that better adhere intrinsic physical laws are sometimes more valuable than those with smaller pointwise errors, often reflected in frequency domain analysis. validate this point and to measure long-term forecasting ability, we introduced metric on time-averaged turbulent statistics, namely the turbulent kinetic energy spectrum (TKES). Specifically, we computed the MAE and RMSE of TKES for the Bounded Navier-Stokes dataset, which exhibits the most prominent turbulent characteristics, and plotted the line graphs of the error of

---

Ground Truth (T = 100)
DeepLag (Ours)
U-Net
LSM
Fact
Prediction Error
Figure 21: Showcases comparison between the most competitive models of the long-term rollout on the Ocean Current benchmark.
T+0
T+10
T+20
T+30
T+40
T+50
T+60
T+70
T+80
T+90
T+100
Ground Truth
Prediet
lon
Ervor
Map
Figure 22: Timewise showcases DeepLag of the long-term rollout on the Ocean Current benchmark.
T+0
T+10
T+20
T+30
T+10
T+50
T+60
T+70
T+80
T+90
T+100
Ground Truth
Predict- Son
Ervor Map
Figure 23: Timewise showcases of FactFormer of the long-term rollout on the Ocean Current benchmark.
T+0
T+10
T+20
T+30
T+10
T+50
T+60
T+70
T+80
T+90
T+100
Ground Trath
Prodict:
Error
Map
Figure 24: Timewise showcases of LSM of the long-term rollout on the Ocean Current benchmark.
T+0
T+10
T+20
T+30
T+40
T+50
T+60
T+70
T+80
T+90
T+100
Ground
Truth
Predict-
Jon
Ervor Map
Figure 25: Timewise showcases of GNOT of the long-term term rollout on the Bounded Navier-Stoker benchmark.
25

---

Table 14: Relative L2 for predicting 100 frames, Last frame ACC, and Average ACC
Model
Relative L.2 (L) Last frame ACC (1)
Average ACC (f)
U-Net
32
0.0764
0.0197
0.0503
FNO
21
0.1675
0.0041
0.0448
Galerkin- Transformer
NaN
NaN
NaN
GNOT 14
0.0513
0.173
0,3586
LSM 22
0.0616
0.0793
0.1650
FactFormer
20
0.0460
0.2647
0.3864
DeepLag (Ours)
0.0423
0.2890
0.4041
promotion
8.7%
9.2%
4.6%
wave number and TKE, as shown Table can be observed both numerically and visually that DeepLag consistently outperforms various baselines to different extents.
Table 15:
Turbulent kinetic energy spectrum (TKES) results on the Bounded Navier-Stoker Both the plot of error of TKES (left) and MAE and RMSE of TKES (right) are presented. A lower TKES error value indicates better performance. Relative promotion is also calculated.
MODEL
MAE (4)
RMSE (4)
U-NET
3.888C+06
3.349E+07
FNO
7.364F+06
7.191
5.956E+06
5.950E+07
GNOT
1.631E+08
LSM
3,403E=06
3.040E+07
PACTFORMER
4.495E+06
3.9688+07
DEEPLAG (OURS)
3.0528+06
2.716E+07
PROMOTION
11.5%
11.9%
Visual Result of the Boundary Condition Generalization Experiment
To demonstrate the generalizing performance of DeepLag, visualize the showcases in Figure 26 between Deepl ag and the best baseline, U-Net. This intuitive result further shows that Deepl adaptively generalizes well on new domains and handles complex boundaries well.
20)
DeepLog
U-Net
48-
0
20
23
129
Figure 26: The visual comparison of zero-shot inference on the new Bounded Navier-Stokes. Stokes.
26

---

K Visualization of the Movement of the Particles
We visualize the particle movements on the Bounded Navier-Stokes dataset Figure 27 Given the dataset's s complex and frequent motion patterns, we have plotted particle offsets between consecutive frames, which effectively reflect instantaneous particle movements, depicted, DeepLag can still learn intuitive and reasonable motion without a standard physical velocity field. Notably, DeepLag performs best in this dataset, highlighting the benefits learning Lagrangian dynamics.
20-
20
30
60
66
100
120
40
60
20
40
40-
80
80
20 40 60 00
100
120
80
320
40
100
120
Figure 27: Visualization of the particle movements on the Bounded Navier-Stokes dataset. The plotted
particle offsets between consecutive frames effectively reflect instantaneous particle movements. As depicted the red boxes, DeepLag can accurately capture the motion mode of complex dynamics like Kármán vortex street.
27

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: In the abstract and introduction, we point out that the Eulerian fluid prediction problem is addressed in this paper, and we tackle by proposing a new architecture that incorporate Lagrangian dynamics to guide the Fulerian field update, which is our main contribution. Moreover, we made an exhaustive and fair comparison and the experiment result proves the effectiveness claim in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In the Section we discussed the hyperparameter M (number of Lagrangian tracking particles) can not automatically adjust to a certain task, which may be not convenient or out of the box for practical application of DeepLag. Plus, all the experiment are repeated 3 times to confirm our claims. Comparison between the benchmarks show that our model and baselines are slower at data of finer resolution, which is inevitable.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
28

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Except for the basic relations (Eq. I 2 between the Eulerian and La- grangian perspectives we only draw inspirations from, no theorem derived or explicitly used in Deepl.ag, since they are usually connected to specific physical equation which will harmful to the generalization of our approach. All the knowledge from fluid dynamies or numerical method are properly referenced.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: To ensure a fair and reproducible result, we unify the settings of all models and conduct experiments 3 times under identical environments (hardware and software), and we make elaborate explanation on every detail of model module implementation the main text and appendix, along with metric and visual results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
29

---

(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e-g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All the data are public. Code is available at this repository: https: htpps:///tttin git Con/thum1/DeepLas We have provided detailed descriptions for these benchmarks and
implementations reproduce.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results. See the NeurlPS code and data submission guidelines (https tttccc/pppo. http:/uullccgdudie httpe//ggiddee/eeeeubbbiisss wwt.eeSuwwissiooooolloooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all the training and test details at the start of section 4 and details on the benchmarks in Appendix C which enough understand and reproduce the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
30

---

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Error bars are not reported because it would be too computationally expensive. That being said, every number results comes from multiple identical experiments with no statistical significant change.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
we provide the information of GPU that we use in training, and we included model efficiency comparison in subsection 4.4
Justification: the start of section 4
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics stttps://neurie htttpp////aarrgss.cc/ppll///c/cp
Answer: [Yes]
Justification: We have carefully read the NeurIPS Code of Ethics and our research conforms with it in every respect. We are sure that the anonymity is preserved during the whole procedure.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
31

---

If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid-
eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: Beyond the importance mentioned at the setup of the Ocean Current dataset, positive societal impacts of a deep model for fluid prediction is in various domain including environmental protection (like pollution control or disaster prevention and mitigation), energy sector (like renewable energy utilization resource management), economic benefits (maritime navigation or agriculture) and scientific advancement itself. Since does not involve the generation of subjective content which many people concern, there is seldom negative societal impacts unless society become overly reliant or blindly trust on automated predictions.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: All data sources in this paper have open access, and the model does not involve the generation of subjective content, their is risk for misuse like this type.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided relcasing unsafe images.
32

---

We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All assets used to generate dataset are properly cited in the paper. The url of repository for Bounded Navier-Stoke generation is hhttps://ggtthu: com/ takah29/
2d-
where no licence found.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of in the package should be provided. For popular datasets, papprssithcomm thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's ' creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We included the demo videos in the supplementary materials.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
33

---

Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
34


Reviews:
[{"review_id": "23xJw21bfJ", "reviewer": "Reviewer_UohQ", "summary": "The paper introduces a novel approach to predicting fluid dynamics by integrating both Lagrangian and Eulerian paradigms. The model, named ‘DeepLag,’ utilizes transformer blocks to process and integrate information from Eulerian and Lagrangian perspectives. Initially, DeepLag predicts the future state of the Eulerian space. Subsequently, it uses this prediction to infer the Lagrangian movements of key tracked particles. This methodology allows the Lagrangian dynamics to inform and guide the evolution of the Eulerian predictions, enhancing the overall prediction accuracy.\n\nDeepLag is evaluated against various baselines across three diverse datasets, including simulated bounded Navier-Stokes equations, real-world ocean currents, and 3D smoke dynamics. These experiments demonstrate the model’s effectiveness in handling both 2D and 3D fluid dynamics in simulated and real-world scenarios.", "questions": "- How to interpret the difference between the left and right parts of Fig. 1?\n- Is there supervision over the Lagrangian space, or does it only happen in the Euler space?\n- Would interchanging the positions of the LagToEu and EuToLag blocks affect the model’s performance or learning dynamics? If so, how?\n- Given that Table 5 shows only a minimal performance boost from the EuToLag attention, how should this be interpreted in the context of the model’s overall efficiency and effectiveness?\n- In the Bounded Navier-Stokes experiments, are the positions of the cylinders consistent across all trials within the dataset, or do they vary?", "limitations": "Limitations have been addressed", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The DeepLag model introduces a novel framework that effectively combines Eulerian and Lagrangian perspectives, allowing for dynamic propagation in both spaces. This dual approach is innovative in fluid dynamics modeling, enhancing the prediction accuracy by leveraging the strengths of both paradigms.\n- The model’s effectiveness is rigorously tested across three challenging datasets—bounded Navier-Stokes equations, real-world ocean currents, and 3D smoke dynamics. The experiments demonstrate superior performance not only in standard scenarios but also in both short-term and long-term prediction tasks, highlighting the model’s versatility and robustness.\n- One of the standout features of DeepLag is its ability to provide interpretable results by showcasing individual particle trajectories via Lagrangian dynamics. This aspect is particularly valuable as it not only enhances the understanding of fluid movements but also aids in validating the model’s predictions through visual and traceable particle paths.", "weaknesses": "- The model’s reliance on extensive hyperparameter tuning for particle tracking could pose challenges in terms of replicability and efficiency. This complexity might limit the accessibility of the model for practical applications without substantial computational resources.\n- Additionally, the particle tracking mechanism struggles to maintain focus on particles near the domain borders. This limitation is particularly concerning in scenarios where significant dynamic changes occur near these borders, such as the presence of obstacles. The inability to track these dynamics could potentially lead to incomplete or inaccurate modeling of fluid behavior in such areas.\n- The paper contains a few grammatical errors that, while minor, could detract from its overall professional presentation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks to Reviewer UohQ for the detailed review and suggestions.\n\n> **Weakness1:** The model’s reliance on extensive hyperparameter tuning for particle tracking could pose challenges.\n\n**The only hyperparameter needed for tuning is the total number of the tracking particles in the finest scale.** Please recall $\\underline{\\text{Table 6 in Appendix A.1}}$ that the number of tracking particles at each scale has an exponential decrease w.r.t the index of that scale. The ablation in $\\underline{\\text{Table 5}}$ shows that the performance always tends to incline as we simply increase this number (which means our DeepLag is scalable), therefore **the hyperparameter choice depends more on the limitation of computing resources rather than blind searching**.\n\n> **Weakness2:** The particle tracking mechanism struggles to maintain focus on particles near the domain borders.\n\nWe respectfully point out that our model works well on boundary particles. $\\underline{\\text{Figure 9 in Appendix G}}$ shows that **DeepLag is apt to sample more points in the complex-dynamics area**, which is near and behind the column obstacles. Plus, the red boxes in $\\underline{\\text{Figure 10,14 in Appendix H and Figure 16 in Appendix I}}$ indicate that **DeepLag is good at capturing the fine details in the wake zone where turbulence and vortex form**.\n\n> **Weakness3:** The paper contains a few grammatical errors.\n\nThank you for your thorough review of our paper and for bringing to our attention the grammatical errors that may detract from its professional presentation. We sincerely apologize for these inaccuracies and have already taken steps to correct them. Specifically, we will address errors such as the **article misusage** in $\\underline{\\text{Line 3 of paper}}$ (\"an Eulerian perspective\" corrected to \"the Eulerian perspective\") and the **preposition error** in $\\underline{\\text{Line 70 of paper}}$ (\"tracking a certain particle with initial position $\\mathbf{s}_0$ with its displacement $\\mathbf{d} = \\mathbf{d}(\\mathbf{s}_0, \\mathit{t})$\" corrected to \"tracking a particular particle from initial position $\\mathbf{s}_0$ by its displacement $\\mathbf{d} = \\mathbf{d}(\\mathbf{s}_0, \\mathit{t})$\"). These corrections are crucial for maintaining the professional standard of our presentation. Moving forward, we will conduct a comprehensive proofreading to ensure that all grammatical errors are identified and rectified.\n\n> **Q1:** About the difference between the two parts of Fig. 1\n\nAs indicated in the figure caption, Fig. 1 presents two distinct visualizations. The left part depicts the trajectories of Lagrangian particles overlaid on the mean state observed from the Eulerian perspective. In contrast, the right part displays successive Eulerian frames with scattered positions of tracked particles. These visualizations align with the description provided in $\\underline{\\text{Lines 42-43 of paper}}$, where **the trajectories of fluid motion are more visibly represented through the dynamic Lagrangian view compared to the density variations observed in static Eulerian grids**. This underscores our motivation to incorporate and study Lagrangian dynamics.\n\n> **Q2:** About the supervision over the Lagrangian space.\n\nThere is no supervision signal in the Lagrangian space due to the following reasons. Firstly, the widely used large-scale and fine-grained data in hydromechanics is all Eulerian since they're relatively easier to collect, represent and store. Secondly, it's impractical to track almost infinite particles precisely in real-world applications like ocean study, so **there is a need for deep models to unsupervised extract Lagrangian information from Eulerian data effectively and efficiently**. Thirdly, following the convention and the setting of baselines allow us a fair and clear comparison.\n\n> **Q3:** The result of interchanging the positions of the LagToEu and EuToLag blocks.\n\n| Dataset / Model | Original | Swap EuToLag and LagToEu |\n| - | - | - |\n| Bounded N-S     |  0.0543  |          0.0545          |\n| 3D Smoke        |  0.0378  |          0.0378          |\n\nWe conducted experiments as per your request, swapping the positions of the EuToLag and LagToEu blocks and validating their effects on both 2D (Bounded Navier-Stokes) and 3D (3D Smoke) datasets. The experimental results indicate that **there was minimal change in performance, suggesting that the flow of information between Eulerian and Lagrangian perspectives is bidirectional**. This insensitivity to the order in which information is transferred between the two perspectives underscores the robustness of our approach. These findings further support our statement in $\\underline{\\text{Line 73}}$ of the manuscript that \"**Two perspectives are constitutionally equivalent**,\" which is substantiated both theoretically ($\\underline{\\text{Eq. (1) and (2) in Section 2.1}}$​) and intuitively.\n\n> **Q4:** Explanation on the ablation of the EuToLag attention.\n\nThe relatively small promotion of EuToLag attention can be interpreted as follows. \n\nFirstly, **EuToLag attention itself adds minimal computational overhead ($O(n)$) and only marginally increases the training time (from 1030s/epoch to 1150s/epoch)**. The primary computational load lies in the LagToEu module, which processes dense Eulerian data using patch embedding and patch recovery. Therefore, EuToLag attention does not significantly impact efficiency.\n\nSecondly, **in scenarios where precision demands capturing global information for effectiveness, integrating EuToLag attention remains justified**. In this paper, we mainly focus on effective Eulerian-Lagrangian collaborative modeling and efficiency optimization is a promising future work.\n\n> **Q5:** Positions of the cylinders in the Bounded Navier-Stokes Benchmark.\n\nThe positions of the cylinders are fixed, but the initial condition varies in different samples, which can simulate a scenario like bridge pillars in a torrential river.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oyOW7nUb84", "reviewer": "Reviewer_rCcn", "summary": "The authors propose a novel neural network architecture in order to leverage the advantages of the eulerian and lagrangian formalisms for fluid prediction. The so called \"EuLag Block\" acts on an eulerian grid based representation of the fluid as well as on a lagrangian particle based representation and enables information flow between both representations through 2 attention blocks:\n1. The \"LagToEu Attention\" block allows the network to pass information from the lagrangian particle based representation to the eulerian grid based representation\n2. The \"EuToLag Attention\" block allows the network to pass information from the eulerian grid based representation to the lagrangian particle based representation\nOn top of these 2 attention blocks, a bilinear interpolation module is used to extract information about local dynamics from the eulerian representation, which is then combined with information about global dynamics from all particles to compute the update of the particle based representation. The distribution of the initial particle positions is generated by a learnable sampling module.\nThe authors evaluate the resulting DeepLag architecture on 3 different fluid prediction tasks and outperform several state of the art methods. Furthermore, ablation studies were performed in order to investigate the effect of different numbers of particles, both attention blocks as well as the learnable sampling module.", "questions": "- How would the particle movements (shown in Figure 5 on the right) look for the Bounded Navier-Stokes dataset (Figure 4)? Would the particles still follow the velocity field although the velocity field is not considered as a field variable in this experiment but only the dye concetration?\n- Could the particle movements be used to extract information about the velocity field in the Bounded Navier-Stokes dataset?\n- Often in fluid dynamics (e.g. \"Stable fluids\" by Jos Stam or \"Accelerating Eulerian Fluid Simulation With Convolutional Networks\" by Tompson et al), solving the pressure field is done in the eulerian frame and requires a global solution. However, dealing with the advection term using a particle tracer requires only the local velocity field. Isn't the global attention for the particle updates a \"slightly wasteful\" overkill?\n- How does the positional embedding look like?\n- How well does your method generalize to new domains?", "limitations": "- How would your method scale to larger domains? I'm wondering if the fully-connected layer in the dynamic sampling module could become a bottleneck for larger domain as it seems to scale quadratically with the domain size.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- the qualitative results look convincing (especially the long term stability shown in the videos)\n- fairly extensive quantitative comparison to other network architectures and ablation studies\n- interesting idea to use LagToEu / EuToLag attention blocks in order to reduce the squared complexity of the attention mechanism wrt the domain size in the eulerian frame of reference to a linear complexity wrt the domain size and the number of lagrangian particles (especially if the number of particles is chosen to be relatively small).", "weaknesses": "- Reference to \"Accelerating Eulerian Fluid Simulation With Convolutional Networks\" by Tompson et al is missing. They use a particle tracer to deal with the advection term in the lagrangian frame of reference and train a CNN to perform a \"pressure projection step\" in the eulerian frame. They showed impressive smoke simulations in 3D similar to the experiments shown in section 4.3. Thus, I'm not sure if I can fully agree with the claim in line 55-57 that this is the first deep fluid prediction model that explicitly combines Eulerian and Lagrangian frameworks.\n- Regarding the efficiency analysis (Figure 7) it would be interesting to see how a U-Net would compare to DeepLag when upscaled up to a similar running time.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Sincerely thank Reviewer rCcn for the detailed review and insightful suggestions.\n\n> **Weakness1:** On the difference between DeepLag and FluidNet [Tompson et al, ICML 2017].\n\nThanks for your recommendation on related work and rigorous questions. We will cite FluidNet in the revised paper. However, we have to say that FluidNet is distinct from DeepLag:\n\n|                              | FluidNet                  | DeepLag                                            |\n| - | - | - |\n| **Main Design**     | Replaces Eulerian pressure projection with CNN module | Combines Eulerian and Lagrangian in Deep Learning |\n| **Method for Lagrangian**    | **Numerical method** | Learning **deep Lagrangian info for end-to-end modeling** |\n\nThe key design of FluidNet is replacing one step of classical method with a learned module implemented by a surrogate CNN for accumulation. However, in the Lagrangian perspective, it still employs the traditional numerical approach (Maccormack method).\n\n**In contrast, DeepLag represents an approach that integrates both Eulerian and Lagrangian within a pure deep learning framework**. This allows for more generalized modeling of Lagrangian dynamics across multiple elements, demonstrating its capability as a fully deep learning-based solution capable of end-to-end autonomous learning. \n\nWe will revise the claim you concern to **\"explicitly combines Eulerian and Lagrangian in a deep learning framework\"** for scientific rigor.\n\n> **Weakness2:** Comparison between DeepLag and U-Net under similar running time.\n\n| Model       | No. of Parameters | GPU Memory | Running Time | Relative L2 |\n| - | - | - | - | - |\n| U-Net-scale |    336,609,604    |   13672M   |  601s/epoch  |     NaN     |\n| DeepLag     |    19,526,827     |   12112M   |  845s/epoch  | **0.0378**  |\n\nAs per your request and following $\\underline{\\text{Figure 7}}$, we conducted experiments on the 3D Smoke dataset that scaled up U-Net to make a comparison between DeepLag and U-Net under similar running time. Concretely, we add more conv layers into the standard U-Net and increase the latent dimension. **The results show that too many parameters overwhelm U-Net, indicating that it has a shortcoming in scalability**. \n\n\n> **Q1:** Visualization of the particle movements of the Bounded Navier-Stokes dataset.\n\nBased on your request, we have visualized the particle movements on the Bounded Navier-Stokes dataset in $\\underline{\\text{Figure 1 in PDF of Global Response}}$. Given the dataset's complex and frequent motion patterns, we have plotted particle offsets between consecutive frames, which effectively reflect instantaneous particle movements.\n\nAs depicted, DeepLag still can learn intuitive and reasonable motion. However, there is a slight decrease in overall quality compared to Ocean Current, which may be because of lacking standard physical quantities. However, it is worth noticing that DeepLag performs best in this dataset, which verifies the benefits of learned Lagrangian dynamics in improving prediction.\n\n> **Q2:** Velocity field extraction from particle movements of the Bounded N-S dataset.\n\nExtracting dense velocity fields from the particle perspective in the Bounded N-S dataset is challenging due to the sparse nature of particles, which is necessary to reduce computational complexity. However, **key pathways of complex motion regions can be identified through adaptive sampling, which already proves valuable for understanding fluid dynamics**.\n\n> **Q3:** Is the EuToLag attention an overkill?\n\nIn theory, relying solely on local velocity fields for particle updates seems sufficient. However, in practical fluid dynamics, the exact velocity of fluid motion is often unknown, and due to the incompressibility of fluids, disturbances from distant regions can influence local dynamics. Therefore, incorporating global attention mechanisms ensures comprehensive modeling, addressing scenarios where distant influences play a role. Take Ocean Current as an instance, the test set Relative L2 increases to 0.0264 from 0.0250 after removing the EuToLag attention (at the 20th training epoch).\n\nMoreover, the EuToLag attention introduces minimal computational overhead ($O(n)$) and slightly increases training time (from 1030s/epoch to 1150s/epoch on Bounded N-S), while yielding noticeable performance improvements. Thus, we believe that EuToLag attention is not 'overkill'.\n\n> **Q4:** About the positional embedding.\n\nWe concatenate two additional channels to the input (three for 3D Smoke), representing normalized (x, y) coordinates (or (x, y, z) for 3D Smoke).\n\n> **Q5:** Generalization on new domains.\n\nTo verify the generalizing ability of DeepLag, we ran a zero-shot test with the old model checkpoint on a newly generated Bounded N-S dataset which has a different number, position and size of obstacles. \n\nDeepLag still has a ~7% promotion w.r.t the best baseline, U-Net, on relative L2 (DeepLag: 0.203, U-Net: 0.217). The visual comparison ($\\underline{\\text{Figure 2 in Global Response PDF}}$) between the two models further shows that DeepLag adaptively generalizes well on new domains.\n\n> **Limitations:** On the scalability to larger domains of the fully-connected layer in the dynamic sampling module.\n\nSorry for the vague description. The \"fully-connected layer\" in $\\underline{\\text{Line 151}}$ **refers to channel dimension MLP rather than spatial.** Therefore, the quadratical complexity for a fully-connected layer doesn't exist.\n\nFurther, we trained a new model on a 256*256 Bounded N-S (4x larger domain) as requested, which still has a ~17% promotion w.r.t U-Net on relative L2 (DeepLag: 0.051, U-Net: 0.060), the comparison below also shows that the increase of the time complexity is minor, underscoring the scalability of DeepLag.\n\n| Resolution | GPU Memory | Running Time | Relative L2 |\n| - | - | - | - |\n| 128*128    |   5420M    | ~1150s/epoch |   0.0543    |\n| 256*256    |   13916M   | ~1300s/epoch |   0.0514    |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RF4XCOgYqq", "reviewer": "Reviewer_PBwE", "summary": "In this paper, the author presents a Lagrangian-Eulerian hybrid paradigm to address the complexities of fluid dynamics. Instead of relying only on Eulerian observations to predict future states, we introduce DeepLag, which uncovers hidden Lagrangian dynamics within the fluid by tracking the movements of adaptively sampled key particles. In experiments, DeepLag show better performance in three demanding fluid prediction tasks both in 2D and 3D, as well as simulated and real-world fluids.", "questions": "See weaknesses.", "limitations": "Yes. The author discuss the limitations in the paper.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The idea of integrating Lagrangian tracking into the deep model for assisting Eulerian based fluid prediction sounds novel.\n- The LagToEu Attention and EuToLag Attention for exchanging information between Eulerian and Lagrangian view is intuitive.\n- The proposed method shows better performance on all three test cases comparing to all baselines.", "weaknesses": "- In the Bounded Navier Stokes, the performance of 10 Frames is significant better than 30 Frames, while for the Ocean Current case, the results of  long-term rollout shows gives better result. Just wondering what is the reason between this discrepancy? Is the proposed method better than in shot-term or long-term tasks?\n- How does the proposed method handle the complex boundaries? e.g., how to impose different boundary conditions into the framework?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank Reviewer PBwE for the detailed review and insightful suggestions.\n\n> **Weakness1:** Explanation of the performance difference between datasets. \"Just wondering what is the reason between this discrepancy? Is the proposed method better than in short-term or long-term tasks?\"\n\nThank you for your thorough review of our paper. The observed performance differences between the Bounded Navier-Stokes and Ocean Current datasets stem from the diversity in dataset characteristics rather than any weaknesses in our model. \n\nThe Bounded Navier-Stokes dataset represents a smaller scale with irregular variations, whereas the Ocean Current dataset spans larger scales with periodicity and oceanic averages. Therefore, for ocean data, achieving accurate predictions on average state can also yield excellent results over the long term.\n\n> **Weakness2:** About the boundary condition processing in the framework. \"How does the proposed method handle the complex boundaries? e.g., how to impose different boundary conditions into the framework?\"\n\nActually, DeepLag is very flexible at adapting to various complex boundary conditions. As depicted in $\\underline{\\text{Figure 9 in Appendix G}}$, whether the boundary is known or unknown, simple or intricate, the dynamic sampling module which is optimized with the pointwise variance of vorticity could recognize the boundary area, wavefront, wake and far-field zone from the features contained in the input distribution. By the way, for datasets whose boundary is known (like the pillars in Bounded Navier-Stokes), we could concatenate the boundary mask as a new channel into model input as a guide to enhance the performance of DeepLag and the baselines.\n\nTo verify the generalizing ability on the new domain of DeepLag, we ran a zero-shot test with the old model checkpoint on a newly generated Bounded N-S dataset which has a different number, position and size of obstacles. **DeepLag still has a ~7% promotion w.r.t the best baseline, U-Net, on relative L2 (DeepLag: 0.203, U-Net: 0.217). The visual comparison** ($\\underline{\\text{Figure 2 in Global Response PDF}}$) **between the two models further shows that DeepLag adaptively generalizes well on new domains and handles complex boundaries well**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jQQy272LfG", "reviewer": "Reviewer_PRfM", "summary": "The authors propose DeepLag as an approach to simulating Eulerian fluid dynamics, which makes use of Eulerian-Lagrangian co-design to improve performance. In particular, the idea is to transfer information back and forth between the Eulerian grid and initially randomly placed Lagrangian particles, which themselves exist on multiple but connected discretization scales. The intuition is that Lagrangian particles are better at capturing interactions with boundaries and, in general, tracking the location of material elements, e.g., smoke. Empirically, the gain in performance over existing models is consistent and around 10%. The limitations of the approach are its algorithmic complexity and hyperparameters.", "questions": "- Fig. 7: in this figure, it looks like the U-Net has fewer parameters than DeepLag, but in lines 345-346, as well as Table 10, you state the opposite. Can you explain that?", "limitations": "- Thinking of Occam's Razor, it is questionable how many people would adopt the proposed approach, as it requires (a) a significantly more involved implementation effort than a typical U-Net, (b) tuning various hyperparameters like number of scales and number of particles per scale, and (c) is ~5x slower (Fig 7), while offering on average 10% performance improvement. Could the authors add some more hints on where their approach would be of practical interest? I would appreciate extending section 5 into, for example, one paragraph dedicated to the summary/strengths/applications, and one with the limitation/future work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The idea of having \"attention\" between the Eulerian grid and Lagrangian particles is interesting and seems to work well.\n- The multi-scale approach makes a lot of sense to map fine details to the coarser resolution. Not sure whether this is only a strength though, as this means higher algorithmic complexity of the model.\n- The approach by construction offers Lagrangian particle trajectories, which might be useful in e.g. ocean dynamics research", "weaknesses": "- **Factual mistakes**: the authors make some very crude false statements here and there, and I'm questioning whether a person or LLM wrote sections of the manuscript, particularly the introduction.\n    - Lines 24-25: \"curse of dimensionality\" is NOT the reason for high computational cost in fluid dynamics! Please open the Wikipedia article on the curse of dimensionality and modify the sentence in the paper.\n    - Lines 40-42 state that \"[Lagrangian approach] helps get around the CFL condition\", which is absolutely wrong.\n- **Related work taxonomy**: in section 2.2, a categorization of neural solvers is presented; however, this grouping of methods is new to me and, as much as I know, not what the community typically uses. \n    - The 1st group, \"ODE-based Generative Models\", refers to some rather old and outdated work, \n    - the 2nd group, \"Neural PDE Solvers\", actually only talks about Physic-Informed Neural Networks, and \n    - the 3rd group, \"Neural Operators for PDE\", basically lists all methods that I would call modern neural PDE solvers and what is typically referred to as neural operator learning (DeepONet, FNO, etc.) is mentioned along with GNNs and CNNs.  \n\n    See PDE Bench [1] and PDE Arena [2] for a more modern categorization of common PDE learning approaches.\n- **Ablations**: I'm missing an ablation on (a) the number of scales and (b) overall model size. Judging by Fig 7, in which the U-Net is 5x faster and also smaller, I wonder how a U-Net with similar (a) parameter count or (b) runtime would perform. I'm a bit worried that the claimed good performance might be just a bad hyperparameter choice of the baselines.\n\n---\n[1] Takamoto et al., \"PDEBench: An Extensive Benchmark for Scientific Machine Learning\", NeurIPS D&B 2022  \n[2] Gupta and Brandstetter, \"Towards Multi-spatiotemporal-scale Generalized PDE Modeling\", 2022", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Special thanks to Reviewer PRfM for their detailed review and insightful suggestions.\n\n> **strength2:** On the algorithmic complexity brought by the multi-scale design.\n\nPlease recall $\\underline{\\text{Table 6 in Appendix A.1}}$ that the number of tracking particles at each scale has an exponential decrease w.r.t the index of that scale, resulting in adding more coarser scales only bringing about a minor fraction of extra computing, while the overall complexity doesn't vary too much. For quantitative results, please refer to **Weakness3** below.\n\n> **Weakness1:** Justification for certain statements.\n\nThank you for carefully reviewing our paper and sorry for the controversial expressions, however, you can rest assured that every word of this paper is written by ourselves not LLM. \n\n- **Regarding 'curse of dimensionality'**, we acknowledge your concern. While the curse of dimensionality itself refers to specific challenges in high-dimensional data spaces and may not directly cause high computational costs in fluid dynamics, we understand that computational challenges in fluid dynamics primarily stem from the complex numerical methods required to solve PDEs over large and intricate grids. We will revise the text to better clarify this distinction.\n\n- **Regarding the statement of CFL condition**. We agree with you that the CFL condition is essential for ensuring numerical stability and accuracy in fluid dynamics simulations, regardless of the method employed. What we are trying to say is that Lagrangian approaches for Eulerian fluid prediction like Semi-Lagrangian method offer more flexibility in time step sizes (e.g. adaptive time stepping) than Eulerian methods. We will remove this claim for scientific rigor.\n\n> **Weakness2:** Reorganize the related works.\n\nIn the previous paper, we follow the survey$^{[1]}$, where 'Neural Solvers' indicates Physic-Informed Neural Networks (PINNs) and 'Neural Operators' refers to the models that map the input function space to the output space, such as FNOs.\n\nFollowing your suggestion, we would rephrase category names to 'Classical ML methods', 'Physic-Informed Neural Networks (PINNs)' and 'Neural Operators'.\n\n------\n\n[1] Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications, arXiv 2022\n\n> **Weakness3:** More ablations.\n\nAs per your request, we conducted further ablations, yielding the following results:\n\n| No. of scales | Relative L2 | Latent dim.   | Relative L2 |\n| - | - | - | - |\n| 1             |   0.0789    | 16            |   0.0656    |\n| 2             |   0.0658    | 32            |   0.0594    |\n| 4 (original)  | **0.0543**  | 64 (original) | **0.0543**  |\n| 5             |   0.0554    | 128           |   0.0614    |\n\nRegarding (a) the number of scales, **increasing the number of scales generally improves performance up to a point**, after which diminishing returns are observed. As for (b) overall model size, **increasing the latent dimension tends to improve performance**, but too many parameters can be unnecessary for the model, as seen in the ablations.\n\nAdditionally, in response to your concern about comparing U-Net with a similar parameter count or runtime to DeepLag, please refer to $\\underline{\\text{Appendix E}}$. **U-Net consistently exhibits significantly larger parameter counts across all datasets compared to DeepLag**, rendering such experiments unnecessary. Furthermore, experiments scaling U-Net to match DeepLag's runtime show:\n\n| Model       | No. of Parameters | GPU Memory | Running Time | Relative L2 |\n| - | - | - | - | - |\n| U-Net-scale |    336,609,604    |   13672M   |  601s/epoch  |     NaN     |\n| DeepLag     |    19,526,827     |   12112M   |  845s/epoch  | **0.0378**  |\n\nIn conclusion, augmenting U-Net with additional convolutional layers and increasing the latent dimension shows that **too many parameters can overwhelm U-Net, indicating a scalability limitation.**\n\n> **Q1:** Elaboration on Figure 7 and Table 10.\n\nThank you for your inquiry regarding the discrepancy between $\\underline{\\text{Figure 7}}$ and $\\underline{\\text{Lines 345-346}}$, as well as $\\underline{\\text{Table 10}}$. Please note that $\\underline{\\text{Figure 7}}$ depicts **memory usage**, while $\\underline{\\text{Table 10}}$ and $\\underline{\\text{Lines 345-346}}$ refer to **parameter counts**. \n\nThus, the confusion arises from the difference in metrics presented. Specifically, U-Net is characterized by a larger number of parameters but a smaller memory footprint, as indicated in our findings. We acknowledge the need for clarity in distinguishing these metrics and will ensure that this distinction is clearly labeled in our revised manuscript.\n\n> **Limitations:** About Occam's Razor, practical interest and writing advice.\n\nMany thanks for your valuable suggestion. Firstly, we want to highlight several properties of our model that correspond to your mentioned question:\n\n(a) About implementation effort: DeepLag can be implemented solely based on Pytorch modules, which may not be as hard as you thought.\n\n(b) About hyperparameter:  The primary consideration is the total number of tracking particles in the finest scale. Ablations in $\\underline{\\text{Table 5 in Section 4.4}}$ demonstrate that performance generally improves with an increased number of particles. Therefore, you can decide this up to your computation resource.\n\n(c) About efficiency: We have to say that DeepLag is slower than U-Net. However, the performance improvement brought by DeepLag can be acknowledged in some applications, such as wind tunnel test, which prefers high precision to efficiency. \n\nIn addition, the learned particle trajectory can be visualized to help research understand fluid dynamics.\n\nFollowing your suggestion, we will extend the Conclusion to discuss more about strengths/applications and limitations/future work. One promising direction is to speed up DeepLag with advanced attention mechanisms, such as FlashAttention.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LWo5de1c5M", "reviewer": "Reviewer_Kve3", "summary": "Real-world processes exhibit multi-scale spatio-temporal dynamics. Not all of this dynamics is accurately modeled by Eulerian (i.e., field-based) modeling of scientific processes and sometimes the fine-grained patterns are only able to be modeled by Lagrangian paradigms. However explicit Lagrangian-only modeling is costly and hence this paper proposes a deep learning based surrogate model that is able to jointly model the Eulerian and Lagrangian perspectives by proposing a novel `EuLag' deep learning block. The authors demonstrate the performance of the proposed architecture on multiple fluid-dynamics related tasks involving multi-phase flows.", "questions": "1. Why have the authors not compared to a Lagrange-Only model like that of [1] for fluid flow modeling. Since the proposed paradigm is an Euler-Lagrange paradigm, baseline with models that only in the Eulerian domain seems somewhat incomplete and for completeness, comparison with one particle based deep-learning model (like [1]) should be included.\n\n\n## References\n\n[1] Sanchez-Gonzalez A, Godwin J, Pfaff T, Ying R, Leskovec J, Battaglia P. Learning to simulate complex physics with graph networks. InInternational conference on machine learning 2020 Nov 21 (pp. 8459-8468). PMLR.", "limitations": "Authors have addressed the potential limitations of their work sufficiently well.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The authors develop a well-motivated and novel deep learning model architecture to combine the Eulerian and Lagrangian computational modeling paradigms thereby leading to improved modeling capacity owing to modeling across a larger variety of scales.\n\n- The proposed model demonstrates performance improvements compared to state-of-the-art models like the Fourier-Neural Operator and transformer based architectures (also a central feature in the proposed EuLag block) like the Galerkin Transformer. Overall, the experimental results are convincing.", "weaknesses": "- More information regarding the experimental setup needs to be included in the main text of the paper. Currently, the main body of the paper (especially related to the experimental setup and the dataset descriptions, model description) cannot stand on its own without the appendix. At least the full description of the model architecture (i.e., upsampling, down sampling and other critical operations) should appear in the main text.\n\n- Comparison with a Lagrange-only model is necessary in addition to the current field-based prediction models for a more holistic experimental comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Special thanks to Reviewer Kve3 for their detailed review and insightful suggestions, your dedication to evaluating our work despite your busy schedule is greatly appreciated.\n\n> **Weakness1:** About the detailed content selected to represent in the main body of the paper.\n\nThank you for your valuable feedback. We acknowledge that the current presentation may pose difficulties for readers seeking detailed technical insights, especially regarding the experimental setup, dataset descriptions, and model architecture. Due to space constraints, our initial focus was on presenting the core ideas and results, with specific implementation details relegated to the appendix. We will address this in our revisions by integrating a comprehensive description of the experimental setup, dataset characteristics, and detailed model architecture directly into the main body of the paper.\n\n> **Weakness2:** Comparison with a Lagrange-only model.\n\nThank you for your comment. We appreciate your suggestion regarding including a comparison with a Lagrange-only model alongside existing field-based prediction models for a more comprehensive experimental evaluation, however, we have the following reasons not to do so:\n\nFirstly, it's important to note that existing benchmarks $^{[1,2]}$ and real-world observational datasets, such as the Ocean Current dataset by ECMWF, predominantly represent Eulerian perspectives due to their ease of collection, representation, and storage.\n\nSecondly, in practical applications like ocean studies, precisely tracking nearly infinite particles is often impractical. Therefore, there is a critical need for deep learning models that can effectively and efficiently extract Lagrangian information from Eulerian data in an unsupervised manner.\n\nThirdly, adhering to established conventions and baseline settings allows for a fair and transparent comparison among different models. As outlined in $\\underline{\\text{Lines 77-78}}$ of our paper, our research is focused on Eulerian data for fluid prediction, aligning with practical applications and existing benchmarks.\n\nRegarding the comparison with a Lagrange-only model, it's important to clarify that such models operate purely on Lagrangian data inputs and outputs. This does not align with the specific problem statement and datasets used in our study, making it challenging to conduct a fair comparison.\n\nIn conclusion, while we appreciate the suggestion, our experimental design focuses on evaluating field-based prediction models within the context of Eulerian data. This approach ensures consistency and relevance to practical applications and existing benchmarks.\n\n------\n\n[1] Takamoto et al., \"PDEBench: An Extensive Benchmark for Scientific Machine Learning\", NeurIPS D&B 2022\n[2] Gupta and Brandstetter, \"Towards Multi-spatiotemporal-scale Generalized PDE Modeling\", 2022\n\n> **Q1:** Comparison with one particle-based deep-learning model\n\nThank you for your inquiry. We appreciate your suggestion regarding comparing our approach with a Lagrange-Only model.\n\nAs outlined in **Weakness2** of our paper, the decision not to include a comparison with a Lagrange-Only model stems from several considerations. Primarily, our research focuses on leveraging both Eulerian and Lagrangian perspectives within a unified framework, termed the Euler-Lagrange paradigm, currently on Eulerian data. This dual perspective allows for a more comprehensive understanding and prediction of fluid dynamics, which is the primary contribution and focus of our study.\n\nHowever, we acknowledge the merit of your suggestion to include comparisons with models that solely operate within the Lagrangian domain. In response to your suggestion, we have contemplated a dual approach where Eulerian information supplements purely Lagrangian particle data for prediction tasks. This idea for proving the effectiveness of the Euler-Lagrange paradigm on Lagrangian data, while relevant and insightful, falls beyond the scope of the current paper but will be considered for future work. We appreciate your valuable input and will duly note it for further exploration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## Global Response and Summary of Revisions

We sincerely thank all the reviewers for their insightful reviews and valuable comments, which are instructive for us to improve our paper further.

This paper proposes a **new deep Eulerian-Lagrangian hybrid paradigm for fluid prediction**, DeepLag, which can benefit from the advantages of both perspectives. DeepLag is derived from the equivalent and complementary theory of the two perspectives as a practically novel end-to-end deep learning framework, which **consistently boosts the performance of three difficult benchmarks, covering both 2D and 3D fluid.**. Detailed visualizations and model analysis are provided.

The reviewers generally held positive opinions of our paper, in that the proposed method is "**innovative**", "**effectively combines Eulerian and Lagrangian perspectives**", "**particularly valuable**", and "**versatile and robust**"; The model’s effectiveness is "**rigorously tested**" and demonstrated "**superior performance not only in standard scenarios but also in both short-term and long-term prediction tasks**".

The reviewers also raised insightful concerns and constructive questions. We made every effort to address all of them by providing detailed clarification, requested results and visualizations. Here is the summary of the major revisions:

- **Explain hyperparameter choosing for particle tracking would not pose challenges (Reviewer UohQ, PRfM):** Firstly, we point out that the only hyperparameter needed for tuning is the total number of the tracking particles in the finest scale. Secondly, we recall the hyperparameter details in the appendix and the ablation result to explain that the hyperparameter choice depends more on the limitation of computing resources rather than blind searching. 
- **Clarify the difference between DeepLag and FluidNet [ICML 2017] (Reviewer rCcn):** Firstly, we emphasize that FluidNet is mainly based on the numerical method, which replaces Eulerian pressure projection with a CNN module, while DeepLag integrates both Eulerian and Lagrangian perspectives within a pure deep learning framework, which learns deep Eulerian and Lagrangian info for end-to-end modeling. Secondly, we will revise the claim to "explicitly combines Eulerian and Lagrangian in a deep learning framework" for scientific rigor.
- **Experiments of scaling up U-Net and comparing it with Deepag under similar running time (Reviewer rCcn, PRfM):** Following the reviewers' request and the efficiency analysis in our paper, we conducted experiments on the 3D Smoke dataset that scaling up U-Net to make a comparison between DeepLag and U-Net under similar running time. The results show that too many parameters overwhelm U-Net, indicating that it has a shortcoming in scalability.
- **Experiments of generalization on new domains and boundary conditions (Reviewer rCcn, PBwE):** Following the reviewers' concern, we ran a zero-shot test with the model checkpoint trained on the original dataset, on a newly generated Bounded N-S dataset which has a different number, position and size of obstacles, to verify the generalizing ability of DeepLag. Both quantitive and visual result shows that DeepLag adaptively generalizes well on new domains and complex boundary conditions.
- **Correct writing issues (Reviewer UohQ, Kve3):** We sincerely thank the reviewers for the valuable feedback from the careful reviewers and explain that the limitations on space constraints make us move some detailed content to the appendix. And we promise to conduct a comprehensive proofreading to ensure that all grammatical errors are resolved and include a comprehensive description of the experimental setup, dataset, and model architecture in the main text.

The valuable suggestions from reviewers are very helpful for us to revise the paper to a better shape. All the above revisions will be included in the final paper. We'd be very happy to answer any further questions.

Looking forward to the reviewer's feedback.

#### **The mentioned materials are included in the following PDE file.**

- **Figure 1 (Reviewer rCcn)**: Visualization of the particle movements on the Bounded Navier-Stokes dataset.
- **Figure 2 (Reviewer rCcn, PBwE)**: The visual comparison of zero-shot inference on the new Bounded Navier-Stokes dataset between the best baseline, U-Net, and DeepLag.

Meta-review: The submitted paper proposes a method for the prediction of fluid dynamics by particle tracking and integration of Eulerian and Lagrangian perspectives. The five experts appreciated novelty, evaluation, performance, and interpretability.

Some minor weaknesses were identified (hyper parameter tuning, dependency on tracking, behavior close to boundaries, some  writing issues and technical problems in formulations), but which do not severely impact readiness for publication, in particular given the authors' answers, which made some reviewers increase their scores.

The AC concurs.
Decision: Accept (poster)