Dual-frame Fluid Motion Estimation with Test-time Optimization and Zero-divergence Loss

Authors: yifei  zhang
huan-ang  gao
zhou  jiang
hao  zhao
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 09:15:55

Abstract: 3D particle tracking velocimetry (PTV) is a key technique for analyzing turbulent flow, one of the most challenging computational problems of our century. At the core of 3D PTV is the dual-frame fluid motion estimation algorithm, which tracks particles across two consecutive frames. Recently, deep learning-based methods have achieved impressive accuracy in dual-frame fluid motion estimation; however, they heavily depend on large volumes of labeled data. In this paper, we introduce a new method that is **completely self-supervised and notably outperforms its fully-supervised counterparts while requiring only 1\% of the training samples (without labels) used by previous methods.** Our method features a novel zero-divergence loss that is specific to the domain of turbulent flow. Inspired by the success of splat operation in high-dimensional filtering and random fields, we propose a splat-based implementation for this loss which is both efficient and effective. The self-supervised nature of our method naturally supports test-time optimization, leading to the development of a tailored Dynamic Velocimetry Enhancer (DVE) module. We demonstrate that strong cross-domain robustness is achieved through test-time optimization on unseen leave-one-out synthetic domains and real physical/biological domains. Code, data and models are available at [https://github.com/Forrest-110/FluidMotionNet](https://github.com/Forrest-110/FluidMotionNet).

Main paper: 
 introduction: 
 Measuring and understanding turbulent fluid flow is a crucial problem as it is ubiquitous in various aspects of our lives, both in nature [27; 37; 20] and within our engineered society [28; 81; 35; 26; 69; 101; 68; 93] . Throughout history, flow visualization techniques have played a vital role in quantifying and analyzing turbulent flow [4; 60; 16; 76] . Among existing flow visualization techniques, 3D particle tracking velocimetry (3D PTV), which tracks individual particles between consecutive frames, distinguishes itself with high spatial resolution and precise measurement of velocity vectors [32; 3] . Before the advent of deep learning, PTV algorithms primarily focus on designing hand-crafted features for particle matching [62; 12; 99] . With the onset of deep learning, deep neural networks (like DeepPTV [50] and GotFlow3D [53] ) have been introduced to solve this task. The core algorithm of 3D PTV is dual-frame fluid motion estimation, as illustrated in Fig. 1 .However, it is important to note that existing state-of-the-art (SOTA) dual-frame fluid motion estimation algorithms (shown in left two panels of Fig. 1 ) have a limitation: they require a large amount of fully annotated data. It is known that deep learning generally requires a substantial amount of Figure 1 : Paradigm Shift: Given two frames of flow particles X t and X t+∆t , DeepPTV [50] adopts a two-stage network for large-and small-scale motion refinement. GotFlow3D [53] trains a correspondence learning network and an RNN-based residual prediction network. They are trained in a fully supervised manner with annotated data and do not support test-time optimization. Our purely self-supervised method diverges from these approaches and employs DVE (see Sec. 3.3) for on-the-fly test-time optimization. The "Snowflake" denotes frozen weights. in-domain data for optimal results. This demand poses a significant challenge to the AI4Science field, especially in 3D PTV where collecting suitable data is complicated due to the need for precisely selected tracer particles, tailored illumination, and camera settings [63] . Additionally, certain scenarios like flow fields under unique geometric conditions or cytoplasmic flows in disease contexts are rare, making it nearly impossible to compile a comprehensive dataset.To alleviate the aforementioned challenge, in this paper, we introduce a novel purely self-supervised framework with test-time optimization designed specifically for dual-frame fluid motion estimation in the 3D PTV process, as highlighted in the right panel of Fig. 1 . Concerning the intrinsic difficulties associated with PTV data collection, especially in specialized contexts [6; 74; 64] , we consider working under a limited size of dataset, as little as 1% typically used by existing fully-supervised methods (notably without accessing labels). Fluid particles have special physical properties, for which we resort to the inherent zero-divergence principle of incompressible fluid velocity fields and design a novel zero-divergence self-supervised loss tailored for fluid. Per implementation, we introduce the successful idea of splat in high-dimensional filtering [2] and random fields [38] and design a splat-based zero-divergence loss that is both efficient and effective. Moreover, since our method is self-supervised, it naturally supports test-time optimization. Thus we introduce a module termed Dynamic Velocimetry Enhancer (DVE), shown in the right panel of Fig. 1 , which optimizes the initial predicted flow during test-time based on the specific input data on the fly, ensuring an improved level of accuracy across various testing scenarios. This is critical for cross-domain robustness. The difficulty in collecting diverse PTV data leads to the common practice of using synthetic datasets. However, since synthetic data is generated based on hand-crafted priors, it cannot accurately represent specific real-world distributions, resulting in models that lack the necessary cross-domain robustness for practical applications.Through comprehensive experiments, we demonstrate that our purely self-supervised framework (right panel of Fig. 1 ) significantly outperforms its fully-supervised counterparts (left two panels of Fig. 1 ), even under data-constrained conditions (using as low as 1% data). Additionally, our cross-domain robustness analyses confirm the framework's intrinsic ability to generalize to unseen domains, including leave-one-out synthetic domains and real-world physical/biological domains, underscoring the practical utility of our approach for real-world 3D PTV applications.To summarize, our main contributions are: 1. A novel self-supervised framework with test-time optimization for dual-frame fluid motion estimation, surpassing fully-supervised methods with minimal samples (as low as 1%). 2. A splat-based zero-divergence self-supervised loss for fluid dynamics, which is both efficient and effective. 3. A test-time optimization module named Dynamic Velocimetry Enhancer (DVE) that significantly improves cross-domain robustness.2 Related WorkQuestion: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes]Justification: Please see Abstract and Introduction.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
test-time optimization and test-time domain adaptation: 
 Test-time optimization, also known as test-time refinement (TTR), exploits the inherent structure of data in a self-supervised manner without requiring ground truth labels [14; 80; 24; 21] . Applications of TTR include point cloud registration [25] , depth estimation [7; 9] , object recognition [80; 89] , human motion capture [86] , and segmentation with user feedback [73; 77; 31] . Test-time domain adaptation (TTA), a specific form of TTR, adapts a model trained on a source domain to a new target domain using an unsupervised loss function based on the target distribution [90; 54; 104] . One significant challenge in TTR is achieving per-sample adaptation at test time without compromising inference efficiency. Recent studies [85; 67] have explored using generative models to enable efficient test-time adaptation. In this work, we introduce our DVE module (Sec. 3.3), which conducts test-time optimization but maintains efficiency when compared with prior methods.We include this section because our research is closely related to point-based, learning-driven scene flow estimation from point clouds-a key component in understanding scenes through point clouds [8; 84; 43; 22] . Both areas of study concentrate on learning flows or correspondences from two frames of data [97; 102; 103; 100; 55] . Advances in scene flow estimation have been driven by benchmarks such as KITTI Scene Flow [59] and FlyingThings3D [58] . Drawing from the related field of optical flow [15; 30; 79; 82] , recent developments in scene flow estimation utilize methods including encoder-decoder architectures [23; 57] , multi-scale representations [11; 44; 94] , recurrent modules [36; 83; 92] , and other strategies [42; 70] .Self-supervision and Test-time Optimization for Scene Flow. Self-supervised learning has received attention for scene flow estimation from point cloud data [95; 61; 5; 40; 46; 75; 45] and monocular images [29; 10; 105] . PointPwcNet [95] introduces cycle consistency loss, inspiring Mittal et al. [61] to incorporate it with nearest neighbor loss for establishing point cloud correspondence. This method also employs Chamfer Distance [19] , smoothness constraints, and Laplacian regularization for selfsupervision. SLIM [5] addresses self-supervised scene flow estimation and motion segmentation simultaneously. Flowstep3d [36] uses a soft point matching module for pairwise point correspondence. Self-supervision naturally supports test-time optimization. Pontes et al. [66] eschew model training for real-time optimization by minimizing the graph Laplacian over source points to enforce rigid flow. Li et al. [48] replace the explicit graph with a neural prior using a coordinate-based MLP to implicitly regularize the flow field. SCOOP [40] combines pre-training on a subset of data to learn soft correspondences and secures initial flows with optimization-based refinement steps.Our work is distinct from these scene flow methods as our data source is a specific domain: flow particles. Fluid particles differ from typical scene flow point clouds due to their disordered local distribution [51] (see Sec. 3.2.1) and unique physical properties. We propose a graph-based feature extractor and zero-divergence regularization to leverage these properties (see Sec. 3.2).Particle tracking is a fundamental tool in turbulence analysis, progressing from traditional methods like streak photography [18] to advanced techniques such as Laser Speckle Photography (LSV) [65] . This evolution establishes the foundation for Particle Tracking Velocimetry (PTV). PTV gains prominence with the development of automatic tracking algorithms [1] , which represents a significant advancement over manual methods [13] . Modern PTV calculates velocities by matching particle pairs between frames [1] and has been applied widely across various fields such as materials science, hydrodynamics, biomedical research, and environmental science [28; 35; 101; 27] .Deep Learning Methods of Dual-frame Fluid Motion Estimation in PTV. Before the advent of deep learning, PTV algorithms primarily focused on improving particle matching by considering group particle movement [62] , using multiple time step data [12] , or conducting spatial area segmentation [99] . With the onset of deep learning, deep neural networks have been designed for particle motion estimation from point cloud pairs [57; 50; 53; 70; 96; 92] . Among them, DeepPTV [50] and GotFlow3D [53] are specifically tailored for fluid flow learning and in a fully supervised manner, asResidual R demonstrated in Fig. 1 . Our work follows these prior efforts, aiming to develop a data-efficient and cross-domain robust motion estimation technique through self-supervision and test-time optimization.Trainable Gradient FlowQuestion: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: Please see Appendix A.6.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
problem formulation: 
 To elucidate the architecture and functionality of our proposed method for dual-frame fluid motion estimation, we outline the problem as follows: The method processes two consecutive, unstructured sets of 3D particles, X t ∈ R n1×3 and X t+∆t ∈ R n2×3 , recorded at times t and t + ∆t. It outputs the predicted flow motion F ∈ R n1×3 , mapping each particle x i from X t to a vector f i that indicates its movement between the two frames, capturing the flow dynamics in the turbulent 3D environment.In the training phase, we aim to learn the patterns of fluid flow using considerably fewer samples, as low as 1% of what conventional approaches require, given the inherent difficulties in gathering data for specific scientific domains. We design the network as depicted at the top of Fig. 2 to train a graph-based feature extractor (Fig. 2b ) that extracts per-point features for the following soft point matching. These features initialize the flow between the point clouds using the optimal transport module (Fig. 2c ), and we employ self-supervision losses, as shown in Fig. A common assumption in LiDAR scene flow estimation is the smoothness of flow. However, this is not enough for fluid particles due to their unique geometric distribution, as shown in the left of Fig. 3 . The fluid velocity field is smooth only at a coarse scale but remains complex at a fine local scale. Therefore, we need a strong relation-based graph feature extractor and more specific regularization to capture the intricate properties of fluid particles.Graph-based Feature Extractor. Point cloud-based extractors, including PointNet [71] and PointNet++ [72] , are commonly used in LiDAR scene flow estimation [40] . While these extractors effectively discern broader spatial structures, their capability to grasp intricate local relationships, which is vital for analyzing fluid dynamics, can be inadequate. In contrast, graph-based feature extractors excel at capturing local patterns by considering the relationships between proximate nodes, or in our context, particles. Hence, drawing inspiration from GotFlow3D [53] , we opt for a graph-based feature extraction backbone, as depicted in Fig. 2b . Initially, we construct a static nearest-neighbor graph from the input point cloud. This graph is then processed through several GeoSetConv layers [72] to form a high-dimensional geometric local feature. To further enrich the feature, we construct a dynamic graph using EdgeConv [91] based on the high-dimensional feature, forming a GNN that outputs static-dynamic features. The dynamic graph expands the receptive field and focuses on geometric feature properties. Further details can be found in Appendix A.1.1.Correspondence. With the static-dynamic feature from the feature extractor (Sec. 3.2.2), we formulate the correspondence linking problem through the framework of optimal transport [87] , where a higher transport cost between two points indicates a lower similarity within the extracted feature space. The optimal transport plan yields the soft correspondence weight between X t and X t+∆t , as shown in Fig. 2c , which can be used to formulate an initial flow estimate F init . This follows the common scene flow method, thus we leave the details to Appendix A.2.Self-supervised Losses. Since manually linking particles between sets is notably intricate, we advocate for the adoption of self-supervised losses (Fig. 2e-g ).A core principle guiding self-supervised flow learning is the fact that X t + F and X t+∆t should be similar. The Chamfer distance (CD) is a standard metric used to measure the shape dissimilarity between point clouds in point cloud completion. Therefore, we adopt it as our reconstruction loss. We also add a regularization term [40] to prevent degeneration:EQUATIONHere, Y ′ represents the estimated point cloud formed by X t + F init , and Y is the target point cloud formed by X t+∆t . p i denotes the confidence of matching in the Optimal Transport (Sec. 3.2.3), which is the weighted sum of transport costs. λ conf term is used to avoid the trivial solution p i = 0.3.2.4.2. Smooth Loss. Given the infinitely differentiable characteristic of the velocity field, it is postulated that the field should exhibit a certain level of continuous and smooth transitions (at a coarse scale). In light of this theoretical underpinning, we introduce a smooth regularization loss to enforce and maintain this continuous behavior in the velocity field, which is defined as,EQUATIONHere, X represents the point cloud formed by X t . N l (x i ) represents the index set of the l closest points to x i . f i and f k denote the estimated flow vectors at points x i and x k , respectively. [2] , embeds input values in a high-dimensional space. Studies like [38] and [78] followed this Splat-Blur-Slice pipeline. Inspired by these, we implemented a splat-based zero-divergence loss: to compute divergence, we need the partial derivative of the field. The irregular arrangement of particles in 3D complicates this. Thus, we propose "splatting" unstructured flow estimates onto a uniform 3D grid, then applying zero-divergence regularization at these grid points, as shown in the right of Fig. 3 . In formal terms, the dense grid is denoted by (sj, sk, sl) T , with j, k, l ∈ Z indicating the 3D indices of the grid point. The parameter s corresponds to the grid's spacing. Then, given a grid point x = (sj, sk, sl) T , we employ the inverse squared distance as interpolation weights to approximate the flow at that particular point,EQUATIONwhere f i is the estimated flow value at point x i . N (x) denotes the neighborhood among the point set of X t for grid point x. The parameter ϵ is introduced to maintain numerical stability. By employing splatting, we convert the variable particle distance into fixed grid spacing, thus achieving efficiency and effectiveness.3.2.4.3.3 Divergence Calculation. Once Splatting has been employed, the divergence at that point, specified by x = (sj, sk, sl) T , can be defined as:(∇ • F)(x) = 3 k=1 f (x+su k )-f (x-su k ) 2s, where u k is a unit vector with 1 at the k-th entry. Finally, the zero-divergence regularization can be formulated as,L div = 1 JKL J-1 j=0 K-1 k=0 L-1 l=0(∇ • F)((sj, sk, sl) T ) 1 , where J, K, and L represent the number of grid points along the respective dimensions.3.2.4.3.4 Zero-Divergence Loss v.s. Smooth Loss Zero-Divergence loss is similar to Smooth loss in that it computes spatial gradients and requires the norm of the gradient to be small, essentially penalizing the case that neighboring flow vectors are totally irrelevant. However, Smooth regularization is too strict for fluid particles. While the divergence constraint only requires the total divergence to be zero, it does not necessitate that any two vectors be oriented in the same direction, thus allowing for locally complex particle dynamics. In practice, we set the neighborhood set size for calculating Smooth Loss to be much larger than that for calculating Zero-Divergence Loss, because smoothness is a more coarse-scale regularization. Finally, we note that Zero-Divergence Loss is calculated along three specific axes, whereas Smooth Loss is not. Therefore, using the same method (KNN) for calculating Zero-Divergence Loss as for Smooth Loss is not efficient.To summarize, our final self-supervised training loss isL train = L recon + λ smooth L smooth + λ div L divAs shown at the bottom of Fig. 2 , with the initial flow estimate from the trained network, we introduce a novel Dynamic Velocimetry Enhancer (DVE) module during the test phase for test-time optimization. This provides added flexibility to accommodate unseen situations and address potential inaccuracies arising from the limited training data context, which will be demonstrated in Sec. 4.4. In principle, our approach seeks a residual flow vector R such that F = F init + R, which can be optimized to rectify the inaccuracies. Formally, DVE is essentially an optimization process using the L recon objective function, with the formulation as follows:EQUATIONThis test-time supervision (Fig. 2 (g*)) is similar to L recon 1 without the regularization λ conf . Solved using an Adam optimizer, it only involves parameters from an n 1 × 3 matrix. Concerning that existing test-time optimization modules [47] are slow, DVE is very efficient, as demonstrated later in Sec. 4.1.We omit both L smooth and L div during the test stage. During the training phase, our objective is to embed prior knowledge about particle flow into the network. L smooth and L div serve not only to foster a comprehensive understanding of fluid behaviors but also function as regularizers, mitigating overfitting caused by the unconstrained reconstruction loss. However, in the test phase, our focus shifts to specific sparse particle sets.In certain scenarios, such as when flows adhere to boundary conditions, these particles may not strictly adhere to the expected norms of ideal smoothness or zero-divergence typical in a flow field. Additionally, given that the initial flow estimate should be sufficiently accurate, regularizers become unnecessary. Our approach to customizing the loss functions in this manner aims to enhance the robustness of our model against the complex challenges encountered in real-world applications, thereby improving data efficiency and cross-domain robustness.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [NA]• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments: 
 We conduct comprehensive evaluations using different data domains on our proposed framework. First, we compare our method with SOTA fully supervised methods (Sec. 4.1). Next, we examine its performance under the constrained size of training data, reflecting real-world situations where domain-specific data is limited (Sec. 4.2). We then assess the framework's performance under different domains with increasing domain shift, highlighting its cross-domain robustness (Sec. 4.3). Additionally, we conduct comprehensive ablation studies on the components of our framework (Sec. 4.4) to validate their effects. Following the previous SOTA method GotFlow3D [52] , our datasets include FluidFlow3D [52] and its six fluid cases, DeformationFlow [98] and AVIC [41] . Due to page limit, experimental settings including implementation details, datasets, and evaluation metrics can be found in Appendix A.3.Since DeepPTV [50] is not open-sourced, we enrich the comparison by including scene-flow methods [56; 70; 96; 92] as our baselines. We benchmark our method against established fully supervised models, such as FlowNet3D [57] , FLOT [70] , PointPWC-Net [96] , PV-RAFT [92] , and GotFlow3D [53] , all utilizing the FluidFlow3D training set. All baseline models are evaluated using the default hyperparameters. As shown in Figure 4 , our purely self-supervised approach outperforms all the fully supervised baselines. Additionally, we introduce Ours (1%)-our method trained on just 1% of the data-which still demonstrates comparable performance.We further analyze our framework's performance across six distinct flow cases from the FluidFlow3D dataset, with details available in Appendix A.3.1. For three representative cases-Uniform Flow, Turbulent Channel Flow, and Forced MHD Turbulence-we present detailed results in Fig. 4 , while the remaining are documented in Appendix A.4.1. In the simple Uniform Flow case, our method shows slight improvement over the baseline. However, in more complex scenarios, such as Forced MHD Turbulence, our method significantly outperforms the baseline, reducing the EPE/NEPE metric by nearly half compared to the SOTA GotFlow3D.Test-time Efficiency: In Figure 4 , the T test column in the table illustrates the time consumption of each method during the test phase. Our method demonstrates time efficiency, incurring less inference time cost than even baseline supervised methods (without test-time optimization). Our method requires only a few epochs to converge (See Appendix A.5.3). Furthermore, our network's relatively small size (refer to P train comparison in Figure 4 ) facilitates a rapid forward pass. Time profiling is conducted on a single RTX 3090 Ti.Handling rare scenarios, such as unique geometric flow fields [88] or cytoplasmic flows in disease contexts, presents challenges in assembling large datasets. To address this, we explore an evaluation setup with limited training data (still without labels) by randomly sampling from the FluidFlow3D training dataset. We established three distinct training settings: a 100% sampling rate (13,621 samples), a 10% sampling rate (1,300 samples), and a 1% sampling rate (130 samples). Testing was conducted on the FluidFlow3D test data (see Appendix A.3.1). We compared our method with FLOT [70] , PV-RAFT [92] , and the current SOTA GotFlow3D [53] . We present the results of the major metric, EPE, with further details in Appendix A.4.2. Fig. 5 (a) illustrates the robustness of our approach to reductions in training data size. Our metrics remain stable even with significant decreases in training samples, while other methods show substantial performance declines. This disparity becomes more pronounced in complex cases, as discussed below. Natural fluids exhibit a range of behaviors, including convection and laminar flow. Gathering data under all possible conditions presents significant challenges. To address this, we examine the crossdomain knowledge transfer capability of our proposed method. We explore a gradual increase in domain shift: initially, we investigate fluid case domain shifts (Sec. 4 In this section, we employ the complete FluidFlow3D training set in a six-fold cross-validation setup, training on five sub-cases and testing on the remaining one. We benchmark our method solely against the state-of-the-art fluid motion learning method, GotFlow3D [53] , as other baselines are not fluid-specific. The EPE metric results, shown in Fig. 5(c ) (with additional results in Appendix A.4.3), indicate that our method outperforms GotFlow3D in various scenarios, especially in complex conditions like MHD and isotropic turbulence. Moreover, our method shows consistent performance, highlighting its robustness in unfamiliar scenarios.Sim2Real Experimental Setting: Synthetic data with ground-truth labels often serves as a benchmark for method evaluation. However, the domain gap between synthetic and real data can negatively impact performance, underscoring the importance of validation on real-world datasets. In this and the following section, we validate our method using two real-world datasets, DeformationFlow [98] and Aortic Valve Interstitial Cell (AVIC) [41] (details in Appendix A.3.1), to demonstrate its practical application potential. Two challenges in real-world evaluation are: 1) the lack of ground-truth labels in real-world data, and 2) the requirement for a complete PTV method for particle tracking application. Therefore, we use the following setting: our method, trained on FluidFlow3D, is integrated into PTV algorithms (specified below) to provide initial motion estimates. Due to the absence of ground truth, we primarily demonstrate the generalizability of our method across various domains through qualitative results. Quantitatively, we emphasize efficiency in the physical domain, where multiple frames are involved. By contrast, in the biological domain, we focus on validating the plausibility of our estimates, especially given the significant cell deformation, where efficiency is less critical. 4.4 Ablation study on different modules.In addition to the aforementioned assessments, an ablation study regarding different proposed modules including different feature extractors, Zero-divergence Loss, and DVE is performed to demonstrate their effectiveness. We show our method w/ and w/o Div Loss (Zero-Divergence Loss) and DVE module in Fig. 4 . The full results are illustrated in Appendix A.5. 
conclusion: 
 In this paper, we introduce a test-time self-supervised framework for learning 3D fluid motion from dual-frame unstructured particle sets. We address the challenge of improving data efficiency and ensuring cross-domain robustness, which are crucial for practical applications. We demonstrate the viability of our approach through two real-world studies and suggest that our findings could inform further research into extensive real-world applications, the exploration of constraints specific to particular scenarios, and the development of novel model architectures for enhanced adaptability. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: We provide time profiling of our method in Appendix 10.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: We confirm this.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: We provide impact statement in Appendix A.7.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our work poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We have cited original papers.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: A.1 Methods in Detail A.1.1 Feature extractorWe draw significant inspiration from GotFlow3D's feature extractor [52] to obtain a robust representation of particle features. We briefly describe the structure of the feature extractor here.The feature extractor is based on a graph neural network. Initially, a static graph G static = (V s , E s ) is established in the 3D spatial space for the input point cloud. V s contains the given input points, while E s consists of connections between the k-nearest neighbors of each point. This process yields the original point cloud feature f i for point i as a hexadecimal set, including the three-dimensional coordinates, the radial distance, the azimuthal angle, and the polar angle. After processing through several GeoSetConv layers [72] , we obtain a high-dimensional geometric local feature:F n i = MaxPooling j∈N k (i) {Φ n (f i , F n-1 j -F n-1 i )},where F n i denotes the feature extracted from the static graph at the n-th layer for point i, and N k (i) represents its k-nearest neighbors in the static graph. Φ n stands for the n-th GeoSetConv layer. F 0 i is initialized by the 3D coordinates of the input point cloud.Subsequently, the dynamic graph is generated. It takes these high-dimensional features F i as inputs and, following the same steps as the construction of the static graph, seeks the nearest neighbors based on these features to establish connections, thus obtaining the local structure on the feature manifold. However, the distinction lies in the fact that the dynamic graph possesses a greater receptive range since it accepts high-dimensional features. Moreover, it is reconstructed during every training session, whereas a static graph is constructed only once.The dynamic feature is obtained in the same manner as F i , with the sole difference being the use of EdgeConv layers [91] instead of GeoSetConv layers. The final feature is obtained by concatenating all hierarchical features from different layers of both static and dynamic graphs.Let Φ X and Φ Y represent the features extracted from two input particle sets, X and Y, respectively. We initiate by computing the point-to-point similarity matrix, given by:EQUATIONIn this context, Φ xi pertains to the feature of the i-th point in Φ X , and analogously for Φ j y . Inspired by pioneering research, we formulate the correspondence linking problem through the framework of optimal transport [39] . Assigning a mass of 1 |X | to each source point x i , we consider its transport to the target point y j with the cost matrix defined by C i,j = 1 -S i,j . In this context, a higher cost indicates a lower similarity between two points within the feature space. Our objective is to identify the optimal transport plan T * that satisfies,EQUATIONwhere KL denotes the Kullback-Leibler (KL) divergence, 1 ∈ R |X |×1 is a vector filled with ones, and the terms involving ϵ and λ are regularizing terms with coefficients controlling their respective strengths.The optimal transport plan T * yields the soft correspondence weight of point x i to point y j as:EQUATIONwhere M Y (x i ) represents the set of L points from Y corresponding to the top L values of T * i,j , and L is a hyper-parameter that can be chosen for specific particle tracking case. Consequently, we can estimate the new location of a given x i , the estimated position is given by,EQUATIONAnd the confidence score p i , which quantifies the reliability of the estimated position y * i for each point x i is given by,EQUATIONFinally, the flow estimate f i is determined as,EQUATIONFluidFlow3D Dataset The FluidFlow3D [53] is a large synthetic dataset designed for the study of 3D fluid flow. Specifically, it offers enough data for training and serves as a benchmark to evaluate the flow estimation capabilities of supervised 3D fluid flow motion learning techniques. This dataset utilizes physically accurate simulated flow structures sourced from public database [49] , ensuring that the provided ground truth flows adhere to computational fluid dynamics principles. It encompasses six typical categories of flow cases, namely, uniform flow, isotropic turbulent flow, magneto-hydrodynamic (MHD) turbulence, fully developed turbulent channel flow, transitional boundary layer flow, and the Beltrami flow [17] . We utilize this synthetic dataset to evaluate our method, comparing it with baselines (see Sec. ). It should be noted that, to our knowledge, this dataset is the only large-scale benchmark specifically designed for dual-frame fluid motion learning. Other fluid datasets, such as CylinderFlow [34] , focus on fluid trajectories across multiple frames, requiring the integration of the full PTV process. This approach deviates from the main problem we aim to address.DeformationFlow The DeformationFlow dataset [98] showcases real-world physical dynamics by recording the indentation created when a stainless steel sphere is placed on a soft polyacrylamide hydrogel due to gravitational forces. Fluorescent fluid particles within the gel were scanned both pre and post-indentation, yielding 3D volumetric images. We employ this dataset because it diverges significantly from the synthetic training set. Notably, the zero-divergence constraint is no longer applicable, making it an ideal candidate to evaluate the cross-domain generalization capability of our proposed test-time self-supervised framework.The AVIC dataset [41] represents real-world biological dynamics revealed through PTV analysis. In the dataset, AVICs, extracted from dissected porcine heart leaflets, were suspended in a peptide-modified PEG hydrogel with fluorescent microbeads and subsequently incubated for 72 hours. We employ this dataset to demonstrate the strong cross-domain versatility of our proposed framework with the data from the biological realm.For a thorough evaluation of our framework's performance and its comparison with previous works, we utilize five prominent metrics commonly applied in particle tracking velocimetry evaluations: EPE, NEPE, Acc Strict, Acc Relax, and Outliers.To introduce the metrics, we first define point error e i and the relative point error e rel i ,EQUATIONwhere e i represents the Euclidean distance (L2 norm) between the predicted flow f * i , and the groundtruth flow f gt i , for a specific point x i . The relative error e rel i provides a normalized measure, indicating the magnitude of the point error in relation to the magnitude of the ground-truth flow at point x i .Then, the metrics used for evaluating the quality of predicted flows are defined as follows. The EPE (End Point Error), calculated as EPE = Furthermore, when performing experiments on real datasets, the absence of ground truth in actual data prompts us to incorporate additional metrics for evaluating our methodology. For the SerialTrack assessment, our evaluative metrics include Matches, Tracking ratio, UpdateNorm, and Time. We use n match to represent the matches identified between the source and target point clouds, and n no_missing signifies the count of source points that are matched in the target point cloud. Consequently, Matches is equivalent to n match , and Tracking ratio is expressed as the fraction nmatch nno_missing . The UpdateNorm metric captures the change in PTV parameters for each iteration, while Time measures the duration taken for a single iteration. For the Fm-track assessment, our primary metric is the neighbor distance score (NDS), delineated for a point x i as,EQUATION)where N (x i ) is the K-nearest-neighbor set of point x i , f i denotes the flow vector at point x i .In our method, there are some pre-defined hyperparameters. For the sake of experimental reproducibility, we list them below.MODEL STRUCTURE For the feature extractor, the K-value of our K-nearest-neighbor is chosen to be 32, the embedding-dim to be 128, and the dropout rate to be 0.5. The grid size for splatting is 10 × 10 × 10.The selected number of neighboring points for the reconstruction loss L recon is 32. Likewise, the number of neighboring points for smooth flow loss L smooth is 32, and for zerodivergence loss L div is 2. λ conf is 0.1, λ smooth is 10, and λ div is 0.1.During the training phase, we utilize a mini-batch training process with a batch size of 4. To achieve convergence, we train the full-data model and 10%-data model for 100 epochs, and 1%-data model for 300 epochs. A default learning rate of 0.001 is set and the training is run on a single RTX 4070TI.During the test phase, DVE runs for 150 steps with an update rate of 0.01.We present a comprehensive comparison with state-of-the-art methods across all flow cases in Table 2 . Notably, our model, trained on only 130 samples, outperforms the current state-of-the-art, GotFlow3D, and is marked in italics. In particularly complex scenarios such as Forced MHD turbulence and Forced isotropic turbulence, our method significantly outperforms GotFlow3D, even when trained with only 1% of the training set samples. In more common scenarios, our model, trained with the full dataset, surpasses GotFlow3D in most metrics, while the model trained with limited samples still demonstrates commendable performance.We present the full results of our limited-data training experiment here in Tab. 3. We present the full results of our cross-domain robustness experiment here in Fig. 7 . Our technique consistently outperforms the state-of-the-art method GotFlow3D across different fluid scenarios in terms of accuracy. Furthermore, our method demonstrates a consistent performance across these scenarios, unlike the pronounced variability observed with GotFlow3D. This suggests that, after training on certain fluid cases, our model can proficiently handle scenarios it hasn't directly observed. This adaptability can be attributed to the DVE module, which allows for on-the-fly adjustments to unseen examples during testing.Experimental Settings: In this work, we analyzed a dataset containing images of porcine aortic valve interstitial cells (AVICs) embedded in a polyethylene glycol (PEG) hydrogel. We employed 0.5 [µm] microspheres as tracking particles to monitor the movements within the PEG hydrogel. For detailed information on AVIC encapsulation within PEG hydrogels, readers are referred to a previous study [33] . We subjected the AVICs to three distinct conditions: initially, under regular conditions; secondly, after exposure to Cytochalasin-D, a compound that disrupts actin polymerization and cellular contraction; and thirdly, following treatment with Endo 1. Direct observation of cell deformation being challenging, we tracked the motion of particles adjacent to the cells within the gel under different treatments as a proxy measure. We utilized Fm-track [41] , a specialized particle tracking velocimetry (PTV) method for cell tracking, providing it with an initialization value. When investigating the intrinsic properties of the velocity field, it is understood that incompressible fluids, by definition, exhibit zero divergence. Consequently, we have introduced a regulation specifically targeting zero-divergence in the estimated velocity field. However, this regulation is not entirely accurate, as many fluids can be compressed. Therefore, this section scrutinizes the effectiveness of the zero-divergence regulation. We tested it using the FluidFlow3D test data(See Table . 6) and its six fluid cases(See Table 7 ).As shown in Table 6 , the zero-divergence regulation appears ineffective for models trained on full samples when tested on the FluidFlow3D-NORM dataset. However, this regulation becomes more effective for models trained on fewer samples. Furthermore, the lower the sample count, the greater the benefit gained from zero-divergence regulation. Our interpretation is that the zero-divergence regulation provides additional prior information, which can be learned with a sufficiently large sample size but may be challenging to access when the sample size is limited. Thus, our zero-divergence regulation plays a compensatory role that improves data efficiency.We conducted further tests to determine the suitability of the zero-divergence assumption for various fluid cases. Our findings, illustrated in Figure 7 , revealed that our regulation performs excellently in fluid cases that exhibit zero divergence, such as Beltrami Flow, Turbulent Channel Flow, Transitional Boundary Flow, and Uniform Flow. Specifically, our model trained with 130 samples (with zerodivergence regulation) outperformed an unregulated model trained with 1300 samples across all metrics in Uniform Flow. In more complex fluid cases, such as Forced MHD Turbulence, where the zero-divergence law does not hold, employing zero-divergence may capture incorrect fluid features if the model is trained with insufficient data. However, with a full training dataset, it still enhances performance. We have developed the Dynamic Velocimetry Enhancer (DVE), which is used in the testing phase of the process to optimize the initial flow and can effectively improve cross-domain robustness and data efficiency. To illustrate this, we trained models with and without the DVE module on different sizes of training sets and compared their performance on various test sets. We tested it using the FluidFlow3D test data(See Table . 8) and its six fluid cases(See Table 9 ).As evidenced by Tab. 8, the model without the DVE module is less effective than the model with the DVE module, even though the latter is trained on only 1% of the samples. Furthermore, we observe that the model without the DVE module is highly sensitive to the training size, and the performance of each metric rapidly declines as the training size decreases. In contrast, the model with the DVE module exhibits a certain degree of robustness. This demonstrates the significant impact of our DVE module on data efficiency.Tab. 9 shows the performance of the DVE module also varies across different flow cases. In simple cases, the presence or absence of the DVE module has minimal impact on the metrics, since the initial flow already closely matches the ground truth with a relatively small error. However, in challenging cases where the initial flow is far from the ground truth, the DVE module plays a crucial role. It significantly enhances the model's accuracy through a straightforward and focused optimization process.Time Profiling of DVE A significant challenge in test-time optimization is implementing persample adaptation at test time without adversely affecting inference efficiency. We demonstrate the time profiling of our method in Table 10 to illustrate the efficiency of our proposed test-time optimization paradigm. Additionally, we present the convergence graph of DVE to study the influence of the number of refinement steps. As shown in Figure 9 , DVE can converge within 150 epochs due to its simple structure, resulting in fast inference and high efficiency.Our research addresses the challenge of estimating dual-frame fluid motion. However, this field suffers from a scarcity of large-scale benchmark datasets. To mitigate this issue, we incorporate real- world datasets and conduct extensive studies across diverse flow cases. Nonetheless, it is anticipated that a broader variety of fluid motion data will become available in the future.Our research focuses on advancing a dual-frame fluid motion estimation method designed for optimal data efficiency and cross-domain robustness in turbulent flow analysis. This innovative approach is anticipated to empower scientists to analyze fluid dynamics with significantly reduced data requirements while enhancing overall method robustness. It is crucial to acknowledge potential limitations, particularly in real-world applications like blood analysis in medical science, where errors arising from our method could potentially be harmful. Our intent is to continually refine and improve our methodology to minimize any such unintended consequences.NeurIPS Paper Checklist 


Decision: All review scores are positive: WA, WA, and BA. While no A and SA are given, AC confirmed that most reviewers' comments are positive. AC also verified that their final evaluations lean toward accepting this paper. Reviewers jdpi and N6ux recognize that the proposed method can train with significantly smaller datasets than current state-of-the-art methods at better performance. Reviewers jdpi and wM8x appreciate that the paper is well organized. Reviewers wM8x and N6ux agree that this work tackles an important and interesting research problem. Finally, reviewers jdpi and N6ux recognize that the experimental results demonstrate the effectiveness of the proposed method. While reviewer N6ux was initially a bit underwhelmed by this paper, the authors' rebuttal did a good job explaining the issues to convince this reviewer. For the reasons stated above, AC recommends this paper for presentation in NeurIPS. Since several issues remain, we expect the authors to revise the manuscript based on the reviewers' comments.

Reviews: review: paper_summary: paper_summary: In this paper, the author proposes a self-supervised learning based 3D particle tracking velocimetry (PTV) technique for dual-frame fluid motion estimation. The proposed method surpasses its supervised counterparts while utilizing only 1% of the training samples (without labels) compared to previous methods. A zero-divergence loss is utilized for turbulence flow and it is implemented in splat-based approach for efficiency and effectiveness. On the benchmarks, the proposed method shows the best performance comparing to baselines.
strength_weakness: strength_weakness: - As for tests on the real-world dataset, the author mentioned the proposed method is integrated with the PTV framework. Just wondering how will the proposed method perform stand alone?
- As mentioned in the paper, there are datasets with sparse flow field, which may influence the performance of the proposed method. It seems that there is no investigation of the particle density. Just wondering how will the method perform given trained on sparse/dense and inference on dense/sparse?
questions: questions: See weaknesses.
limitations: limitations: Yes. The author discusses the limitations in the appendix.
paper_summary: paper_summary: The paper proposes a self-supervised method to learn 3D particle tracking and modelling turbulent fluid flow. They regularize their method with a zero-divergence loss function and inspired by the splat operation they propose a splat-based implementation for this loss. They also incorporate a GCNN feature extractor to learn geometric features. Their method also supports test-time optimization, with their Dynamic Velocimetry Enhancer (DVE) module.
strength_weakness: strength_weakness: * The method needs test-time optimisation, which will make the result a bit questionable if you optimize the test set.
* The test time complexity is not reported clearly. In Fig 4 (top table, ours (1%) w/o DEV) it is not clear whether the time reported is the time for the forward pass or includes the optimization.

**Suggestion:**
Please use single tables format their different formats in the main text and appendices. Also, splitting the tables and figures will make it easier to follow the paper and understand the result.
And remove the unnecessary **bold** used in the main text.
questions: questions: In Fig 4 (top table, ours (1%) w/o DEV) it is not clear whether the time reported is the time for the forward pass or includes the optimization, can you comment on that?
limitations: limitations: Yes.
paper_summary: paper_summary: In this paper the authors propose a graph based network that combines feature extraction with test time optimization to perform two-frame particle tracing. The core problem is that given two sets of points, e.g., point cloud data, to find the correspondences between the point clouds and estimating their velocity from this correspondence. While this problem can be well analyzed in synthetic data, real world problems are often sparse in data and, accordingly, using methods that do not require labeled data (and not a lot of it) would be beneficial. The proposed methodology works by proposing an unsupervised loss formulation based on assuming smoothness and divergence-freedom of the solution space and uses parts of this loss function as a test-time target for optimization. The method is evaluated and trained on synthetic data and then evaluated on dissimilar data to show the generalization capabilities of the approach.
strength_weakness: strength_weakness: Overall the paper has an interesting core idea, however, several issues exist, especially regarding evaluation and the lack of statistical evaluations and ablation studies in general. There are a few core weaknesses listed below in a short list. Following these a list of all potential issues and weaknesses is included for completeness at this point. Note that many of these issues are also addressed separately and are not questions in themselves.
* The authors perform no evaluation regarding the statistical significance of their proposed method. While the computations are deterministic (as they note in the checklist), this does not mean that network initialization does not play a role, especially when many of the evaluation metrics are relatively close and some results seem counter intuitive, e.g., the EPE improving for smaller datasets. Furthermore, there is no clear evaluation on the choice of the 1% of data, i.e., did the authors try different subsets? Were the 1% chosen consistently across class? Across different trained methods?, etc.
* It is not clear why the proposed DVE test-time optimization cannot be performed on top of data trained on supervised data, especially as the authors change the loss functions anyways from training to testing.
* The authors make a fairly broad claim in the introduction as to synthetic data being always limited due to hand-crafted priors but show that their method does not suffer from this problem, even though it uses the same synthetic data. This needs to be clarified.
* The divergence-free loss term is not very well evaluated. On the one hand the assumption seems to be relatively central to the approach and the authors go out of their way to state that assuming divergence-freedom can even be assumed in compressible cases (where it does not hold), but then also state that they disable the divergence-free term during test-time optimization especially for cases where it does not hold. * Either the authors need to more clearly evaluate the impact of the divergence-free loss term during test-time optimization in _all_ cases, or consider the importance of the argument.
* The  formatting of the paper at times is very odd with subsubsubsubsections and randomly highlighting of statements with inconsistent (and incomplete) highlighting in tables. This should be improved.

General:
* In the paper checklist under 7 the authors note that their ‘method is deterministic’ and thus does not require error bars. However, choosing random subsets of data, training different architectures and more usually involves sources of randomness and seed variance can be quite significant. This needs to be evaluated properly and adjusted accordingly.
* Table 3 seems odd as the EPE _improves_ for the proposed method when training on smaller datasets.
* Table 9: Why is the Epoch count increased to 300 for the 1% train size and is this done consistently across all methods and what would the results be with only 100 Epochs or a normalized training with the same number of weight updates across dataset sizes?
* It is wholly unclear as to why the proposed scheme is 6 times slower during testing just because it is trained on a larger dataset.
* Why can other methods not utilize DVE as a test-time optimization process? This would make for much fairer comparisons
* The authors should make it more clear in the related work as to which graph-based feature extractor they use as their architecture is heavily informed by prior work.

Dataset:
* The authors claim that synthetic data is always limited as it leads to ‘hand-crafted priors’ but they utilize a synthetic dataset to train their method and show that it is not limited by the biases induced from the prior, i.e., their method can generalize even from synthetic data.
* The authors state that in cases of boundary conditions, or other situations, smoothness and divergence-freedom is not holding up. This should be more clearly shown, i.e., a demonstration that using  these regularization terms leads to issues in such cases.

Reduced Data Training:
* The 1% setup is interesting, however, the authors do not clearly show how sensitive their method (or methods in general) are to selection, i.e., there is no clear evaluation on how the choice of the 1% affects the outcome
* There are no statistical evaluations, which would be especially important for the 1% evaluations and with the difference between methods being so narrow at times, initial seed choice might be important too
* There is no evaluation as to what the method would do if the data is incomplete, i.e., if particles are missing from one of the frames. This would be important for real-world applications but due to their reliance on hand-crafted synthetic priors and other narrow datasets, this is not evaluated clearly.

Divergence-Freedom/Loss Terms:
* The Divergence-free loss term is interesting but not using it during testing seems odd as it should be helpful in all cases. However, even the authors note that this would be a limitation in non divergence free settings (which they evaluate one of). Considering the prominence of the divergence-free claim, this needs to be further elucidated (especially considering the statement in line 674)
* The authors claim (178) that even for compressible fluids divergence-freedom can be assumed in practice. While sometimes done, this should be more clearly evaluated as the authors clearly do not use the divergence-free loss in datasets where the assumption would not hold.
* The distinctions when smoothness and divergence-freedom are not to be used for optimization are not clear as there might be reason to not use the divergence-free loss, i.e., if one expects a dataset with divergence, but not requiring smoothness is odd. This should be done in an ablation study.
* The neighborhood sizes are odd and should be more clearly written as the information in 715 is not easily understood, i.e., is the divergence-loss computed with only 2 points?
* The formulation of the divergence as a central finite difference scheme is not necessary, other information would be more prudent.

Formatting/Writing:
* The formatting is very inconsistent at times, e.g., table layouts, subsubsubsubsection headings are sometimes italicized, sometimes underlined and sometimes bold.
* The authors do not need to state that they are excluding information due to page limits, twice.
* The highlighting in Figure (mostly table) 4 is inconsistent. In the top part the second best performing method is not highlighted with underlining but also boldfaced and the T_test column has no highlighting
* Table 6 should also highlight cases where the method without divergence-freedom performs better

Missing Details:
* How were the lambda parameters chosen?
* KNN should be properly introduced (208) and evaluated as to why it is not efficient

Evaluation:
* The method performs significantly worse in some cases, e.g., the EPE is worse by a factor of 2 for the MHD case compared to the turbulent channel flow. It would be nice if this was more clearly evaluated and highlighted, especially for the Beltrami case (Figure/Table 5) where Gotflow performs worse by a factor of 2 relative to the proposed method, which is fairly close considering this case is about 8 times worse than the best case for the proposed method and only 3 times worse for Gotflow.
* The improvement of 22882 matches over 22001 seems fairly minor (315) and it would be prudent to add the tracking timing in this section in the main paper.
* How does the proposed method perform on its own (323), i.e., when it is not used as an initializer?
* The difference in EPE from 100% to 10% with and without DVE is relatively close even though the text (800) notes that the method is highly sensitive.
questions: questions: Note that these questions are not in order of importance.
1. What is the variance and statistical certainty of the training across different initializations for the proposed methods?
2. Why is the EPE error lower for the 1% case in Table 3?
3. What would the training results be with equal weight updates across methods, and if they do not change from the proposed hyperparameters, why were these hyperparameters chosen?
4. Is it possible to apply DVE to other state of the art methods and, if so, what would the results be?
5. How was the 1% subset of the data chosen? Does this influence the result, i.e., do different subsamplings (and strategies) result in consistent (within a small deviation) results?
6. Is there any data to highlight the limitations of including the regularization terms during test time optimization?
7. How were the lambda parameters chosen?
8. Why is the computational time of the method at test time different based on the dataset size used during training?
9. What was the total time to train a network from start to finish for the proposed method with and without the various terms?
limitations: limitations: The primary limitations of the paper relate to the evaluation of the methodology and the divergence-free loss term. In summary there are three key issues:
1. The evaluation should include a broader evaluation that at least considers seed initialization and variance due to the 1% sampling. As the paper currently stands, it is impossible to tell, with certainty, that the results are consistent and not just due to a lucky seed. While performing such evaluations for all setups is computationally prohibitive, at least the core contributions should be evaluated
2. There is a lot of inconsistency regarding the divergence-free loss. On one hand it is argued as very important during training and how it can work even in compressible cases. Then the term is wholly disabled during test time optimization because it isn’t needed and then towards the end in a case with divergence the term is left off anyways. There is no clear evaluation as to the influence of including the divergence-free condition during test optimization, especially for cases which are compressible (and/or not divergence-free). This makes it difficult to judge how the method truly generalizes
3. While not a direct limitation, the authors themselves state that the use of synthetic data leads to hand-crafted biases (in the introduction), but they also demonstrate how their method is immune to this. As this does not seem to be a limitation of the synthetic data nature, this should be either clarified in the introduction or an actual limitation where the method does not generalize due to its limitation from hand-crafted data.
Rating:

Comments: title: Response to the reply.
Request from AC
Request from AC
Request from AC
comment: Thanks for the detailed replies to my concerns. I don't have further questions and I remain positive about this paper.
Dear Reviewer N6ux,

Thank you very much for your detailed review comments.
The authors carefully provided comments in the rebuttal. Would you please reply to them, if needed?

Best,
Dear Reviewer wM8x,

Thank you very much for your detailed review comments.
The authors carefully provided comments in the rebuttal. Would you please reply to them, if needed?

Best,
Dear Reviewer jdpi,

Thank you very much for your detailed review comments.
The authors carefully provided comments in the rebuttal. Would you please reply to them, if needed?

Best,