SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Each simulation within the BubbleML dataset tracks the velocities in the  $x$  and  $y$  directions, temperature, and a signed distance function (SDF),  $\phi$ , which denotes the distance from the bubble interface. The SDF can be used to get a mask of the bubble interfaces or determine if a point is in the liquid or vapor phase. These variables are stored in HDF5 datasets. For instance, the temperature is stored as a tensor with a shape  $t \times x \times y \times z$  ( $t \times x \times y$  in 2D), which allows indexing with xyz-spatial coordinates or time. The HDF5 files also include any constants or runtime parameters input to the simulation. Some of these parameters, such as thermal conductivity or Reynolds number, are constants used in the PDEs that govern the system. Including these variables and parameters in the dataset enables comprehensive analysis and modeling of the boiling phenomena.
Table 2: Results of pre-trained and fine-tuned RAFT and GMFlow models on optical flow data (B) generated from PB-Saturated dataset. A 80:20 split results in 2000 training and 500 validation image pairs. The pre-trained models include C trained on FlyingChairs dataset, C+T trained further on FlyingThings3D, and C+T+S further fine-tuned for the Sintel test benchmark. Model+B represents models fine-tuned on the BubbleML optical flow dataset.
Model	Method	Chairs (Val)	Things (Val)	Sintel (Train)	KITTI (Train)	Boiling (Test)
Clean	Clean	Final	F1-EPE	F1-all
C	RAFT	0.82	9.03	2.19	4.49	9.83	37.57	4.20
GMFlow	0.92	10.23	3.22	4.43	17.82	56.14	4.73
C+B	RAFT	0.91	11.22	2.55	5.16	13.7	44.44	2.33
GMFlow	1.31	11.99	3.78	5.12	21.91	63.04	2.36
C+T	RAFT	1.15	4.39	1.40	2.71	5.02	17.46	4.72
GMFlow	1.26	3.48	1.50	2.96	11.60	35.62	7.98
C+T+B	RAFT	1.28	7.69	1.69	2.95	9.96	23.61	2.38
GMFlow	1.39	3.88	1.61	2.91	14.49	43.09	2.51
C+T+S	RAFT	1.21	4.69	0.77	1.22	1.54	5.64	8.39
GMFlow	1.53	4.09	0.95	1.28	3.04	13.61	14.65
C+T+S+B	RAFT	1.37	6.59	0.89	1.60	1.83	6.44	2.34
GMFlow	1.65	4.49	1.07	1.45	4.06	18.99	2.56
BubbleML follows the FAIR data principles [51] as outlined in Appendix A.2. It is essential to validate simulations against experimental observations due to inherent approximations in numerical solvers and simplified models of real-world phenomena. Appendix A.4 provides a detailed validation of the BubbleML dataset.
4 Benchmarks of BubbleML: Optical Flow and SciML
4.1 Optical Flow
Generation of Optical Flow Dataset. Optical flow computes the velocity field of an image based on the relative movement of objects between consecutive frames. This method holds significant implications for downstream tasks, such as extracting side-view boiling statistics and applying SciML to real-world experimental data. Although many datasets capturing spatiotemporal dynamical systems can be repurposed to create optical flow datasets, the inherent non-rigidity of bubbles introduces unique physical phenomena that are not present in other datasets. For instance, consider the scenario where a bubble detaches from a heater surface: the bottom region of the bubble exhibits significantly higher velocity compared to the top region, resulting in a velocity gradient that forces the bubble into a spherical shape. At higher heater temperatures, deformation and detachment processes might occur more frequently, leading to different flow patterns and bubble behaviors illustrated in Figure 1.
We create an optical flow dataset from BubbleML to learn bubble dynamics. Liquid and vapor phases are distinguished using  $\phi$ . In the generated image sequences, bubble trajectories are tracked across consecutive timesteps. Note that the optical flow dataset only includes bubble velocities at each timestep. Excluding liquid velocities focuses the learning task on capturing observable objects (bubbles). The bubble velocities in non-dimensional units are converted to pixels per frame units (see Appendix B.1) before being written to the widely used Middlebury [34] flow format, resulting in a sequence of images and flow files that resemble the Sintel dataset [36]. For training and validation of optical flow models, PyTorch dataloaders are provided for the generated dataset 3. This allows for easy integration and fine-tuning of existing optical flow models using the BubbleML dataset.
Learning Bubble Dynamics. We evaluate and fine-tune two state-of-the-art optical flow models, RAFT [52] and GMFlow [53], using the BubbleML optical flow dataset (B). We consider three different pre-trained models for each method: the first model is trained exclusively on FlyingChairs (C), the second is trained on FlyingChairs and FlyingThings3D  $(\mathrm{C} + \mathrm{T})$ , and the third model is finetuned for the Sintel Benchmark  $(\mathrm{C} + \mathrm{T} + \mathrm{S})$ . To assess the performance of the trained models, we measure the end-point error. Table 2 summarizes the results for one dataset. Refer to Appendix B for results on the other datasets.
Initially, the pre-trained models exhibit subpar performance on the BubbleML data. To address this, each model is fine-tuned for 3-4 epochs with a low learning rate of  $10^{-6}$ . After fine-tuning, we observe a significant improvement in predictions for the test data (see Figures 7 and 8 in Appendix B). While all fine-tuned models tend to converge to similar levels of accuracy for pool boiling datasets, fine-tuning the pre-trained FlyingChairs models (C) on the BubbleML (B) dataset gives the best results. This could be attributed to the similar nature of the datasets consisting of 2D objects in motion. In the case of flow boiling, the best results are achieved by fine-tuning models initially trained for the Sintel benchmark  $(\mathrm{C + T + S})$ . Flow boiling images have an extremely high aspect ratio (8:1), which is similar to the Sintel (3:1) and the KITTI (4:1) datasets. Note that although training the models on the boiling dataset for more epochs improves performance on our specific task, it adversely affects the models' generalization capabilities, leading to increased errors on the other datasets.
Open problems. Error analysis (see Appendix B.3) highlights the shortcomings of state-of-the-art optical flow models in accurately capturing the turbulent dynamics of bubbles. Although fine-tuning improves the overall performance, the high errors at the bubble boundaries remain an ongoing challenge. This underscores the need for novel optical flow models that incorporate physical insights to accurately model the complex and chaotic behavior of boiling. BubbleML bridges the gap for physics-informed optical flow datasets.
4.2 Scientific Machine Learning
SciML Preliminaries. SciML baseline experiments use neural PDE solvers to learn temperature and flow dynamics. We focus on two classes of neural PDE solvers: (a) Image-to-image models, widely used in computer vision tasks, such as image segmentation [54]. These may not always be suitable as PDE solvers, since they are limited to fixed resolution, but they remain competitive in many baselines [55, 28]. (b) Neural operators are neural networks that learn a mapping between infinite-dimensional function spaces. As they map functions to functions, neural operators are discretization invariant and can be used on a higher resolution than they were trained [15]. The seminal neural operator is the Fourier Neural Operator (FNO) [13]. Refer to Appendix C for further details.
For both classes of models, we employ the auto-regressive formulation of a forward propagator, denoted as  $\mathcal{F}$ . For timesteps  $\{t_1,\dots ,t_{max}\}$  discretized such that  $t_{k + 1} - t_k = \Delta t$ , the forward propagator  $\mathcal{F}$  maps the solution function  $u$  at  $k$  consecutive time steps  $\{t_{m - k},\ldots ,t_{m - 1}\}$  to the solution at time  $t_m$ . For brevity, we use  $u([t_{m - k},t_{m - 1}]) = \{u(t_{m - k}),\ldots ,u(t_{m - 1})\}$ . The operator  $\mathcal{F}$  can be approximated using a neural network  $\mathcal{F}_{\theta}$  parameterized by  $\theta$ . This network is trained using a dataset of  $N$  ground truth solutions  $D = \{u^{(n)}([0,t_{max}]:n = 1\dots N\}$ . By applying a standard gradient descent algorithm, we find parameters  $\hat{\theta}$  minimizing some loss function of the predictions  $\mathcal{F}_{\hat{\theta}}\{u^{(n)}([t_{m - k},t_{m - 1}])\}$  and the ground truth solutions  $u^{(n)}(t_m)$ . Thus, given solutions for  $k$  initial timesteps of an unseen function  $u$ , we can obtain an approximation  $\mathcal{F}_{\hat{\theta}}\{u([0,t_{k - 1}])\} \approx u(t_k = t_{k - 1} + \Delta t)$ . Using this approximation for  $t_k$ , we can step forward to get  $\mathcal{F}_{\hat{\theta}}\{u([t_1,t_k])\} \approx u(t_{k + 1})$ . This process is called rollout and is repeated until reaching  $t_{max}$ . While in principle, rollout can be done for arbitrary time, the quality of approximation worsens with each step [28, 56]. We implement several strategies that attempt to mitigate this deterioration [57, 58]. However, achieving a long and stable rollout is still an open problem.
Baseline Implementations. We implement several baseline image-to-image models—including UNet<sub>bench</sub> and UNet<sub>mod</sub>—and neural operators—including FNO, UNO, F-FNO, G-FNO, and T-FNO. Detailed descriptions and comparisons of the models are included in Appendix C.1.
Training Strategies. Detailed descriptions for each of the training strategies we used are listed in Appendix C.2. We implement teacher-forcing training [59], temporal bundling, and the pushforward trick [57]. Models trained with the pushforward trick are prefixed with "P-". A discussion of hyperparameter settings can be found in Appendix C.3.
Metrics. We draw inspiration from PDEBench and adopt a large set of metrics that include the Root Mean Squared Error (RMSE), Max Squared Error, Relative Error, Boundary RMSE (BRMSE), and low/mid/high Fourier errors [17]. These metrics provide a comprehensive view of the physical dynamics, which may be missed when only using a global loss metric. For instance, when predicting temperature, we find that the max error can often be very high due to the presence of sharp transitions between hot vapor and cool liquid. Even a one-pixel misalignment in the model's prediction can cause the reported temperature to be the opposite extreme. Metrics that report a global average
(i.e., RMSE) could mask these errors because they get damped by the average. We incorporate an additional physics metric: the RMSE along bubble interfaces (IRMSE). Accuracy along both the domain and immersed boundaries is of significant importance. Boundary conditions determine if the solution to a PDE exists and is unique. In the case of the multiphysics BubbleML dataset, accurate modeling of the system requires satisfying the conditions at the liquid-vapor interfaces accurately.
Learning Temperature Dynamics. One application of SciML using the BubbleML dataset is to learn the dynamics of temperature propagation within a system. In this context, the system's velocities serve as a sourcing function, influencing the temperature distribution. Notably, UNet-based models perform best across all datasets (see Figure 3b-d). For a complete listing of error metrics for each model and dataset pairing, refer to Appendix C.4. UNet models may have some advantage in predicting the interfaces and boundaries (IRMSE and BRMSE) because they naturally act as edge-detectors. The temperature also propagates smoothly, so it is likely unnecessary to use global filters like the FNO variants. In contrast, FNO models rely on fast Fourier transforms and weight multiplication in the Fourier space, which, while capable of handling global and local information simultaneously, might not be as effective at capturing local, non-smooth features. Several recent studies report similar observations about auto-regressive UNet and FNO variants [28, 55, 56].
Figure 3: Temperature and Heat Flux Prediction. (a) Cross-validated heat flux  $q / q_{\text{max}}$  estimates for subcooled and saturated boiling. (b), (c), and (d) show results for the fully trained forward propagator. In (b), accuracy degradation is minimal, with spikes occurring during timesteps of violent turbulence caused by rapid bubble detachment from the heater surface. (c) and (d) compare frames from the Flash-X simulation and predictions by the forward propagator for subcooled boiling.
The trained model can be a valuable tool to get fast estimates of heat flux, discussed in section 3.2. Heat flux is influenced by steep temperature gradients and dynamic temporal changes which presents a challenging problem. To further validate our models, we perform cross-validation to predict the heat flux trends observed in Figure 5. For each heat flux prediction, we holdout a simulation and train a forward propagator on the remaining simulations within the dataset. Even with partial training-50 epochs for subcooled boiling models and 100 epochs for saturated boiling models—we achieve compelling results. The heat flux predictions by  $\mathrm{UNet_{bench}}$  remarkably track the expected trend, as seen in Figure 3a.
Learning Fluid Dynamics. As an additional benchmark, we use the BubbleML dataset to train models to approximate both velocity and temperature dynamics. A challenging problem! Detailed results are in Appendix C.5. These follow similar training settings to the temperature-only models. Strikingly, we observe nearly the opposite results to predicting only temperature: the UNet<sub>bench</sub> model struggles when predicting both velocity and temperature fields jointly, while the UNet<sub>mod</sub> and the FNO variants perform comparatively better. All the models have difficulty capturing the trails of condensation that form in the temperature field. The vapor trails form but dissipate more quickly than expected. An example rollout of UNet<sub>mod</sub>, trained using the pushforward trick, is shown in Figure 4. We see that the flow closely aligns with the ground-truth simulation.
Open Problems. We reiterate several open problems in SciML that BubbleML offers an avenue to explore. The first is the creation of a new class of models that can learn multiple interrelated physics. We find that while UNet architectures work well at predicting temperature and FNO variants work well at predicting velocity, neither excels at joint prediction of temperature and velocity. The CNN-based UNet architectures slightly outperform FNO and its variants when predicting temperature, potentially due to CNN's capacity to naturally act as edge-detectors, and thus handle non-smooth
Figure 4: Velocity and Temperature Rollout. The left figure shows the first 80 timesteps of P-UNetmod's rollout, where color indicates velocity and streamlines illustrate direction of flow. Both the flow magnitude and direction align exceptionally well with the ground truth. On the right, (a) and (b) show the rollout errors for temperature and velocity predictions. The prefix "P-" denotes that the model is trained with the pushforward trick [57]. Notably, UNetmod starts with slightly better initial accuracy, but it degrades more quickly than the model trained with the pushforward trick. P-UNetmod behaves more stably during rollout.
interfaces more easily. On the other hand, FNO variants perform quite well at predicting velocities but still do not perform particularly well at temperature estimation, especially in capturing condensation trails. This is related to the second problem: developing neural operators that can handle nonsmooth and irregularly shaped interfaces. FNO variants seem to encounter difficulties in modeling temperature fields, which have sharp jumps along bubble interfaces where the temperature transitions from cool liquid to hot vapor. Conversely, the velocity field appears relatively smooth and thus may be composed of lower frequencies better captured by FNO variants. However, these models still miss sharp and sudden changes in velocity along bubble interfaces that are important for accurately modeling long-range dynamics. The third problem is improving stability during long rollouts. This is explored within the context of other datasets [28, 56], but it is particularly relevant for BubbleML. For instance, in subcooled boiling, after bubbles depart from the surface, they undergo condensation and generate vortices that gradually dissipate as they move upstream. To model these extended temporal processes accurately, autoregressive models must be stable across long rollouts. However, we observe that models experience instability, leading them to slowly diverge from the ground truth. The BubbleML dataset presents an opportunity to study these challenges in SciML.
5 Conclusions and Limitations
This paper introduces BubbleML, which addresses a critical gap in ML research for multiphase multiphysics systems. By employing physics-driven simulations, the dataset provides precise ground truth information for a number of boiling scenarios, encompassing a wide range of parameters. BubbleML is validated against experimental observations and trends, establishing its reliability in multiphysics phase change research. The two BubbleML benchmarks demonstrate applications in improving the accuracy of optical flow estimation and SciML modeling encountered in multiphase systems. Importantly, BubbleML extends its impact beyond its immediate applications. It resonates with broader challenges in SciML, serving as a foundational platform to study several open problems.
Limitations. Combining datasets might pose challenges due to their varying sizes. Because the resolution scales proportionally with the domain size, the constant relative spacing between grid cells allows the UNet model to be effectively trained on the merged boiling dataset. However, this approach does not extend to FNO, requiring domain decomposition methods [60] or downscaling strategies [61] to accommodate variable domain sizes. Note that the dataset is also exclusively composed of simulations due to the unavailability of experimental data with velocity, pressure, and temperature fields. Future work will involve collaboration with experimentalists to augment the dataset.
This work was partially supported by the National Science Foundation (NSF) under the award number 1750549, the Office of Naval Research (ONR) under grant number N00014-22-1-2063 (supervised by program manager Dr. Mark Spector), the Exascale Computing Project (ECP), a collaborative effort of the US Department of Energy (DOE) Office of Science and the National Nuclear Security Administration (NNSA) under grant number 17-SC-20-SC, and Laboratory Directed Research and Development (LDRD) program at Argonne National Lab under contract number DE-AC02-06CH11357. We thank Kamyar Azizzadenesheli, Colin Wright, and Nikola Kovachki for their suggestions in training FNO. We gratefully acknowledge the GPU computing resources provided on HPC3, a high-performance computing cluster operated by the Research Cyberinfrastructure Center at the University of California, Irvine.
[1] Samuel H Rudy, Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Data-driven discovery of partial differential equations. Science advances, 3(4):e1602614, 2017.
[2] Hanfeng Zhai, Quan Zhou, and Guohui Hu. Predicting micro-bubble dynamics with semi-physics-informed deep learning. AIP Advances, 12(3), 2022.
[3] Youngjoon Suh, Ramin Bostanabad, and Yoonjin Won. Deep learning predicts boiling heat transfer. Scientific reports, 11(1):1-10, 2021.
[4] Matthew T Hughes, Girish Kini, and Srinivas Garimella. Status, challenges, and potential for machine learning in understanding and applying heat transfer phenomena. Journal of Heat Transfer, 143(12), 2021.
[5] Zhihao Zhang, Xuehui Wang, and Yuying Yan. A review of the state-of-the-art in electronic cooling. e-Prime-Advances in Electrical Engineering, Electronics and Energy, 1:100009, 2021.
[6] Nugroho Agung Pambudi, Alfan Sarifudin, Ridho Alfan Firdaus, Desita Kamila Ulfa, Indra Mamad Gandidi, and Rahmat Romadhon. The immersion cooling technology: Current and future development in energy saving. Alexandria Engineering Journal, 61(12):9509-9527, 2022.
[7] Qi Lu, Yu Liu, Jian Deng, Xiaowei Luo, Zhiyong Deng, and Zhengpeng Mi. Review of interdisciplinary heat transfer enhancement technology for nuclear reactor. Annals of Nuclear Energy, 159:108302, 2021.
[8] Mahmoud Shatat and Saffa B Riffat. Water desalination technologies utilizing conventional and renewable energy sources. International Journal of Low-Carbon Technologies, 9(1):1-19, 2014.
[9] Daniel Attinger, Christophe Frankiewicz, Amy R Betz, Thomas M Schutzius, Ranjan Ganguly, Arindam Das, Chang-Jin Kim, and Constantine M Megaridis. Surface engineering for phase change heat transfer: A review. MRS Energy & Sustainability, 1:E4, 2014.
[10] Maziar Raissi, Alireza Yazdani, and George Em Karniadakis. Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science, 367(6481):1026-1030, 2020.
[11] Yohei Sato and Bojan Niceno. Pool boiling simulation using an interface tracking method: From nucleate boiling to film boiling regime through critical heat flux. International Journal of Heat and Mass Transfer, 125:876 - 890, 2018.
[12] Christian Kunkelmann and Peter Stephan. CFD simulation of boiling flows using the volume-of-fluid method within OpenFOAM. Numerical Heat Transfer, Part A: Applications, 56(8):631-646, 2009.
[13] Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021.
[66] Yuxin Wu and Kaiming He. Group normalization. In Proceedings of the European conference on computer vision (ECCV), pages 3-19, 2018.
[67] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016.
[68] Md Ashiqur Rahman, Zachary E Ross, and Kamyar Azizzadenesheli. U-NO: U-shaped neural operators. Transactions on Machine Learning Research, 2023.
[69] Jacob Helwig, Xuan Zhang, Cong Fu, Jerry Kurtin, Stephan Wojtowytsch, and Shuiwang Ji. Group equivariant fourier neural operators for partial differential equations. arXiv preprint arXiv:2306.05697, 2023.
[70] Jean Kossaifi, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, and Anima Anandkumar. Multi-grid tensorized fourier neural operator for high resolution PDEs, 2023.
[71] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter Battaglia. Learning mesh-based simulation with graph networks. In International Conference on Learning Representations, 2021.
[72] Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph networks. In International conference on machine learning, pages 8459-8468. PMLR, 2020.
[73] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019.
[74] Jiawei Zhao, Robert Joseph George, Yifei Zhang, Zongyi Li, and Anima Anandkumar. Incremental fourier neural operator. arXiv preprint arXiv:2211.15188, 2022.
[75] Frédéric Gibou, Liguo Chen, Duc Nguyen, and Sanjoy Banerjee. A level set based sharp interface method for the multiphase incompressible Navier-Stokes equations with phase change. Journal of Computational Physics, 222(2):536-555, 2007.
[76] Gihun Son and Vijay K. Dhir. A Level Set Method for Analysis of Film Boiling on an Immersed Solid Surface. Numerical Heat Transfer, Part B: Fundamentals, 52(January 2015):153-177, 2007.
[77] Gihun Son and Vijay K. Dhir. Numerical simulation of nucleate boiling on a horizontal surface at high heat fluxes. International Journal of Heat and Mass Transfer, 51(9-10):2566-2582, 2008.
[78] Miad Yazdani, Thomas Radcliff, Marios Soteriou, and Abbas A. Alahyari. A high-fidelity approach towards simulation of pool boiling. Physics of Fluids, 28(1):1-30, 2016.
[79] Yohei Sato and Bojan Ničeno. A sharp-interface phase change model for a mass-conservative interface tracking method. Journal of Computational Physics, 249:127 - 161, 2013.
A Additional BubbleML details
A.1 Dataset URLs and Links