SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Dataset: Links to download all the BubbleML datasets in Table 1 are available on the GitHub homepage. The dataset homepage is hosted at https://hpcforge.github.io/BubbleML/. All future versions of the dataset with the new links will be uploaded here.
Code: Code for training and evaluation of all the benchmark models are available within the same GitHub repository.
Model Weights: Weights for all the benchmark models are available in the model zoo. All the relevant benchmark results can be accessed on the same page.
DOI: The BubbleML dataset has a DOI from Zenodo: https://doi.org/10.5281/zenodo.8039786.
Documentation: We provide detailed descriptions for the fields tracked in the simulation data: https://github.com/HPCForge/BubbleML/blob/main/bubbleml_data/DOCS.md. The documentation discusses the layout of the data, important metadata, and some potential pitfalls.
Tutorials: We provide example Jupyter notebook to enable reproducibility of the benchmarks and findings in this study: https://github.com/HPCForge/BubbleML/tree/main/examples. These tutorials cover accessing simulation data, dataset schema, querying simulation parameters, and training a Fourier Neural Operator—discussed in Section C.1—using PyTorch.
A.2 Maintenance and Long Term Preservation
The authors of BubbleML are committed to maintaining and preserving this dataset. The authors will likely make extensions as we use BubbleML for future research. Ongoing maintenance also encompasses tracking and resolving issues identified by the broader community after release. User feedback will be closely monitored via the GitHub issue tracker. All data is hosted on AWS, which guarantees reliable and stable storage. Depending on usage, we may migrate to archival storage for long-term preservation.
Findable: All data is stored in an Amazon AWS S3 instance. All present and future data will share a global and persistent DOI https://doi.org/10.5281/zenodo.8039786.
Accessible: All data and descriptive metadata can be downloaded from the public links listed on the GitHub homepage. For added convenience, we provide a bash script for users to download the entire dataset at once.
Interoperable: All BubbleML data is provided in the form of standard HDF5 files that can be read using many common libraries, such as h5py for Python. Relevant metadata is stored with each simulation.
Reusable: BubbleML is released under the Creative Commons Attribution 4.0 International License.
A.3 Justification of simulation parameters
We discuss the rationale guiding the selection of simulation parameters in Table 1, addressing both the quantity of simulations and their resolutions/timesteps.
Number of simulations: The studies performed in BubbleML encompass diverse two-phase boiling phenomena. The number of simulations is determined based on the distribution of the variable being studied, keeping other factors constant. For example, in the case of saturated boiling, we choose a wall temperature range from  $60^{\circ}C$  to  $120^{\circ}C$ , with uniform intervals of  $5^{\circ}C$ , resulting in 13 simulations. This range captures the boiling transition from the bubbly regime to chaotic dynamics as the heat flux approaches criticality. However, when studying the effects of gravity, the scaling factor,  $\mathrm{Fr}^{-2}$  (Fr is the Froude Number), is chosen from the range  $10^{-4}$  to 1 using a logarithmic scale resulting in 9 simulations. This scale is essential to cover the vastly different gravity conditions spanning from the Earth's surface to the International Space Station.
Domain size: Exploring phenomena across varying scales and geometries is common practice. Such variations in sizes enable the exploration of a broad spectrum of heat transfer dynamics, bubble
dynamics, and phase change behaviors. The domain and heater sizes are chosen to replicate typical ranges in experiments [62, 63] while also taking into account the computational costs of simulations.
Spatial resolution: The domain resolution depends on the grid size of an individual 2D block. A block with spatial dimensions of  $0.5 \times 0.5$  (in non-dimensionalized units) is discretized into a grid of size  $16 \times 16$ . This resolution is determined based on grid sensitivity studies conducted for a single bubble case to ensure high-fidelity simulations [31]. This results in the spatial resolution sizes in Table 1.
Temporal resolution and timesteps: The temporal discretization in the majority of BubbleML datasets is set at 1 non-dimensional unit, equivalent to 0.008 seconds for FC-72. Additionally, we include several datasets with a finer discretization of 0.1 non-dimensional time. In both cases, we intentionally chose a discretization that is much larger than what the CFL condition mandates for the Flash-X solver. A significant advantage of neural PDE solvers is their ability to maintain reasonable approximations while taking much larger timesteps than traditional numerical simulations. However, this choice introduces a trade-off between dataset size and ease of use. The datasets with a 1 time unit discretization are potentially more accessible but might present challenges in achieving accurate results. Conversely, the datasets with a 0.1 time unit discretization are less accessible and may require distributed training. Yet, they are likely to achieve more accurate results due to more training data (i.e., more timesteps) and the presence of fine-grained physics.
A.4 Dataset Validation
Saturated and subcooled boiling. We first validate two distinct boiling phenomena. Saturated boiling refers to the state of a liquid when it reaches its boiling point, known as the saturation temperature,  $T_{sat}$ . At this temperature, the liquid is in equilibrium with its vapor phase, and bubbles start forming at the heated surface. The liquid is on the verge of vaporization, and any further increase in temperature can lead to the formation of vapor bubbles. This bubble formation is called nucleate boiling; a far more effective way to transfer heat than natural convection on its own. On the other hand, subcooled boiling is a complex process with evaporation and condensation occurring simultaneously. The heat flux imposed on the wall produces a thermal layer around it in which bubbles may nucleate and grow. However, condensation occurs as a bubble migrates into the bulk liquid region with temperature under saturation point,  $T_{bulk} < T_{sat}$ . Experimental findings [64] indicate that the heat flux increases linearly with the heater temperature,  $T_{wall}$  until reaching CHF, marking the transition from nucleate pool boiling to film boiling.
Figure 5: Bubble dynamics for saturated and subcooled pool boiling at different wall temperatures. (a) ONB during saturated boiling is marked by a bubbly flow regime. (b) CHF behavior, indicated by chaotic bubble dynamics and vapor film covering the heater surface. (c) Normalized heat flux,  $\overline{q} / \overline{q}_{\text{max}}$ , plotted against temperature difference between heater and liquid saturation temperature ( $T_{\text{sat}} = 58^{\circ}\text{C}$ ) aligns with the boiling curve for FC-72 in [64].
Figure 5 presents the bubble dynamics in two different regimes of saturated pool boiling: onset of nucleate boiling (ONB) and CHF. ONB occurs at low wall superheat and exhibits structured bubbly flow with consistent shape and size of departing bubbles from the heater surface. In contrast, the CHF regime is characterized by chaotic bubble dynamics. Remarkably, the heat flux,  $\overline{q}$  at the heater surface for both subcooled and saturated boiling shown in Figure 5c closely match the experimental boiling curve, specifically for Si(100) surface [64]. Beyond CHF, the heat flux reaches a plateau, indicating a stasis marked by the presence of large pockets of vapor cover on the heater surface, as shown in Figure 5b.
Figure 6: Gravity scaling of subcooled pool boiling: Bubble dynamics (black lines) and temperature distribution (contours) on (a) Earth gravity,  $a / g = 1.0$ ,  $t_1 < t_2 < t_3$  and (b) low gravity,  $a / g = 0.001$ . (c) Wall heat flux vs. gravity for 2D simulations and its comparison to the scaling model in [62].
Boiling at low gravity. Boiling is the most efficient mode of heat transfer on Earth and serves as a cooling mechanism for various thermal applications. However, in low gravity environments such as the International Space Station (ISS), the dynamics of bubble growth, merger, and departure, which significantly impact thermal efficiency, are influenced by the interplay between surface tension and gravity. Quantifying these effects is crucial for developing phase change heat transfer systems in such environments.
Figure 6 provides an overview of simulations conducted to compare against the gravity-based heat flux model proposed in [62, 63]. These simulations cover a range of gravity levels,  $Fr = 1 - 100$ . The gravity scaling,  $a / g = \frac{1}{Fr^2}$ , is used to scale values relative to the Earth's gravity ( $g$ ). Gravity separates pool boiling into two distinct regimes: buoyancy dominated boiling (BDB) and surface tension dominated boiling (SDB). The transitional acceleration,  $a_{trans}$ , which depends on the size of the heater, serves as the boundary between these two regimes.
In BDB, bubbles periodically detach from the heater surface when buoyancy takes over surface tension. Figure 6 illustrates the temporal evolution of temperature and the liquid-vapor interface during bubble departure events for  $a / g = 1$ . After departing from the surface, the bubbles undergo condensation due to subcooling, and generate vortices that gradually dissipate as they move upstream toward the outflow. This dissipation occurs as the vapor completely condenses into liquid. Decreasing gravitational acceleration results in larger departing bubbles and reduces the wall heat flux,  $\overline{q}$ . The scaling of  $\overline{q}$  with respect to gravity,  $\overline{q}_a / \overline{q}_g$ , follows the slope  $m_{BDB}$ . In SDB, the dynamics are dominated by the presence of a central bubble that remains on the heater surface and acts as a vapor sink for smaller satellite bubbles, as depicted in Figure 6 for  $a / g = 0.001$ . The figure also captures the transient behavior of the central bubble, which fluctuates in size due to the balance between evaporation and condensation leading to different types of vortical structures. The heat flux drops sharply by a value  $K_{jump}$ , which depends on the size of the central bubble [32], and the slope,  $m_{SDB} = 0$ . The gravity scaling of heat flux computed from simulations accurately matches the expected trend from the model [62, 63], validating the simulation results.
B Optical Flow
The BubbleML velocity data is in non-dimensional units (refer to appendix D). The non-dimensional velocities are first converted to real-world units by multiplying them with the characteristic velocity of the simulation. The velocities are then multiplied by a scaling factor (number of pixels per unit length in the image) and then divided by the frames per second value. This converts the velocities to pixels per frame units compatible with state-of-the-art optical flow models.
B.2 Benchmark Models
We consider two state-of-the-art optical flow models, RAFT [52] and GMFlow [53].
1. RAFT, Recurrent All-Pairs Field Transforms (RAFT), processes per-pixel features and forms multi-scale 4D correlation volumes between all pixel pairs. The flow field is iteratively updated through a recurrent unit that performs lookups on the correlation volumes. RAFT
begins with a feature encoder that extracts a feature vector for each pixel. This is followed by a correlation layer that produces a 4D correlation volume for all pairs of pixels. Subsequently, this data is pooled to create lower resolution volumes. Finally, a GRU-based update operator utilizes the data from the correlation volumes to iteratively refine the flow field, starting its calculations from zero.
2. GMFlow reformulates optical flow as a global matching problem, which identifies the correspondences by directly comparing feature similarities. It extracts dense features from a convolutional backbone network, uses a transformer for feature enhancement, a correlation and softmax layer for global feature matching, and a self-attention layer for flow propagation. It also has a refinement step to exploit higher resolution features, reusing the GMFlow framework for residual flow estimation.
B.3 Learning Bubble Dynamics Results
Analysis. For pool boiling test images, the models trained on FlyingChairs and fine-tuned on Boiling images  $(\mathrm{C} + \mathrm{B})$  achieve the best performance, as seen from Tables 2, 3, and 4. However, for flow boiling test images, a different trend emerges. As observed in Tables 5 and 6, the lowest errors are achieved by fine-tuning the Sintel benchmark models  $(\mathrm{C} + \mathrm{T} + \mathrm{S} + \mathrm{B})$ .
Figures 7 and 8 show a visual comparison between the outputs of the best fine-tuned and pre-trained models and the corresponding ground truth optical flows for RAFT and GMFlow respectively. In many instances (highlighted in red), the pre-trained models fail to capture the direction of movement of bubbles which are visibly corrected by fine-tuning. However, as seen in the ground truth optical flows, the bubble boundaries generally have different values than the interior regions. This gradient is responsible for the dynamic changes in bubble shapes between consecutive frames. The end point errors at these bubble-liquid interfaces remain significantly large even after fine-tuning. The bubble-liquid interface is the most important region, where a highly accurate optical flow prediction is necessary to enable downstream tasks such as scientific machine learning on experimental boiling data. This suggests the need to incorporate physics-informed learning rules and architectures into optical flow models to improve the interface end point errors.
Table 3: Results on optical flow data (B) generated from PB-Subcooled simulations. It consists of around 1100 training image pairs and around 300 test image pairs. It is to be noted that heater temperatures lower than  $90^{\circ}C$  are not used due to the lack of trackable bubbles in many frames.
Model	Method	Chairs(Val)	Things(Val)	Sintel(Train)	KITTI(Train)	Boiling(Test)
Clean	Clean	Final	F1-EPE	F1-all
C	RAFT	0.82	9.03	2.19	4.49	9.83	37.57	2.41
GMFlow	0.92	10.23	3.22	4.43	17.82	56.14	1.92
C+B	RAFT	0.89	11.64	2.58	5.17	14.01	47.53	0.63
GMFlow	1.34	12.23	3.93	5.27	22.83	64.41	0.63
C+T	RAFT	1.15	4.39	1.40	2.71	5.02	17.46	1.91
GMFlow	1.26	3.48	1.50	2.96	11.60	35.62	4.76
C+T+B	RAFT	1.24	6.45	1.54	2.82	7.35	20.27	0.70
GMFlow	1.46	4.04	1.66	3.11	14.50	44.35	0.67
C+T+S	RAFT	1.21	4.69	0.77	1.22	1.54	5.64	6.07
GMFlow	1.53	4.09	0.95	1.28	3.04	13.61	9.21
C+T+S+B	RAFT	1.35	6.49	0.87	1.48	1.79	6.31	0.64
GMFlow	1.66	4.43	1.04	1.42	3.99	18.83	0.65
Compute Resources. All models are fine-tuned using a single NVIDIA V100 GPU. The implementations use the recommended versions of PyTorch and other libraries used in the official repositories of GMFlow and RAFT.
Table 4: Results on optical flow data (B) generated from PB-Gravity simulations. It consists of around 1400 training image pairs and around 360 test image pairs.
Model	Method	Chairs(Val)	Things(Val)	Sintel(Train)	KITTI(Train)	Boiling(Test)
Clean	Clean	Final	F1-EPE	F1-all
C	RAFT	0.82	9.03	2.19	4.49	9.83	37.57	3.30
GMFlow	0.92	10.23	3.22	4.43	17.82	56.14	2.40
C+B	RAFT	0.95	10.98	2.78	5.76	15.52	49.88	0.90
GMFlow	1.42	12.57	4.06	5.41	23.61	66.38	0.90
C+T	RAFT	1.15	4.39	1.40	2.71	5.02	17.46	2.75
GMFlow	1.26	3.48	1.50	2.96	11.60	35.62	3.60
C+T+B	RAFT	1.31	6.65	1.64	2.93	8.39	21.36	0.96
GMFlow	1.41	3.95	1.64	2.97	44.07	14.41	0.97
C+T+S	RAFT	1.21	4.69	0.77	1.22	1.54	5.64	4.74
GMFlow	1.53	4.09	0.95	1.28	3.04	13.61	5.49
C+T+S+B	RAFT	1.35	6.44	0.90	1.49	1.81	6.27	0.92
GMFlow	1.63	4.43	1.03	1.40	4.29	20.93	0.92
Table 5: Results on optical flow data (B) generated from FB-Inlet Velocity simulations. It consists of around 1200 training image pairs and around 300 test image pairs.
Model	Method	Chairs(Val)	Things(Val)	Sintel(Train)	KITTI(Train)	Boiling(Test)
Clean	Clean	Final	F1-EPE	F1-all
C	RAFT	0.82	9.03	2.19	4.49	9.83	37.57	16.01
GMFlow	0.92	10.23	3.22	4.43	17.82	56.14	21.44
C+B	RAFT	1.21	14.99	3.62	6.78	22.07	60.64	10.13
GMFlow	1.50	13.96	4.52	5.89	23.45	65.96	7.01
C+T	RAFT	1.15	4.39	1.40	2.71	5.02	17.46	25.14
GMFlow	1.26	3.48	1.50	2.96	11.60	35.62	19.39
C+T+B	RAFT	1.62	9.09	2.19	3.77	13.81	32.13	9.19
GMFlow	1.45	4.15	1.78	3.05	15.74	48.34	7.24
C+T+S	RAFT	1.21	4.69	0.77	1.22	1.54	5.64	21.23
GMFlow	1.53	4.09	0.95	1.28	3.04	13.61	47.88
C+T+S+B	RAFT	2.18	9.32	1.54	2.90	2.57	10.93	9.68
GMFlow	1.72	4.80	1.18	1.65	4.69	24.56	6.88
Table 6: Results on optical flow data (B) generated from FB-Gravity simulations. It consists of around 1000 training image pairs and around 250 test image pairs.
Model	Method	Chairs(Val)	Things(Val)	Sintel(Train)	KITTI(Train)	Boiling(Test)
Clean	Clean	Final	F1-EPE	F1-all
C	RAFT	0.82	9.03	2.19	4.49	9.83	37.57	20.42
GMFlow	0.92	10.23	3.22	4.43	17.82	56.14	14.05
C+B	RAFT	1.14	12.65	3.30	6.13	17.94	53.76	4.45
GMFlow	1.30	13.02	4.05	5.41	23.17	65.33	4.24
C+T	RAFT	1.15	4.39	1.40	2.71	5.02	17.46	21.04
GMFlow	1.26	3.48	1.50	2.96	11.60	35.62	13.37
C+T+B	RAFT	1.41	6.85	1.97	3.89	10.96	27.82	4.24
GMFlow	1.37	3.89	1.71	3.05	14.29	45.03	3.94
C+T+S	RAFT	1.21	4.69	0.77	1.22	1.54	5.64	21.04
GMFlow	1.53	4.09	0.95	1.28	3.04	13.61	22.84
C+T+S+B	RAFT	1.59	7.29	1.13	2.14	2.22	9.34	4.12
GMFlow	1.62	4.42	1.04	1.43	3.86	17.76	3.93
Pool Boiling
Flow Boiling
Figure 7: Comparison of the best performing pretrained RAFT models with finetuned RAFT models on Pool Boiling and Flow Boiling images. In each row, (a) is the ground truth optical flow, (b) the optical flow predictions from the pretrained model, and (c) the optical flow predictions from the finetuned model. (d) shows the normalized end-point error at every pixel for the pretrained model output. (e) shows the normalized end-point error at every pixel for the finetuned model output
Flow Boiling
Figure 8: Comparison of the best performing pretrained GMFlow models with finetuned GMFlow models on Pool Boiling and Flow Boiling images. In each row, (a) is the ground truth optical flow, (b) the optical flow predictions from the pretrained model, and (c) the optical flow predictions from the finetuned model. (d) shows the normalized end-point error at every pixel for the pretrained model output. (e) shows the normalized end-point error at every pixel for the finetuned model output
Figure 9: Flow field color coding used in the above images. The displacement of every pixel is the vector from the center of the square to this pixel.
C Scientific Machine Learning
C.1 Benchmark Models
We focus on two classes of neural PDE solvers: Image-to-image models and neural operators. The Image-to-image models are variants of UNet, commonly used in computer vision tasks, such as image segmentation. These models are not always suitable as neural PDE solvers, where training data generated from numerical simulations may require using different resolutions. However, modern versions of UNet are still competitive in many benchmarks [55, 28].
1.  $\mathbf{UNet}_{\mathrm{bench}}$  is a simple variant of the original UNet architecture from 2015, which was modified in PDEBench [17] to use batch normalization. We also replace the tanh activation with GELU [17, 54, 65].
2.  $\mathbf{UNet}_{\mathrm{mod}}$  is a modern variant of UNet that gets impressive as a PDE solver. It has been modified to use wide residual connections and group normalization [55, 66, 67].
Neural operators have found great success as PDE solvers [15]. Typically, given an initial condition  $u_0$ , a neural operator is a function  $\mathcal{M}:[0,t_{max}]\times \mathcal{A}\to \mathcal{A}$ , where  $\mathcal{A}$  is an infinite dimensional function space, that satisfies  $\mathcal{M}(t,u_0) = u_t$  [13, 15]. I.e., they map some initial condition to any (discretized) time up to  $t_{max}$ . Training a model in this form demands many valid initial conditions and solved simulations to form a large training set. However, generating such a dataset proves infeasible in the case of BubbleML due to the complexity and computational cost of the simulations. To overcome this obstacle, we adopt the autoregressive approach, even for neural operators, a practice also observed in [28, 56, 57].
1. FNO is the Fourier Neural Operator [13, 15]. FNO learns weights in Fourier space to act as a resolution-invariant global convolution. This seminal work has been extended by many other neural operators. One notable downside is the memory requirements: each of the Fourier domain weight matrices consumes  $\mathcal{O}(H^2 M^D)$ , where  $H$  is the hidden size,  $M$  is the number of Fourier modes used after truncating high frequencies, and  $D$  is the problem dimension. In the case of BubbleML,  $D$  is 2 or 3.
2. UNO is a U-shaped variant of FNO [68]. This mimics the skip connections of a standard UNet but replaces the convolutional layers with Fourier blocks. We use an 8-layer architecture that mimics the vanilla UNet. The encoder reduces the input size and Fourier modes but increases the number of channels. The decoder increases the input size and Fourier modes while decreasing the number of channels. Since the number of Fourier modes decreases, much of the network only accounts for very low frequencies.
3. F-FNO is the Factorized Fourier Neural Operator [58]. As the name suggests, F-FNO factorizes the Fourier transform over problem dimensions. This reduces the number of parameters per Fourier space weight matrix to  $\mathcal{O}(H^2 MD)$ . This enables using more modes, scaling to deeper models, or learning higher dimensional problems. Further, F-FNO suggests changing the structure of the residual connections, so that they are applied after the non-linearity.
4. G-FNO is the Group-Equivariant Fourier Neural Operator [69], which extends group convolutions to the frequency domain. The proposed G-Fourier layers are equivariant to rotation, reflection, and translation. They also preserve the resolution invariance of neural operators. The Fourier space weight matrix has  $\mathcal{O}(H^2 M^D S)$  parameters, where  $S$  is the number of elements in the "stabilizer" of the group under consideration [69]. We focus on the symmetry group  $p4$ , which captures  $90^\circ$  rotations and translation. It's worth noting that the BubbleML simulations are not rotation equivariant due to the global effects of temperature and gravity. However, it is not uncommon for models that bake in some assumption of equivariance to perform well, even in cases where such assumptions are not entirely valid. Despite this incompatibility, we observe that G-FNO can still be successfully applied on BubbleML and achieves performance that matches or surpasses the standard FNO.
5. T-FNO is the "Tuckerized" Fourier Neural Operator [70], which replaces FNO's dense weight matrix with a low-rank approximation. Tucker decomposition significantly reduces the number of parameters while maintaining accuracy. Notably, T-FNO has been shown to improve model performance in low-data regimes [70]. This is particularly relevant for
BubbleML, where the complexity of simulations limits the size of the available dataset. The concept of decomposing the weight matrix can potentially be extended to other models, such as G-FNO, to yield comparable results.
C.2 Training Strategies
We implement several popular training strategies designed to make neural PDE solvers robust to instability during rollout. One source of this instability is distribution shift, which occurs when the model takes its prior outputs as input. The errors in the model's predictions accumulate, and the model is likely to diverge from the ground truth during long rollouts.
Teacher-forcing is the default training strategy [59]. The input to the model is always the ground truth of the previous timesteps. I.e., to predict the solution  $u(t)$ , the model input is the ground-truth solutions  $u([t - k, t - 1])$ .