PointSAGE: Mesh-independent superresolution approach to fluid flow predictions

Venue: ICLR_cc_2024_Workshop_AI4DiffEqtnsInSci
Published: 2024-02-09 18:51:04

Abstract: Computational Fluid Dynamics (CFD) serves as a powerful tool for simulating fluid flow across diverse industries. High-resolution CFD simulations offer valuable insights into fluid behavior and flow patterns. As resolution increases, computational data requirements and time rise proportionately, posing a persistent challenge in CFD. Recent efforts focus on accurately predicting fine-mesh simulations from coarse-mesh data, employing deep learning techniques such as UNets to address this challenge. Existing methods face limitations with unstructured meshes, due to their inability to convolute. Incorporating geometry/mesh information during training brings drawbacks like increased data requirements, and challenges in generalization to unseen geometries. To address these concerns, we propose a novel framework, PointSAGE a mesh-independent network that leverages the unordered, mesh-less nature of Pointcloud to learn the complex fluid flow and directly predict fine simulations, completely neglecting mesh information. With an adaptable framework, PointSAGE accurately predicts fine data across diverse point cloud sizes, regardless of the training dataset's dimension. Evaluations of various datasets and scenarios demonstrate notable results, showcasing a significant acceleration in computational time for generating fine simulations compared to standard CFD.

Main paper: 
 introduction: 
 The Navier-Stokes equation offers insight into complex physics, but its non-linearity presents challenges. Computational Fluid Dynamics (CFD) addresses these challenges with various numerical methods. Predicting intricate flows often requires fine resolutions, increasing computational requirements. In Direct Numerical Simulation (DNS), fine simulations demand significant CPU hours and approximately a terabyte (TB) of memory, while coarse simulations require half the time and 1/100th of the memory for the same phenomena (Hawkes et al. (2005) ). Hence, coarse grid simulations have become important due to computational efficiency. Yet, the persistent pursuit of understanding complex phenomena drives the ongoing demand for fine-mesh simulations. Recent advancements inspired by super-resolution techniques have introduced deep-learning methodologies for predicting fine-mesh simulations from coarse-mesh counterparts. Utilizing established architectures like MLP (Nair & Goza (2020) , U-Nets (Pathak et al. (2020) ), and GANs (Xie et al. (2018) ), GNNs (Pfaff et al. (2020) ) these approaches show promise in overcoming computational challenges to generate fine simulations. Despite recent advancements, current research faces challenges as models follow a traditional super-resolution approach, coarsening fine simulations through down-sampling techniques (Bode et al. (2021) ; Esmaeilzadeh et al. (2020) ). The training bias introduced by this process results in accurate outcomes, but real-life scenarios often involve coarse data not directly down-sampled from the fine counterpart. Researchers are now incorporating actual coarse data, aligning more closely with real-world scenarios (Sarkar et al. (2023) ).However, the current research primarily focuses on regular structured data (Sarkar et al. (2023) ; Pathak et al. (2020) ), limiting the model's adaptability to various data formats, including unstructured or irregular data. When facing unstructured data, these models struggle due to their inability to undergo convolution. Integrating mesh information during training adds complexity and data requirements, prolonging training time. Model training on specific geometries restricts predictive capabilities, especially for unseen geometries under same physical phenomena, leading to generalization issues. Accurately obtaining mesh information in real-world scenarios is challenging, com-promising the model's robustness for precise predictions in fine-mesh simulations. A framework that is independent of geometry/mesh information is crucial. Point clouds provide a comprehensive representation of 3D space by capturing the spatial coordinates of individual points within the domain. In recent times, they have gained prominence due to their ability to handle unstructured data, thanks to their unordered nature (Qi et al. (2017) ). This flexibility allows point clouds to capture intricate details and irregularities, making them well-suited for modeling diverse and complex scenarios.To overcome these challenges, we propose PointSAGE, a mesh-independent framework that utilizes point clouds derived from entities like cell centers and nodes. This approach provides flexibility for managing irregular and unstructured data due to its unordered nature. Inspired by the "classification network" in PointNet (Qi et al. (2017) ) and leveraging SAGEConv (Hamilton et al. (2018) ), PointSAGE captures global and local inter-dependencies within fluid flow features respectively. The model seamlessly predicts fine mesh data, irrespective of training set size/dimensionality, enabling accurate predictions for any value of n. Extensive testing across diverse datasets and scenarios demonstrates PointSAGE's remarkable performance, with significant reductions in training time compared to state-of-the-art (SOTA) techniques. 
pointsage super-resolution on point cloud: 
 In this section, we present the architecture for super-resolving coarse simulation point cloud data to match fine simulation point cloud data. The model f : C → F accurately maps the non-linear relationship between the coarse point cloud C ∈ R m×d and the fine point cloud F ∈ R n×d , where m ≪ n. Importantly, our approach is independent of mesh information, focusing solely on statevariable information. The architecture, shown in Figure 1 , comprises an Inverse Distance Weighting (IDW) upsampler, a Global Feature Extractor, and a Local Feature Extractor. to align its resolution with the fine-point cloud n. The interpolated feature V (n) at point n is calculated as:shared • • • • • • • • • • • • • • • • TNet • • • • • • • • • • • 𝑛 × 𝑑 𝑛 ×V (n) = ( n i=1 V i • e -(Di/D0) 2 )/ n i=1 e -(Di/D0) 2where D i is the Euclidean distance between coarse points m and fine points n, and D 0 is the correlation distance. The interpolated point cloud is then fed into the Global and Local Feature Extractors.Global Feature Extractor: The interpolated point cloud undergoes an input transformation using a TNet followed by two Conv1D layers. Another TNet is applied for feature transformation, and multiple Conv1D layers aggregate features to extract a global feature vector of dimension 1024, inspired by PointNet's classification section.Local Feature Extractor: Simultaneously, the interpolated point cloud undergoes local feature extraction using a graph G construction technique based on a radius near-neighbor approach. Multiple SAGEConv layers are then applied to capture local geometric structures and relationships, resulting in a local feature vector.The global and local feature vectors are concatenated and passed through MLPs to map the coarse point cloud C to the fine point cloud F . The Mean Squared Error is used as the loss function for training. Importantly, our model remains entirely mesh-independent throughout, ensuring robustness and versatility across diverse mesh representations. 
results and discussion: 
 To showcase our model's effectiveness, we experimented with three diverse datasets. We assessed our model's performance using 2D point cloud data from the forward-facing step simulation and extended it to 3D lid-driven cavity simulation data. Additionally, we compared our model to recent work, specifically PIUNet, which focuses on regular grid fluid flow prediction using 2D methane combustion data. This comparison aims to highlight the advantages of our approach, PointSAGE, utilizing point cloud data in the simulation.The transient CFD simulation of the forward step reveals insights into complex flow phenomena related to separated and reattached flows in a step-like configuration. We focused on predicting the shock wave generated from supersonic flow at the inlet to a rectangular geometry with a step. Details of the simulation are explained in the appendix A.1.  
conclusion: 
 We present PointSAGE, a point cloud-based superresolution model capable of predicting fine-mesh data from coarse-mesh input without prior knowledge of mesh characteristics. Leveraging point cloud data, PointSAGE demonstrates robust performance across diverse datasets and unseen geometries, showcasing generalizability and adaptability to different shapes and sizes. In a forwardfacing step simulation, our model accurately captures shock formations, achieving substantial improvements in RMSE and MAE compared to existing techniques. Similarly, in Lid-driven cavity simulations, PointSAGE exhibits superior predictive capability in turbulent scenarios within a 3D computational domain, showcasing significant reductions in training time and improved MSE. The key innovation lies in our mesh-independent approach, eliminating the need for detailed mesh information and setting the stage for future advancements in fluid flow simulations. 
 
 Appendix: A.1 CASE STUDY 1: FORWARD FACING STEP SIMULATIONIn this case study we investigate transient simulation of supersonic flow over a forward-facing step using sonicFoam solver in OpenFOAM. The sonicFoam solver is designed to solve compressible trans-sonic/supersonic laminar gas flow. The problem description involves a flow of Mach 3 at an inlet to a rectangular geometry with a step near the inlet region that generates shock waves and propagates downstream and get reflected from the walls and creates reflected shocks in the remaining length after the forward step till the time it reaches its steady state. This case study we have selected from the OpenFOAM tutorial this link.The 2D computational domain features a step with a height of 20% located at a distance of 0.6m from the inlet as shown in the Figure 3 . The experiment is conducted in a gas medium with a speed of sound given by √ γRT = 1 m/s. Thus, at the inlet, the flow is supersonic with a Mach number of 3 (U ∞ = 3 m/s), along with a pressure of 1 Pa and a temperature of 1 K. The aspect ratio of the Mass continuity:∂ρ ∂t + ∇ • (ρU) = 0 (1)Ideal gas:p = ρRT (2)Momentum equation for Newtonian fluid:EQUATIONThe energy equation for fluid (ignoring some viscous terms): The mesh is generated using the blockMesh utility, dividing the domain into uniform rectangular cells. For the fine mesh, the cells have dimensions of 0.03 m in the x-direction and 0.025 m in the y-direction, resulting in 42702 points for the point cloud. Conversely, the coarse mesh divides the domain into cells with dimensions of 0.12 m in the x-direction and 0.1 m in the y-direction, yielding 3352 points for the coarse point cloud used in our neural network. In this scenario, we conducted experiments using a dataset where the inlet velocity was varied in the range of 2 m/s to 5 m/s, with intervals of 0.25 m/s, while maintaining a constant aspect ratio of 3, as mentioned earlier. The simulations were performed on an Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz. For deep learning experiments, we partitioned the dataset into 80%/10%/10% for training, validation, and testing respectively, and executed the entire experiment on a Tesla P100 GP with 16GB VRAM.EQUATIONThe simulations conducted were of a transient nature, and PointSAGE demonstrated commendable accuracy in predicting features such as pressure and velocity at different time intervals, as illustrated in Figure 6 and Figure 7 respectively. Both figures reveal that our PointSAGE model effectively captures the propagation of shocks and their reflection within the rectangular domain following the step location. The training and validation for the PointSAGE training can be observe in this Figure 5 . In this particular scenario, we conducted experiments utilizing a dataset wherein the inlet velocity was systematically varied within the range of 2 m/s to 5 m/s, with intervals of 0.5 m/s. Additionally, we varied the aspect ratio within the range of 3 to 6. The variation in aspect ratio involves an increase in the length after the step location at 0.6 m. For instance, in the case of an aspect ratio of 3, the length of the section after the step is 2.4 m (total length = 0.6 + 2.4 = 3m). On the other hand, for an aspect ratio of 4, the length of the section after the step is 3.4 m (total length = 0.6 + 3.4 = 4m). In the context of deep learning experiments, we partitioned the dataset for training using aspect ratios 3 and 4, for validation with aspect ratio 5, and for testing with aspect ratio 6.The objective of this scenario is to evaluate the model's proficiency in effectively understanding and adapting to physical phenomena, specifically shock formation and reflection, within a given aspect ratio. Furthermore, the model is challenged to extend its predictions to another aspect ratio, adding complexity as an increase in the length after the step leads to intensified shock reflection and sudden alterations in flow behavior downstream. As illustrated in Figure 8 , PointSAGE demonstrates satisfactory predictions for essential features such as pressure and velocity. This success underscores the model's ability to effectively capture and forecast the dynamic behaviors of shocks under varying aspect ratios, emphasizing its efficacy in handling complex flow phenomena. As explained above, the computational domain comprises a 3D cube cavity with a width of 1m, as illustrated in Figure 9 . The cavity's height adjusts proportionally based on the aspect ratio. In this study, varying Reynolds numbers are achieved by altering the kinematic viscosity while maintaining a constant U lid value. Specifically, Reynolds numbers of 6000, 8000, 10000, and 12000 are considered, with grid sizes of 1/20, 1/30, and 1/40 employed for simulating coarse mesh data. For fine mesh data, a grid size of 1/120 is utilized. To enhance turbulence capture within the cavity, wall refinement is implemented in the fine mesh data's wall region, as depicted in the Figure 10 , whereas such refinement is omitted for coarse data. Scenario -2: This scenario illustrates Reynolds number extrapolation, where training and validation are conducted for flow at Re -6000, 8000, 10000, and the model is subsequently tested at a different Reynolds number, 12000. The grid size (1/30) and aspect ratio (1) remain constant. Figure 13 showcases the model's predictive prowess, with velocity contours plotted. The model accurately captures the turbulent nature of the flow, achieving an MSE of 3.5e-4 within just 120 seconds. Figure 14 showcase the training and validation loss.In this case, since the grid size and the aspect ratio remain constant, the number of points in the input/output point cloud remains constant, i.e., 27,000. However, if we alter either of them, the number of points changes. In the next two scenarios, the grid size is modified, leading to different point cloud dimensions for the training and testing datasets. 


Decision: The authors present in their paper PointSAGE, a point cloud-based superresolution model capable of predicting fine-mesh
data from coarse-mesh input without prior knowledge of mesh characteristics. It is interesting work that has been tested on numerous test-cases but the work is still preliminary in nature, given that the three cases are laminar flows and it's not clear how well this would scale to more complex unstructured 3D flows that are turbulent. Nevertheless the work is interesting and it should be accepted for a poster session.

Reviews: review: title: title: PointSAGE: Mesh-independent superresolution approach to fluid flow predictions
main_review: main_review: With the proposed method, the authors are predicting the fine mesh solution from the coarser mesh solution. Once the mapping is learned, the trained networks can be applied with varying field variables (say, velocity) or geometric shapes. 

In my opinion, the obtained results are not good, especially for the Euler equations, where it is important to capture shock waves with good accuracy and at exact locations. The author(s) needs to work harder before solving such problems.
title: title: A good paper and interesting for community, however, there are some serious questions that needs to be answered before acceptance.
main_review: main_review: The paper introduces an approach for more efficient prediction of fluid flow, by downscaling the mesh sizes in CFD simulations. Both the approach and results are interesting and should be useful for the community. I believe the work is of a good quality to be accepted for the workshop. However, there are some major (as well as some minor) comments I would like to address before final acceptance:


•	The authors claimed the model is mesh-independent, however, it is not very clear why it is. Maybe a few sentences will help. 
•	Regarding the previous comments, as I see in the appendices (e.g., Figs. 4, and 10), none of the cases use unstructured meshes, which is in contradiction to one of the main claimed novelties of the work. 
•	The separation of data for training of testing has not been discussed clearly (although I see a discussion about it in the appendix). It would be nice if the authors could mention how they were separated. What was the difference between the properties of the system comparing the training and testing data? that is important since if for instance the geometry or boundary conditions were different between two datasets, the outcomes of the study would be a higher quality. 
•	Regarding the previous comment, what are the extrapolation capabilities of the PointSAGE method? Is it reliable to train it once and then use it for many other cases? 
•	There are some problems with citations. Maybe authors could use \citep instead of \cite in the cases with parenthetical citations. 
•	Table 2 shows that for some cases the PointNet approach is superior considering both accuracy and run-time. What do the authors think about it?