Physics-Informed Machine Learning for Fluid Flow Prediction in Porous Media

Authors: ali  takbiri-borujeni
mohammad  kazemi
sam  takbiri
Venue: ICLR_cc_2024_Workshop_AI4DiffEqtnsInSci
Published: 2024-02-11 03:39:58

Abstract: The objective of this work is to predict grid-level flow fields in porous media as a priori to determining the permeability of porous media.
		A physics-informed ML model is developed by using the results from numerical fluid flow simulations of randomly distributed circular grains to represent the porous media. The deep U-Net and ResNet neural network architectures are combined to train a deep learning model that avoids vanishing gradient issues. 
		The model integrates continuity and momentum conservation equations into the loss function to ensure physical consistency. Additionally, we modify the padding function in convolutional layers to use circular paddings, mimicking periodic boundary conditions in LB simulations. By learning inter-grid communications, the ML model achieves precise flow predictions for new simulation sets with high accuracy. The robustness of the developed model is then tested for numerous variations of porous media that have not been used for developing the model.

Main paper: 
 introduction: 
 Rock permeability plays a crucial role in various fields of engineering science. Accurate estimation of rock permeability is essential for reservoir engineering (especially in the petroleum industry), geotechnical engineering, hydrogeology, and environmental projects Hughes (2013) ; Kazemi & Takbiri-Borujeni (2015) ; ?. Understanding rock permeability helps optimize fluid flow, predict groundwater movement, and design effective reservoir management strategies. Traditional methods for determining rock permeability and flow properties struggle with complex rock morphology and heterogeneity. High-fidelity numerical simulations based on image data, including the lattice Boltzmann Method, offer promise but remain computationally intensive (Sukop, 2006; Borujeni et al., 2013) . Leveraging recent advances in ML presents an opportunity to reduce computational costs while preserving physical interactions (Mo et al., 2019; Zhu & Zabaras, 2018) .With recent advances in data-driven methods, promising new approaches, have been introduced, significantly reducing the computational cost of the numerical simulations while preserving the fluid-fluid and fluid-solid physical interactions. Mo et al. (2019) developed a deep encoder-decoder convolutional neural network (CNN) for uncertainty quantifications of dynamic multiphase flow. Zhu & Zabaras (2018) used the Bayesian neural network to predict the pressure and velocity fields in a channelized permeability field. Araya-Polo et al. (2019) and Sudakov et al. (2019) developed a deep learning model to predict the permeability from images of porous structures. Sun et al. (2020) proposed a physics-constrained deep learning approach that enforces the initial and boundary conditions and the Navier-Stokes equations in the loss function to predict the flow velocities in various geometries. Several other implementations of deep learning have been published in recent years Kamrava et al. (2019) ; Mosser et al. (2017); Tian et al. (2020) .Building data-driven models requires computationally expensive lattice Boltzmann or numerical Navier-Stokes simulations. While post-processing these simulations can yield permeability values, a significant amount of high-fidelity data (e.g., velocity vectors at grid blocks) is lost. (Takbiri-Borujeni et al., 2020) introduced a deep learning algorithm that predicts pixel-scale velocity vectors for randomly generated 2D circle packs. This data-driven surrogate model replaces the numerical simulator, enabling accurate predictions without new lattice Boltzmann simulations. The model, based on pixel-level velocity vectors computed using LBM, employs a CNN with contracting paths and residual blocks. Despite achieving less than 15% error, the velocity profiles remain non-smooth. This work builds upon the work done by (Takbiri-Borujeni et al., 2020) by developing a physicsinformed ML model to predict pixel-level velocity fields in randomly generated circle packs. By embedding known conservation laws into the loss function, this approach significantly improves the smoothness and accuracy of grid-level velocity profiles compared to previous methods Takbiri-Borujeni et al. (2020) . 
lattice boltzmann simulations: 
 This research employs lattice Boltzmann (LB) simulations, known for effectiveness in irregular geometries and parallelizability (Sukop, 2006) . LB models fluid behavior with discrete lattice meshes, governed by the Boltzmann equation. For more detail on LB simulation refer to the Appendix. 
deep learning model development: 
 To incorporate physics into the proposed ML model, changes are made to the architecture and loss function of our former model (Takbiri et al., 2022; Takbiri-Borujeni et al., 2020) . This includes switching padding from reflective to circular to mimic LB simulations' periodic boundary conditions and adding divergence-free requirements to the loss function. To address non-smooth velocity predictions, circular padding was implemented to mimic periodic boundary conditions, ensuring values on opposite sides of the domain were close. Figure 1 compares velocity predictions in the x-direction using reflective padding from our previous work (Takbiri-Borujeni et al., 2020) with modified circular padding. Circular padding improves accuracy, particularly near boundaries.Physics-informed machine learning combines deep learning algorithms with physics, employing various integration techniques. Data for model training can originate from physical systems via experiments or simulations. Integration levels range from incorporating physics into loss functions to interleaving full physical simulations with machine learning outputs, particularly beneficial for systems with temporal evolution. This work builds upon previous research (Takbiri-Borujeni et al., 2020) to address limitations observed in velocity profiles. By integrating physics into the model architecture and modifying the loss function to reduce grid-level fluctuations, this work improves velocity predictions.Consider that the numerical domain constructed by spatial locations s, where the pixels of the binary input image x(s) and S = {s 1 , ..., s ns } are the index set for the spatial grid locations, s ∈ S ⊂ R ds (d s = 1, 2, 3) are the spatial locations. The simulations can be considered as mapping of x ∈ {0, 1} ⊂ R dxns to its corresponding solution u ∈ U ⊂ R duns ,EQUATIONwhere u = η(x). The purpose for building the surrogate model is to develop a new mapping function, û = F(x, θ), to be trained using a limited number of simulation data,D = {x i , u i } N i=1with θ as model parameters and N as the number of samples in the training, to approximate the predictions made by η mapping.The approach used in Takbiri-Borujeni et al. (2020) was to develop a model that estimates the target variable û(x(s), θ). The loss function of the standard approach is thus minimized such that,EQUATIONEQUATIONwhere Ω(θ) represents regularization (a measure of complexity of the model), and λ is the regularization strength, θ * denotes a set of (sub)optimal weights obtained from the optimization Here, we consider leveraging the governing equations in the loss function so that governing equations for the model predictions, û, are enforced (Sun et al., 2020) . The loss function, L phy , is defined by enforcing the governing equations Karpatne et al. (2017) .To develop L phy (θ), the mass and momentum conservation equations are considered. It should be noted that Navier-Stokes equations can be derived from LB models using the appropriate local equilibrium distribution function (Chen & Doolen, 1998) . The mass conservation equation for incompressible fluid turns into a divergence-free condition. However, since the density is an intrinsic variable in the LB equation, all LB simulation schemes are compressible to some extent and are never divergence-free (Luo et al., 2011) .EQUATION)On the other hand, the flow of incompressible fluids is described through Navier-Stokes conservation of momentum equation,∂u ∂t + (u.∇)u -ν∇ 2 u + b f = 0 (5)where ν is kinematic viscosity, and b f is the body force. Due to the steady-state condition of the flow, the term with time in the above equation is zero. Furthermore, for an incompressible fluid at a low Reynolds number, the equation reduces to,EQUATIONThe L phy (W, b) for the deep learning architecture is consisted of the continuity equations, Equation ( 4). Therefore, our loss function becomes,EQUATION)Figure 2 demonstrates the results for the same circular pack discussed in Fig. 1 . The smoothness issue seems to be resolved significantly in the physics-informed case. There are some irregularities still observed in the profile.Another loss function definition approach was implemented to improve the model further. In this approach, the mass and momentum conservation constraints for LB simulation results are subtracted from those for the model predictions, and thus, the modified loss function, L dev , is defined as,L dev (θ) = 1 n n i=1Deviations from Mass Conservation constraintsEQUATION)The final loss function can then be written asEQUATIONEQUATIONwhere θ * denotes a set of (sub)optimal weights obtained from the optimization. The next section describes the results of the final model. 
overview of model architecture: 
 512 2D circle pack images of size 128 × 128 pixels are generated by randomly placing circles of radius 10 to 30 pixels within a 128x128 pixel grid domain. The number of circles varies from 1 to 10, resulting in porosity values ranging from 20 to 60%. LB simulations are then used to determine the velocity fields and permeability of these images.The neural network architecture, referred to as U-ResNet, is inspired by U-Net and residual networks (Ronneberger et al., 2015; Hinton & Salakhutdinov, 2006) . It consists of downsampling followed by residual blocks and upsampling layers (Hinton & Salakhutdinov, 2006) . Convolutional layers, instead of pooling layers, are utilized for downsampling. Residual blocks enhance computational capacity at the end of downsampling.The network employs five strided convolutions to reduce input size by 64, compressing multiple 3x3 convolutions into one 7x7 convolution, thus reducing computational costs. Multi-resolution features are used to capture long-range dynamics, downsampling the first hidden layer five times before upsampling. Circular padding is implemented for all convolution layers to model periodic boundary conditions in LB simulations.Gradient vanishing with increasing network depth is mitigated by employing Resnet blocks (Zagoruyko & Komodakis, 2016) . Evaluation metrics like RMSE and R 2 assess model performance, with Fig. 3 illustrating the network architecture.Several metrics are considered to quantitatively evaluate models on validation data, namely, root mean squared error (RMSE),1 N N i=1 ∥ ûi -u i ∥ 2 2 , and the coefficient of determination (R 2 ), 1 - N i=1 ∥û i -u i ∥ 2 2 N i=1 ∥ū i -u i ∥ 2 2 . 
training, validation, and testing: 
 The ML model is trained on the input images and corresponding LB simulation results. All velocity values are normalized between zero and one. Model validation is conducted by evaluating its performance on the training data. Fig. 4 compares grid-level x-direction velocity components predicted by the physics-informed model with LB simulation results for two different input images. Contour maps of predicted velocity values closely resemble those from LB simulation. Fig. 4 shows the absolute error distribution at the grid level, indicating an accurate prediction of velocity values inside solid grains, near grain interfaces, and away from interfaces. Error distribution plots reveal negligible errors, with the majority within 2.5%. The physics-informed model is further tested by predicting velocity fields for images not included in the training set.Grid-level results are presented for two test images, showing x-direction velocity in Fig. 5 . Strong agreement is observed between the predicted velocity values by the physics-informed model and LB results, with grid-level errors within 6% and the majority within 2.5%.Grid-by-grid comparison of velocity field components is discussed, revealing predictions by the physics-informed model closely matching LB simulation results, with velocity profiles demonstrating agreement within 10%. Additionally, the permeability tensor's diagonal elements closely align with the unit slope line across all test cases, indicating robust agreement between the physicsinformed model and LB simulation results. 
conclusions: 
 We develop a physics-informed ML model using numerical fluid flow simulations from randomly distributed circular grains to represent the porous media. (Sukop, 2006) . Different models like single-relaxation-time (SRT), two-relaxation-time (TRT), and multiple-relaxation-time (MRT) offer variations in collision approximation (Ginzburg, 2005; Lallemand & Luo, 2000) .The D2Q9 model represents a two-dimensional system with nine possible fluid movement directions. External forces, like body forces, affect fluid velocity in equilibrium distributions (Sukop, 2006; Takbiri Borujeni, 2013) . LBM simulations apply body forces until kinetic equilibrium, extracting velocity fields for further model development (Sukop, 2006; Takbiri Borujeni, 2013) . A body force of 10 -7 in lattice units is used in x direction, with simulations running until kinetic equilibrium, and velocity fields are extracted for model development.  
 
 Appendix:  


Decision: The paper showed the application of a physics-based deep learning approach in the pore-scale simulation of single-phase flow in porous media, as a replacement for the Lattice-Boltzmann (LBM) approach. The work builds upon prior work in this area and whilst it is an interesting development, there is a lack of details on exactly how it is improving and the differences to prior methods. It is useful however to the community and should be accepted for a poster session.

Reviews: review: title: title: Physics-Informed Machine Learning for Fluid Flow Prediction in Porous Media
main_review: main_review: This work presents a simulation of porous media using physics-informed ML models. The idea is not new and has already been done in the literature; see, for example,

1. Faroughi, S. A., Soltanmohammadi, R., Datta, P., Mahjour, S. K., & Faroughi, S. (2023). Physics-informed neural networks with periodic activation functions for solute transport in heterogeneous porous media. Mathematics, 12(1), 63.
2. Lehmann, François, Marwan Fahs, Ali Alhubail, and Hussein Hoteit. "A mixed pressure-velocity formulation to model flow in heterogeneous porous media with physics-informed neural networks." Advances in Water Resources 181 (2023): 104564.
3. Faroughi, Salah A., Ramin Soltanmohammadi, Pingki Datta, Seyed Kourosh Mahjour, and Shirko Faroughi. "Physics-informed neural networks with periodic activation functions for solute transport in heterogeneous porous media." Mathematics 12, no. 1 (2023): 63.
title: title: The topic is interesting; and novelty of the work is acceptable. However the results are not clearly demonstrating the novelties applied.
main_review: main_review: The paper showed the application of a physics-based deep learning approach in the pore-scale simulation of single-phase flow in porous media, as a replacement for the Lattice-Boltzmann (LBM) approach. It seems the improvements applied in the training algorithm could bring some advantages compared to the basis model (previous work). I do believe the applied changes are very useful in improving the results, however, the importance is not shown very clearly. Also, 

	I understand the importance of developing reliable deep-learning methods for the simulation of pore-scale flow processes. I also understand the novelty of the work in using physics-based algorithms (PINNs). However, I believe the positive impact of using PINNs compared to the previous work (basis) is not shown properly. I think there could be cases that PINNs could bring much better responses compared to the basis model, to demonstrate the novelty of the work. It is important since adding a physics-based loss term adds significant time in the training of the model.  
	As far as I understood, there were two changes in the model compared to the basis model: using circular padding, and physics-based loss function. The relative advantage of each is not clear by looking at the results. However, I understand the limitations in the extended abstract, maybe it can be applied to the full paper later. 
	Equation 8 seems to be unclear to me. Did you use the residuals (PDE errors) from the LBM simulations? Why it needs to be applied if it is really an error? 
	should you add parentheses in eqs. 7 and 8 to apply the impact of sigma two both terms? 
	The paper starts with the importance of the measurement of permeability, … while in the results there is no information about the permeability predictions. It would be even better if there were a report about the improvements in the permeability prediction compared to the basis method. 

	There are a variety of writings and referencing issues, such as 4th line in the introduction section.
	Unclear sentence: “With recent advances in data-driven methods, promising new approaches, have been introduced, significantly reducing the computational cost of the numerical simulations while preserving the fluid-fluid and fluid-solid physical interactions.”
	The authors mention the term ‘pixel-level’, while it is not really representative of the author's meaning, I guess. At least at some point in the abstract, I prefer to clarify it by a more understandable term such as ‘pore-level’ or ‘pore-scale’ …
	Figure 2 does not seem to show what has been mentioned in the caption.
	Figure 5: it is not clear if it is showing testing data or training data.
	The size of the dataset used for the training is not clear enough.