# Convolution neural operator preconditioning for the solution of some heterogeneous PDES

Yanfei Xiang, Luc Giraud

► To cite this version:

Yanfei Xiang, Luc Giraud. Convolution neural operator preconditioning for the solution of some heterogeneous PDES. DTE - AICOMAS 2025 - Digital Twins in Engineering & Artificial Intelligence and Computational Methods in Applied Science, Kenjiro Terada, John Dolbow, Ferdinando Auricchio, Harald van Brummelen, Dominique Baillargeat, Charbel Farhat, Georges Karniadakis, Stefan Kollmannsberger, Trond Kvamsdal, Vissarion Papadopoulos, Ruben Sevilla, Feb 2025, Paris, France. hal-04961884

HAL Id: hal-04961884

https://inria.hal.science/hal-04961884v1

Submitted on 22 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# CONVOLUTION NEURAL OPERATOR PRECONDITIONING FOR THE SOLUTION OF SOME HETEROGENEOUS PDES

Yanfei Xiang* and Luc Giraud†

*

Concace, Inria project with Airbus CR & T and Cerfacs, France
CERFACS, 42 Av. Gaspard Coriolis, 31100, Toulouse, France
yfxiang0amber@gmail.com and https://www.linkedin.com/in/yanfei-xiang-b2a461209/?originalSubdomain=fr

†

Concace, Inria project with Airbus CR & T and Cerfacs, France
CERFACS, 42 Av. Gaspard Coriolis, 31100, Toulouse, France
luc.giraud@inria.fr and https://concace.gitlabpages.inria.fr/members/luc-giraud/

**Key Words:** Heterogeneous equations, Poisson equations, Darcy flow, Diffusion-Advection equations, Scientific machine learning, Convolution neural networks, Unsupervised operator learning, Subspace iterative methods on CPUs and/or on GPUs.

## ABSTRACT

This work exclusively focuses on using convolution neural operator learning for accelerating the solution of some heterogenous PDEs (including Poisson equations, Darcy flow, Diffusion-Advection equations) using the flexible GMRES [1] method. We use operator learning with U-Net [2] neural network architecture. For the sake of learning general information, the neural operator is trained with randomly generated datasets using an unsupervised approach. The trained neural operator exhibits significant generalization features with respect to different aspects. That includes the ability to address varying source terms, diffusivity terms, velocity field for advection, and varying boundary conditions for these heterogeneous equations. Furthermore, it also shows promising results for addressing wider range of the advection dominant situations, which is the challenging case for the advection-diffusion equations. Overall, this work demonstrates the efficiency of applying the neural operator learning as felxible preconditioner for subspace iterative linear solvers.

## REFERENCES

[1] Y. Saad, A Flexible Inner-Outer Preconditioned GMRES Algorithm. SIAM Journal on Scientific Computing. 1993

[2] O. Ronneberger, F. Fischer, and T. Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Lecture Notes in Computer Science, Springer International Publishing, 2015.
<figure><img src="image_2.png" /></figure>

# Unsupervised convolution neural operator preconditioning for the solution of some heterogeneous PDEs

MS044B - DTE & AICOMAS 2025, Paris, France

February 21, 2025

Yanfei Xiang (yfxiang0amber@gmail.com)

Concace team, Centre Inria de l'Université de Bordeaux
# Some heterogeneous fluid equations

**Parametric Poisson equation** (a common elliptic equation):

$$ \nabla^2 u(x) = \rho(x) \qquad (1) $$

**Heterogeneous Darcy flow** (groundwater flow through porous media):

$$ -\nabla(\rho(x)\nabla u(x)) = \rho(x) \qquad (2) $$

**Heterogeneous Convection-Diffusion equation:**

$$ -\nabla(d(x)\nabla u(x)) + c(x)\nabla u(x) = \rho(x) \qquad (3) $$

* let $x \in \mathbb{R}^d$ be the grid points in the $d$-dimensional $\Omega$ ($d = 1, 2, 3$);

* $u(x) : \mathbb{R}^d \to \mathbb{R}$ is the solution to be computed;

* $\rho(x) : \mathbb{R}^d \to \mathbb{R}$ is a parametric source/forcing term;

* $p(x) : \mathbb{R}^d \to \mathbb{R}_+$ represents the heterogeneous permeability of the porous medium (non-uniform $p(x)$ varies from point to point);

* $d(x) : \mathbb{R}^d \to \mathbb{R}_+$ denotes the heterogeneous diffusivity or viscosity field associated with diffusion;

* $c(x) : \mathbb{R}^d \to \mathbb{R}_+$ represents the heterogeneous velocity or force field responsible for convection.

<figure><img src="image_3.png" /></figure>
Varying Boundary Conditions (BCs)

For the heterogeneous fluid PDEs described in Equation (1)-(3), we consider three common classical BCs, Dirichlet, Neumann, and Cauchy:

$$
\left\{
\begin{array}{ll}
\text{Dirichlet BCs: } u(x) = \phi(x) & (\text{like } \phi(x) = 0 \text{ - zero-Dirichlet}); \\
\text{Neumann BCs: } \frac{\partial u}{\partial \bar{n}}(x) = C & (\text{like } C = 0); \\
\text{Cauchy BCs: } \frac{\partial u}{\partial \bar{n}}(x) + \alpha(x)u(x) = \gamma(x) & (\text{like } \alpha(x) = 1, \gamma(x) = 0).
\end{array}
\right.
\quad (4)
$$

Visualize Dirichlet, Neumann, and Cauchy BCs on a 2D domain Ω:

<figure><img src="image_4.png" /><figcaption>(a). Dirichlet BCs</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>(b). Neumann BCs</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>(c). Cauchy BCs</figcaption></figure>

<figure><img src="image_7.png" /></figure>
Varying Boundary Conditions (BCs)

For the heterogeneous fluid PDEs described in Equation (1)-(3), we consider three common classical BCs, Dirichlet, Neumann, and Cauchy:

$$
\left\{
\begin{array}{ll}
\text{Dirichlet BCs: } u(x) = \phi(x) & (\text{like } \phi(x) = 0 \text{ - zero-Dirichlet}); \\
\text{Neumann BCs: } \frac{\partial u}{\partial \bar{n}}(x) = C & (\text{like } C = 0); \\
\text{Cauchy BCs: } \frac{\partial u}{\partial \bar{n}}(x) + \alpha(x)u(x) = \gamma(x) & (\text{like } \alpha(x) = 1, \gamma(x) = 0).
\end{array}
\right.
\qquad (4)
$$

Visualize **their mixtures**, i.e., the combinations across different boundary:

Mixed Dirichlet and Cauchy (mix_ DC) BCs:

<figure><img src="image_8.png" /><figcaption>(d). mix\_DC BCs</figcaption></figure>

Mixed Dirichlet and Neumann (mix_DN) BCs:

<figure><img src="image_9.png" /><figcaption>(e). mix\_DN BCs</figcaption></figure>

Mixed Dirichlet, Cauchy and Neumann (mix\_DCN) BCs:

<figure><img src="image_10.png" /><figcaption>(f). mix\_DCN BCs</figcaption></figure>
Convolution neural networks preconditioning fluid operators

Transfer the differential expression of equations into the discrete form:

$$
\text{Fluid PDEs:} \quad \left\{
\begin{array}{l}
\mathcal{A}([p, \text{ or } d, c, \text{BCs}]) = \rho \\
\quad + \text{ the three classical BCs or their three mixtures on domain } \Omega \\
\quad \quad \text{Discrete derivatives } \downarrow \text{FDM, FEM, FFT, FVM, ...} \\
\mathcal{A}([p, \text{ or } d, c, \text{BCs}])u = b, \text{ with LinearOp } A \in \mathbb{C}^{n \times n}, u, b \in \mathbb{C}^n
\end{array}
\right.
$$

After discretization, numerical linear algebra methods, like subspace
methods, can be used to solve the discrete linear systems. However,

- purely using subspace methods *without preconditioning* is ineffective;

- generate a properly algebraic preconditioner could be possible (if the *n* is not too big) but *is* as challenging as solve the system directly;

- generally, the algebraic preconditioning *needs to be re-generated* for each of the parametric equations;

* except for the varying heterogeneous parameters, the application of different BCs also strongly effects the property of the linear systems
Convolution neural networks preconditioning fluid operators

Transfer the differential expression of equations into the discrete form:

$$
\text{Fluid PDEs:} \quad \left\{
\begin{array}{l}
\mathcal{A}([p, \text{ or } d, c, \text{BCs}]) = \rho \\
\quad + \text{ the three classical BCs or their three mixtures on domain } \Omega \\
\quad \text{Discrete derivatives } \downarrow \text{ FDM, FEM, FFT, FVM, ...} \\
\mathcal{A}([p, \text{ or } d, c, \text{BCs}])u = b, \text{ with LinearOp } A \in \mathbb{C}^{n \times n}, u, b \in \mathbb{C}^n
\end{array}
\right.
$$

In recent decade, the thriving **neural networks (NNs) solvers**, like the
**physics-informed neural networks (PINNs)**ª, is used to solve
**PDEs without discretization. However, these NNs solvers**

- are usually costly in training<sup>b</sup>, and may fail to solve challenging PDEs if without finely tuning of the hyper-parameters of the NNs;

- solve the PDEs without the theoretical convergence guarantee;

- generally reach limited accuracy and exhibit limited or NO network generalizability, thus re-training is required even it is costly.

<figure><img src="image_11.png" /></figure>

<footnote>ªLu et al., Physics-Informed Neural Networks with Hard Constraints for Inverse Design. SISC. 2021</footnote>

<footnote>ᵇStrubell et al., Energy and policy considerations for deep learning in NLP. ACL meeting, Italy. 2019</footnote>
Convolution neural networks preconditioning fluid operators

Transfer the differential expression of equations into the discrete form:

Fluid PDEs:
$$
\left\{
\begin{array}{l}
\mathcal{A}([p, \text{ or } d, c, \text{BCs}]) = \rho \\
\quad + \text{the three classical BCs or their three mixtures on domain } \Omega \\
\text{Discrete derivatives } \Downarrow \text{ FDM, FEM, FFT, FVM, ...} \\
\mathcal{A}([p, \text{ or } d, c, \text{BCs}]) = \mathbf{u} = \mathbf{b}, \text{ with LinearOp } A \in \mathbb{C}^{n \times n}, \mathbf{u}, \mathbf{b} \in \mathbb{C}^n
\end{array}
\right.
$$

* Goal of this work ⇒ To learn neural operator $\mathcal{F}_{\theta}$ that approximates

$A^{-1}$ by $\mathcal{F}_{\theta}([b, p, \text{ or } d, c, \text{BCs}]) \rightarrow u_{\theta} \sim A([b, p, \text{ or } d, c, \text{BCs}])^{-1}b$

Operator $\mathcal{F}_{\theta}$ can be used as a **flexible preconditioner** for the FGMRES method to accelerate the fluid equations with **varying b**, **varying p**, or **varying d, c, varying BCs**, and **varying domain Ω without re-training**.

* $\mathcal{F}_{\theta}$: CNNs (Convolution NNs) with U-Net architecture$^a$ with 4 depth

* Loss-function:
$$ \min_{\theta} \frac{\|Au_{\theta} - b\|_{2}^{2}}{\|b\|_{2}^{2}} \qquad (6) $$

<footnote>aRonneberger et al., U-Net: Convolutional Networks for Biomedical Image Segmentation. Inria 2015</footnote>
The generalized Arnoldi relation

Originally, the FGMRES methodª have the generalized Arnoldi relation

$$AZ_m = V_{m+1} \bar{H}_m, \quad (7)$$

where $V_{m+1} = [v_1, ..., v_{m+1}]$ is the Krylov basis, and $Z_m = [z_1, ..., z_m]$ is the preconditioned Krylov basis. $z_j = M_j v_j$ ($j = 1, ..., m$) with $M_j \sim A^{-1}$.

In our NNs preconditioned FGMRES case, we have

$$z_j = \mathcal{F}_{\theta}([v_j, *, *]), \quad j = 1, \dots, m, \quad (8)$$

where $\mathcal{F}_{\theta}$ is the trained non-linear neural operator satisfies $\mathcal{F}_{\theta} \sim A^{-1}$. This part is computed with single machine precision (float 32), the one used in the training process, and other parts are in double precision.

⇒ Given there is no information about the data structure of the Krylov basis, the neural operator $\mathcal{F}_{\theta}$ is trained with randomly generated datasets.

<footnote>ªY. Saad, A Flexible Inner-Outer Preconditioned GMRES Algorithm. SISC. 1993</footnote>
# Neural operator preconditioning for Poisson eqs.

<figure><img src="image_12.png" /></figure>

## Training (on 2 V100 GPUs, depth = 4, train. time: 1.28 hour):

* Fixed grid point $x \in \mathbb{R}^{n^2}$ in the 2-dimensional domain: $64 \times 64$ thus $n = 64$

* Random source term $b \in \mathbb{C}^{n^2} \sim \mathcal{N}(0, 1)$ satisfying standard normal distribution

* Training with zero-Dirichlet BCs suffice to obtain effective preconditioning operator

* Discrete with FFT instead of FDM: Preconditioning operator leverages the statistical properties of fluid data and captures the local sparsity structure of the discrete equations

* U-Net 2d: Training with single machine precision (float 32); Trainable params. - 831 K

## Testing:

⊗ Trained U-Net preconditioner can accelerate the solution of Poisson Eq. (1) with varying $b$, BCs, and varying domain size $\Omega$ (because of the discretisation invariance property from the convolution property)
Neural operator preconditioning for Darcy flows

<figure><img src="image_13.png" /></figure>

Training (on 2 V100 GPUs, depth = 4, train. time: 1.33 hour):

■ Fixed grid point $x \in \mathbb{R}^{n^2}$ in the 2-dimensional domain: $64 \times 64$ thus $n = 64$

■ Random source term $b \in \mathbb{C}^{n^2} \sim N(0, 1)$ satisfying standard normal distribution

■ Random heterogeneous permeability $p \in \mathbb{R}^{n^2+} \sim U(1, 2)$ uniformly distributed on the interval $[1, 2]$

■ U-Net 2d: Training with single machine precision (float 32); Trainable params. - 831 K

Testing:

◎ Trained U-Net preconditioner can accelerate the solution of Darcy flows. (2) with varying **b**, varying **p**, varying **BCs**, and varying domain size $\Omega$
# Neural operator preconditioning for Convection-Diffusion eqs.

<figure><img src="image_14.png" /></figure>

## Training (on 2 V100 GPUs, depth = 4, train. time: 1.44 hour):

* Fixed grid point $x \in \mathbb{R}^{n^2}$ in the 2-dimensional domain: $64 \times 64$ thus $n = 64$

* Random source term $b \in \mathbb{C}^{n^2} \sim N(0, 1)$ satisfying standard normal distribution

* Random heterogeneous diffusion term $d \in \mathbb{R}^{n^2+} \sim U(1, 2)$ uniformly distributed on the interval $[1, 2]$

* Random heterogeneous convection term $c \in \mathbb{R}^{n^2+} \sim U(1, 2)$

* U-Net 2d: Training with single machine precision (float 32); Trainable params. - 831 K

## Testing:

* Trained U-Net preconditioner can accelerate the solution of Convection-Diffusion equations (3) with varying **b**, varying **d**, varying **c**, varying **BCs**, and varying domain $\Omega$
General testing results

Solve three different fluid equations (eqs.): $A([p, or d, c, BCs])u = b$ on 2D domain $64 \times 64$ (thus $A \in \mathbb{C}^{64^2 \times 64^2}$) with zero-Dirichlet BCs:

- Stopping criterion: $\eta_b = \frac{\|Au - b\|}{\|b\|} \le \varepsilon$ with
  $\varepsilon = 10^{-12}$ for Poisson eqs. (1);
  $\varepsilon = 10^{-8}$ for Darcy flows (2) and Convection-Diffusion eqs. (3);

- Maximum dimension of the Krylov search space is set to be 512; number of restarts is set to be 10. Thus the whole maximum iteration is 5120;

- Source term $b = x + 0j \in \mathbb{C}^{64^2}$ with $x \in \mathbb{R}^{64^2} \sim \mathcal{N}(0, I)$.

<figure><img src="image_15.png" /><figcaption>Poisson eqs.: $Au = b$, with
epoch = 467,
val_loss = 0.05100</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Darcy flows: $A(p)u = b$, with epoch = 401,
val_loss = 0.15009</figcaption></figure>

<figure><img src="image_17.png" /></figure>

Convection-Diffusion eqs.:

$A([d, c])u = b$, with
epoch = 483,
val_loss = 0.09784
# 1. Varying the source term b

Solve three different fluid equations (eqs.): $A([p, or d, c, BCs])u = b$ on 2D domain 64 × 64 (thus $A \in \mathbb{C}^{64^2 \times 64^2}$) with zero-Dirichlet BCs:

Results of parametric Poisson eqs.: $Au = b$

<figure><img src="image_18.png" /><figcaption>(a). 5 GMRES results</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>(b). 5 FGMRES results</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>(c). 100 FGMRES results</figcaption></figure>

<figure><img src="image_21.png" /></figure>
# 1. Varying the source term b

Solve three different fluid equations (eqs.): $A([p, or d, c, BCs])u = b$ on 2D domain $64 \times 64$ (thus $A \in \mathbb{C}^{64^2 \times 64^2}$) with zero-Dirichlet BCs:

Results of heterogeneous Darcy flows: $A(p)u = b$

<figure><img src="image_22.png" /><figcaption>(d). 5 GMRES results</figcaption></figure>

<figure><img src="image_23.png" /><figcaption>(e). 5 FGMRES results</figcaption></figure>

<figure><img src="image_24.png" /><figcaption>(f). 100 FGMRES results</figcaption></figure>

<figure><img src="image_25.png" /></figure>
# 1. Varying the source term b

Solve three different fluid equations (eqs.): $A([p, or d, c, BCs])u = b$ on 2D domain $64 \times 64$ (thus $A \in \mathbb{C}^{64^2 \times 64^2}$) with zero-Dirichlet BCs:

Results of heterogeneous Convection-Diffusion eqs.: $A([d, c])u = b$

<figure><img src="image_26.png" /><figcaption>(g). 5 GMRES results</figcaption></figure>

<figure><img src="image_27.png" /><figcaption>(h). 5 FGMRES results</figcaption></figure>

<figure><img src="image_28.png" /><figcaption>(i). 100 FGMRES results</figcaption></figure>

<figure><img src="image_29.png" /></figure>
## 2. Varying the boundary conditions (BCs)

Solve three different fluid equations (eqs.): $A([p, or, d, c, BCs])u = b$ on 2D domain 64 × 64 (thus $A \in C^{64^2 \times 64^2}$) with 1 dirac distr. b:

### (a). GMRES results

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

### (b). FGMRES results

<figure><img src="image_33.png" /></figure>

<figure><img src="image_34.png" /></figure>

<figure><img src="image_35.png" /><figcaption>Poisson eqs.</figcaption></figure>

<caption>Darcy flows</caption>

<caption>Convection-Diffusion eqs.</caption>
### 3. Varying the permeability or diffusion term

Solve two different fluid equations (eqs.): $A([p, or d, c, BC])u = b$ on 2D domain $64 \times 64$ (thus $A \in C^{64^2 \times 64^2}$) with 1 dirac distr. b:

**Heterogeneous Darcy flows:** $A(p)u = b$

<figure><img src="image_36.png" /></figure>

<figure><img src="image_37.png" /></figure>

<figure><img src="image_38.png" /></figure>

**Heterogeneous Convection-Diffusion eqs.:** $A([d, c])u = b$

<figure><img src="image_39.png" /></figure>

<figure><img src="image_40.png" /></figure>

<figure><img src="image_41.png" /></figure>
## 4. Varying the convection term

Solve heterogeneous Convection-Diffusion eqs.: $A([d, c, BCs])u = b$ on 2D domain $64 \times 64$ (thus $A \in C^{64^2 \times 64^2}$) with 1 dirac distr. b:

<figure><img src="image_42.png" /></figure>

<figure><img src="image_43.png" /></figure>

<figure><img src="image_44.png" /></figure>
## 4. Varying the convection term

Solve heterogeneous Convection-Diffusion eqs.: $A([d, c, BCs])u = b$ on 2D domain $64 \times 64$ (thus $A \in \mathbb{C}^{64^2 \times 64^2}$) with 1 dirac distr. b:

<table><thead><tr><th>#<br>d. e. in R<sup>64</sup> on</th><th>Method</th><th>L</th><th>its</th><th>ET</th></tr></thead><tbody><tr><td colspan="5">Balanced case with d<sub>e</sub> ≈ 1:</td></tr><tr><td>d. e. [1, 2]</td><td>GMRES</td><td>1</td><td>408</td><td>35.5709s</td></tr><tr><td>d. e. [1, 2]</td><td>FGMRES</td><td>1</td><td>85</td><td>3.3399s</td></tr><tr><td>d. e. [25, 50]</td><td>GMRES</td><td>1</td><td>420</td><td>50.2798s</td></tr><tr><td>d. e. [25, 50]</td><td>FGMRES</td><td>1</td><td>233</td><td>17.5740s</td></tr><tr><td>d. e. [50, 100]</td><td>GMRES</td><td>1</td><td>460</td><td>54.7812s</td></tr><tr><td>d. e. [50, 100]</td><td>FGMRES</td><td>1</td><td>250</td><td>20.2132s</td></tr><tr><td>d. e. [100, 200]</td><td>GMRES</td><td>1</td><td>468</td><td>57.3766s</td></tr><tr><td>d. e. [100, 200]</td><td>FGMRES</td><td>1</td><td>283</td><td>24.2253s</td></tr><tr><td colspan="5">Diffusion-dominant case with d<sub>e</sub> ≫ 1:</td></tr><tr><td>d. e. [25, 50], e. [1, 2]</td><td>GMRES</td><td>1</td><td>325</td><td>21.8579s</td></tr><tr><td>d. e. [25, 50], e. [1, 2]</td><td>FGMRES</td><td>1</td><td>276</td><td>16.8212s</td></tr><tr><td>d. e. [50, 100], e. [1, 2]</td><td>GMRES</td><td>1</td><td>325</td><td>30.7067s</td></tr><tr><td>d. e. [50, 100], e. [1, 2]</td><td>FGMRES</td><td>1</td><td>302</td><td>30.4414s</td></tr><tr><td colspan="5">Convection-dominant case with d<sub>e</sub> ≪ 1:</td></tr><tr><td>d. e. [1, 2], e. [25, 50]</td><td>GMRES</td><td>1</td><td>2854</td><td>283.7008s</td></tr><tr><td>d. e. [1, 2], e. [25, 50]</td><td>FGMRES</td><td>1</td><td>1021</td><td>107.3359s</td></tr><tr><td>d. e. [1, 2], e. [50, 100]</td><td>GMRES</td><td>1</td><td>4436</td><td>597.5046s</td></tr><tr><td>d. e. [1, 2], e. [50, 100]</td><td>FGMRES</td><td>1</td><td>3937</td><td>374.7240s</td></tr></tbody></table>

Three cases for the Convection-Diffusion eqs.:

* Balanced case with $d_c \approx 1$: convection and diffusion effects are balanced

* Diffusion-dominant case with $d_c \gg 1$: diffusion significantly outweighs convection

<figure><img src="image_45.png" /></figure>

* Convection-dominant case with $d_c \ll 1$: convection dominates diffusion
# 5. Varying the domain size

Solve three different fluid equations (eqs.): $A([p, or, d, C, BCs])u = b$ from 2D domain $64 \times 64$ until $512 \times 512$ (i.e., from linear operator $A \in \mathbb{C}^{64^2 \times 64^2}$ - (F)GMRES to $A \in \mathbb{C}^{512^2 \times 512^2}$ - (F)GMRES $8^2$):

<figure><img src="image_46.png" /><figcaption>(b). 100 FGMRES results</figcaption></figure>

<figure><img src="image_47.png" /></figure>
Take home message of this work

**Goal:** To learn the **neural operator preconditioning** $F_\theta$ for accelerating the solution of some heterogeneous fluid equations.

Machine learning + iterative methods through **flexible preconditioning**

- Convolution neural networks with **U-Net** architecture

- **Unsupervised training** with randomly generated dataset

⇒ Trained neural operator preconditioners can be applied to:

- heterogeneous fluid equations with varying parameters; with varying boundary conditions;

- Convection-Diffusion equations with even larger range (up to 100 times) of the convection and diffusion terms; and apply to the challenging Convection-dominant cases.

Research report version of this work is accessible online at
Inria-HAL (https://inria.hal.science/hal-04886933v2).

→ Yanfei Xiang (yfxiang0amber@gmail.com)
# Overview of SciML + numerical iterative methods

Previous efforts in integrating recent Scientific Machine Learning (SciML) techniques with traditional numerical iterative methods (not exhaustively):

*   **Preconditioning learning.** Fahy et al., 2024, Li et al., 2024, Azulay & Treister, 2022, Xiang 2022, Ackmann et al., 2021, Battaglia et al., 2018, Götz & Anzt, 2018, ...

*   **Initial guess learning.** Aghili et al., 2025, Luna et al., 2021, ...

*   **Optimal parameters learning.** Khodak et al., 2024, ...

*   **Alternative algorithms.** Illarramendi et al., 2020, Rizzuti et al., 2019, ...

*   **Multigrid and algebraic multigrid.** Caldana et al., 2024, Dong et al., 2024, Han et al., 2024, Antonietti et al., 2023, Luz et al., 2020, Greenfeld et al., 2019, He & Xu, 2019, Hsieh et al., 2019, ...

*   **Domain decomposition methods.** Dolean et al., 2024, Heinlein et al., 2024, Howard et al., 2024, Klawonn et al., 2024, SISC, Klawonn et al., 2024, CSE, Verburg et al., 2024, Moseley et al., 2023, Heinlein et al., 2021, Heinlein et al., 2019, ...

*   **Recommendation system.** Chen et al., 2019, Sood, 2019, Yamada et al., 2018, Peairs & Chen, 2011, ...

On the other hand, integration SciML with some statistical approaches or non-iterative standard numerical linear algebra methods (not exhaustively):

*   **Random feature & Randomization.** Nelsen & Stuart, 2024, Lanthaler & Nelsen, 2023, Chen et al., 2022, ... & Boullé & Townsend, 2023, Boullé et al., 2023, ...

*   **Reduced order models.** Borcea et al., 2024, Coscia et al., 2024, Gowrachari et al., 2024, Ivagnes et al., 2024, Quaini et al., 2024, Demo et al., 2023, Gonnella et al., 2023, Ivagnes et al., 2023, Khamlich et al., 2023, Romor et al., 2023, Siena et al., 2023, ...
Registration and travel support for this presentation was
provided by the Concace team, Centre Inria de l'Université de Bordeaux

Thank you for your attention!

Questions ?
U-Net architecture

U-Netª architecture with 4 depth:

<figure><img src="image_48.png" /></figure>

Each blue box corresponds to a multi-channel feature map. The number of channels is denoted on top of the box. The x-y-size is provided at the lower left edge of the box. White boxes represent copied feature maps. The arrows denote the different operations.

<footnote>ªRonneberger et al., U-Net: Convolutional Networks for Biomedical Image Segmentation.</footnote>
Visualize Darcy flow with different permeability term p

Heterogeneous Darcy flows: $A(p)u = b$

<figure><img src="image_49.png" /></figure>

$p \sim U(1, 2)$

<figure><img src="image_50.png" /></figure>

$p \sim U(0.5, 2)$

<figure><img src="image_51.png" /></figure>

$p \sim U(1, 2.5)$

<figure><img src="image_52.png" /></figure>

$p \sim U(0.5, 2.5)$

<figure><img src="image_53.png" /></figure>

$p \sim U(0.5, 5)$

<figure><img src="image_54.png" /></figure>

$p \sim U(0.1, 2)$
Visualize three cases of the Convection-Diffusion eqs.

<figure><img src="image_55.png" /></figure>

d, c ∈ U(1, 2)

<figure><img src="image_56.png" /></figure>

d, c ∈ U(25, 50)

<figure><img src="image_57.png" /></figure>

d, c ∈ U(50, 100)

<figure><img src="image_58.png" /></figure>

d, c ∈ U(100, 200)

<figure><img src="image_59.png" /></figure>

<figure><img src="image_60.png" /></figure>

d ∈ U(25, 50),
c ∈ U(1, 2)

<figure><img src="image_61.png" /></figure>

d ∈ U(50, 100),
c ∈ U(1, 2)

<figure><img src="image_62.png" /></figure>

d ∈ U(1, 2),
c ∈ U(25, 50)

<figure><img src="image_63.png" /></figure>

d ∈ U(1, 2),
c ∈ U(50, 100)
Visualization of three 25 × 25 FDM coefficient matrices

<figure><img src="image_64.png" /></figure>
Visualization of three 25 × 25 FFT coefficient matrices

<figure><img src="image_65.png" /></figure>
Visualization of three 25 × 25 FFT coefficient matrices

<figure><img src="image_66.png" /></figure>
Visualization of three 25 × 25 FFT coefficient matrices

<figure><img src="image_67.png" /></figure>

<figure><img src="image_68.png" /></figure>

<figure><img src="image_69.png" /></figure>
0 [OL Survey] Nicolas Boullé, Alex Townsend. A Mathematical Guide to Operator Learning. arXiv, 45 pages, 2023, https://doi.org/10.48550/arXiv.2312.14688

1 [FGMRES] Y. Saad. A Flexible Inner-Outer Preconditioned GMRES Algorithm. *SIAM Journal on Scientific Computing*, 14(2), 1993, DOI: 10.1137/0914028

2 [Fluid Simulation] J. Tompson, K. Schlachter, P. Sprechmann, and K. Perlin. Accelerating Eulerian Fluid Simulation with Convolutional Networks. In *Proceedings of the 34th International Conference on Machine Learning*, volume 70, pages 3424–3433. PMLR, 2017, https://proceedings.mlr.press/v70/tompson17a.html

3 [PINNs] L. Lu, R. Pestourie, W.-J. Yao, Z.-C. Wang, F. Verdugo, and S. G. Johnson. Physics- Informed Neural Networks with Hard Constraints for Inverse Design. *SIAM Journal on Scientific Computing*, 43(6):B1105–B1132, 2021, DOI: 10.1137/21M1397908

4 [DeepXDE] L. Lu, X.-H. Meng, Z.-P. Mao, and G. E. Karniadakis. DeepXDE: A Deep Learning Library for Solving Differential Equations. *SIAM Review*, 63(1):208–228, 2021, DOI: 10.1137/19M1274067

5 [Theory in DNN] B. Adcock and N. Dexter. The Gap between Theory and Practice in Function Approximation with Deep Neural Networks. *SIAM Journal on Mathematics of Data Science*, 3(2):624–655, 2021, DOI: 10.1137/20M131309X

6 [OL with DeepONet] A. Kopaničáková and G. Em Karniadakis. DeepOnet Based Preconditioning Strategies For Solving Parametric Linear Systems of Equations. *arXiv*, 2024, DOI: 10.48550/arXiv.2401.02016

7 [U-Net] O. Ronneberger, F. Fischer, and T. Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. *In Lecture Notes in Computer Science*, Springer International Publishing, 2015, DOI: 10.1007/978-3-319-24574-4_28