Fengbo: a Clifford Neural Operator pipeline for 3D PDEs in Computational Fluid Dynamics
Authors: Alberto Pepe
Mattia Montanari
Joan Lasenby
Venue: ICLR 2025 Poster (2025)
Keywords: Clifford Algebra
Neural Operator
PDE Modelling
Geometric Machine Learning
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: Fengbo is a Clifford Algebra pipeline for solving 3D PDEs in CFD. With 42M parameters, it achieves competitive accuracy, outperforms five out of six models and enables 3D visualization, offering an efficient and interpretable geometry-aware solution.

Abstract: We introduce Fengbo, a pipeline entirely in Clifford Algebra to solve 3D partial differential equations (PDEs) specifically for computational fluid dynamics (CFD). Fengbo is an architecture composed of only 3D convolutional and Fourier Neural Operator (FNO) layers, all working in 3D Clifford Algebra. It models the PDE solution problem as an interpretable mapping from the geometry to the physics of the problem. Despite having just few layers, Fengbo achieves competitive accuracy, superior to 5 out of 6 proposed models reported in \cite{li2024geometry} for the $\emph{ShapeNet Car}$ dataset, and it does so with only 42 million trainable parameters, at a reduced computational complexity compared to graph-based methods, and estimating jointly pressure \emph{and} velocity fields. In addition, the output of each layer in Fengbo can be clearly visualised as objects and physical quantities in 3D space, making it a whitebox model.  
By leveraging Clifford Algebra and establishing a direct mapping from the geometry to the physics of the PDEs, Fengbo provides an efficient, geometry- and physics-aware approach to solving complex PDEs.

Published as a conference paper at ICLR 2025
FENGBO: A CLIFFORD NEURAL OPERATOR PIPELINE FOR 3D PDES IN COMPUTATIONAL FLUID DYNAMICS
Alberto Pepe 2: Joan Lasenby' Signal Processing and Communications Lab Department of Engineering
Mattia Montanari'
Impacc Engineering Laboratory
Department of Engineering Science University of Oxford
University of Cambridge
Trumpington Street, Cambridge, CB2 IPZ, UK {ap2219, 9, j1221 e cam. ac. uk
Parks Road, Oxford, OXI 3PJ, UK exct 37 900ox. ac. uk
2 Intelligent Cloud Technologies Laboratory Huawei Research Center Germany Austria Riesstraße 25, 80992 München, Germany
ABSTRACT
We introduce Fengbo, pipeline entirely in Clifford Algebra to solve 3D par- tial differential equations (PDEs) specifically for computational fluid dynamics (CFD). Fengbo is an architecture composed of only 3D convolutional and Fourier Neural Operator (FNO) layers, all working in 3D Clifford Algebra. It models interpretable mapping from the geometry the physics of the problem. Despite having just few layers, Fengho achieves compet- itive accuracy, superior to 5 out of 6 proposed models reported in Li et al. (2024) for the ShapeNet Car dataset, and it does so with only 42 million trainable param- eters, at a reduced computational complexity compared to graph- methods. and estimating jointly pressure and velocity fields. In addition, the output of each layer Fengbo can be clearly visualised as objects and physical quantities in 3D space, making it whitebox model. By leveraging Clifford Algebra and estab- lishing direct mapping from the geometry to the physics of the PDEs, Fengho provides efficient, geometry- and physics-aware approach to solving complex PDEs.
the PDE solution problem
INTRODUCTION
Many natural phenomena and complex systems, including electromagnetism and seismic waves, governed by partial differential equations (PDEs). Solving these PDEs enables the prediction of a system's s state evolution over time, which is valuable in applications such stock price estimation and weather forecasting. While PDEs often provide precise models of these systems, they are typi- cally too complex to solve analytically. Numerical methods, such as finite element analysis (FEA) and finite difference methods (FDM), are some of the well-established techniques for approximating solutions in complex geometries and handling boundary conditions Perrone & Kao (9755); Liszka & Orkisz 1980); Friswell & Mottershead (995).. However, these methods require significant com- putational resources, particularly when high-resolution solutions are needed.
In the past decade, machine learning (ML) methods have been applied to solve PDEs Carleo et al. (2019) Willard al. (2020); Karniadakis et al. (202I). ML--based methods can be several orders of magnitude faster than traditional numerical approaches, enabling rapid simulations while maintain- ing acceptable accuracy. This is particularly useful for applications requiring real-time predictions, such as weather forecasting and fluid dynamics simulations. Most MI. approaches blend physical laws with large datasets to efficiently approximate PDE solutions, significantly reducing computa- tional costs. Neural operators, in particular, have gained attention PDEs Li et al. 20020 a): Kovachki et al. (2023). They extend the idea of neural networks by learn- ing mappings between function spaces, rather than finite-diments dimenssinn vectors. Unlike conventional
novel approach to solving
"This work initiated during an internship at 2 and was completed thereafter.

---

Published as a conference paper at ICLR 2025
neural networks, which approximate specific solutions, Neural Operators aim to approximate the underlying operator of a PDE, allowing for generalization across different input conditions and con- figurations, making them highly efficient at solving PDEs.
Fengbo. This paper introduces a Neural Operator pipeline for computational fluid dynamics (CFD) cast entirely in Clifford Algebra. Named after the Taoist deity of the wind, Fengbo leverages the embedding of data within an algebra of choice in the form multivectors, which are the fundamental objects in Clifford Algebra, integrate physics and geometry data throughout the architecture. Its operators, layers and neurons are all expressed as multivectors in Clifford Algebra.
Multivectors are linear combination of objects, e.g. points, vectors, and planes, which can be employed to represent geometrical shapes but also physical quantities (c.g. pressure and velocity fields). This representation allows for an expressive and flexible encoding of complex relationships, ultimately leading to strong inductive bias to the neural network. This bias preserves geometric relationships between different quantities, can ensure equivariance under transformations Ruhe al. (2024); Peppe et al. (2024a), and allows for more descriptive models [randstettee et al. (2022); Pepe ct al. (2024c). As a result, we can achieve high performance using far fewer parameters compared to conventional models.
pressure geometry multivector
features
global features
feotures
pressure scaler field
3D Clifford Geometry Block
3D Clifford Physics Block
P
P
3D Clifford FNO
velocity geomatry multivector
velocity 3D vector field
3D Cliffrd Geometry Block
3D Clifford Physics Block
geometry domain
phyysios Jomain
Figure 1: The Fengbo architecture. Irregular geometries are discretised into fixed-resolution vol- umes of multivectors, blocks operate on the geometry of the PDEs domain, capture local features, ensure grade mixing and upsample the inputs; (ii) The Clifford FNO establishes mapping between the PDEs geometry and their solution; (ii) The physics blocks operate on physical quantities, i.e. target of the regres- sion. The entire architecture sits in 3D Clifford Algebra, guaranteeing interpretability.
which Fengho operates. It consists of three steps: (i) The geometry
Fengbo has three main components: (i) 3D Clifford Geometry block(s): one for each input geometry in the dataset, to mix elements of different grades in multivectors with geometrical meaning. (ii) 3D) Clifford Fourier Neural Operator (FNO): to capture global interactions and map multivectors from the geometry to the physics domain. We extend their implementation in BBrandsttterr al. (2022) to process full-grade 3D multivectors as opposed to only vector and bivector components. (iii) 3D
Clifford Physics block(s): Similar to (i), but for multivectors with physical meaning. There is one Physics block for each output physical quantity to estimate in the dataset.
We tested Fengbo on the two available 3D computational fluid dynamics (CFD) datasets generated and analysed in Li et al. (2024). Fengbo takes input multivectors representing the shape of the vehicles and estimates the pressure field on their surfaces well the velocity field defined the domain. does so with fewer than half the parameters required by the GINO architecture (Li al. (2024)) and by directly processing the geometries employed in CFD. Since every intermediate output in Fengbo is multivector with geometrical or physical meaning, Fengbo is a whitebox model that allows for clearer understanding of how data are processed and transformed from geometry to physics.
2 RELATED WORK
Learning methods in PDE modelling. A key challenge in applying machine learning (ML) to PDEs is ensuring that the model does not simply perform pattern recognition. Instead, the objective is for the model to capture the underlying physical principles governing the PDEs and accurately represent the geometry of the domain in which these equations are defined. Consequently, most models designed for PDES designed to address these requirements.
2

---

Published as a conference paper at ICLR 2025
Physics-Informed Neural Networks (PINNs) Raissi et al. (2017 2019): Cuomo al. (2022) for example, do so by incorporate the governing PDEs into the neural network training process and learn directly from them. This integration helps ensure that the solutions respect physical constraints and produce realistic outcomes, addressing the limitation of simpler MI. models that might fail to generalise unseen data. However, PINNs are limited to specific PDE and often require an additional Graph Network for spatial generalisation.
A similar philosophy is found in solver-in-the-loop methods Um al. (2020); Brahmachary & Thuerey (2024); Lippe et al. (024).. These hybrid methods combine an ML architecture with a relatively simple numerical solver. The numerical solver helps to refine the predictions of the ML model. ensuring that the solutions remain grounded in the physical constraints of the problem. Deep Galerkin Method (DGM) algorithms [Sirignano & Spiliopoulos (2018); Li et al. (2022a); AAl-Araii et al. (202) also fall in the same category. DGM algorithms are trained to satisfy the differential operator, initial conditions, and boundary conditions, proving to be particularly suitable deal with high-dimensional PDEs.
As CFD structures are often represented via point clouds, several approach exist in the literature based on extensions of the PointNet Qi et al. (2017a); Kashefi et al. 20 Nemati Taher & Subaşi (2024); Kashefi (2024) and PointNet++ architectures Qi et al. (2017b): Zhang & Caol (2024); [ao ct al. (022)). PointNet is simple yet effective way to handle irregular point clouds without resorting to grids, but fails at capturing global geometric context. PointNet++ mitigates this issue via multi- scale data processing the expense of higher computational cost.
Albeit versatile and flexible, PointNet-based methods are less accurate compared to more advanced PDE surrogates, including Transformer-based models Cao (2021): Li et al. (2022b); Xiao et al. 2023); Wu et al. (2024). Attention layers, especially when tailored to PDES, offer a significant accuracy boost while keeping the model size small. The major drawback of Transformers their computational complexity, generally ( Several attempts have been made in order to reduce the computational cost of such models, most notably the Galerkin Transformer Cao (202T). which reduces the cost of the quadratic Fourier-attention from O(N2d) down to (Ndd). with d the di- mensionality of feature space, and the Transolver Wu et al.] (2024), that introduces the Physics- Attention layer to learn over slices of the domain Ωdp with complexity of (N; ON), with f being a function linear in N with dependence on the model parameters 0. Given the significant computa- tional complexity of Transformer models, we regard them distinct category and instead focus our analysis on Neural Operators, of which Fengbo is an example. To justify our choice, a discus- sion computational complexity of such models and their comparison to Fengho is provided in Appendix E.
Neural operators have recently emerged as key architecture to tackle the problem of PDE mod- elling Li et al. (2020a b): Lütjens et al. (2022); Raonic et al.] (2024) Azizzadeneshelli et al. (2024). Neural operators differ from neural networks since they learn mappings between function spaces, or domains, instead of being function approximators like neural networks. When tackling PDEs, Neural Operators learn a mapping from input functions, which represent the initial or boundary con- ditions, to output functions, which represent the solution to the PDEs. They come come in several versions: Fourier Neural Operators, for example, operate in the frequency domain, where convolu- tions are more efficient at capturing long-range dependencies and periodic patterns in the data [i] ctt al. (2020a) 2023 2024), but Convolutional, Laplace and Graph Neural Operators have also been reported in the literature address specific problem requirements. GINO et al. (2024), for exam- ple, is a pipeline combining a Graph Neural Operator, that handles irregular shapes and maps them onto a regular grid in latent space, and Fourier Neural Operator, that processes the transformed input in latent space, that achieves state-of-tte the-the-rt performance on large scale 3D PDEs.
Hybrid methods that combine Neural Operators and Transformers also exist, such the general Neural Operator Transformer in Hao et al. (203)), which introduces the heterogeneous normalized attention and the geometric gating mechanism for 2D PDEs. Such methods, however, complex and up to x4 times larger than most Transformer-basee models as shown in Wu et al. (2024).
Clifford Algebra Networks. Clifford Algebra introduces multivectors extend linear algebra into a framework designed to couple multidimensional data and geometric transformations. Clifford Al- gebra has been shown be a valuable resource in several fields, including physics. computer vision and computer graphics Lasenby & Lasenby (2001); Lasenby & Doran (2001); Doran & Lasenby
3

---

Published as a conference paper at ICLR 2025
Dorst & Lasenby 2011). We present brief overview of Clifford Algebra we use in this pa- per in Appendix A. Clifford Algebra Networks are architectures that work with multivector-valiee inputs, outputs, weights and biases, and that perform geometric transformations in Clifford Al- gebra. The renewed interest in this type of networks arose precisely due to their potential in PDE modelling, but they have demonstrated promising results in several other fields, including computer vision and bioinformatics Brandstetter et al. ;: Roy et al. (2024); Ruhe et al. (2024); Pepe et al. (2024d a b): Hockey al. (2024). By encoding the geometry and physical properties directly into the algebra, they can represent and solve PDEs by capturing the relationships between variables in geometrically meaningful way. This approach allows for smaller yet more expressive and descriptive
2003);
models that can better generalise the PDEs solution.
3 METHOD
Notation. Unless stated otherwise, we will employ lowercase Latin letters for scalar quantities (e.g- Pl, v1). boldface Latin letters for vectors (c.g. x, n, p. v), uppercase Latin letters for multivectors (e.g. P.V.Q, B, W) or integers (e.g. K, M, C), lowercase Greek letters for real-valued maps (e.g. o, 1)) and uppercase, boldface Greek letters for multivector-valued maps (e.g. o, E). We use dash symbol distinguish multivectors describing geometrical quantities from those describing physical ones (e.g. P. P").
Navier-Stokes equations. The Navier-Stokes equations describe the motion of fluids. They read as follows:
ap
+ V. (pv) 0
Ot
Eq. represents the conservation of mass in fluid flow. It states that the time derivative of the fluid density plus the divergence of the mass flux pú must be zero. Here vó represent the fluid velocity. This ensures that mass is conserved within the fluid domain.
Oy P 0t
p(v V)
Eq. describes the conservation of momentum. It accounts for: the time rate of change of momen- Dd tum p Bt the convective term p(p V) which represents the transport of momentum due to the fluid's s velocity, the pressure gradient force <v the viscous forces y V which resist the flow of the fluid, and external forces f applied to the fluid. Here y is the dynamic viscosity of the fluid and é is the pressure. As many CFD applications, Eq. [ can be simplified by assuming that the fluid incompressible and forcing terms are present. We shall restrict this work to the steady state model, in which partial derivatives in time are null.
3.1 ARCHITECTURE
The Fengbo architecture, shown in Fig. D an architecture that maps the geometry of the domain of the PDEs onto their solution. Specifically, we are interested in estimating jointly the scalar pressure field ((x) PDp C R R and in the vector velocity field (xxx) Un I - R that satisfy Eq. 1-2 over irregular domain Ω2p We do as follows:
Voxelisation of the fluid domain. To deal with an irregular domain we need to support un- structured meshes which are commonly used in CFD. We do by generating regular grid of M x M x M voxels inside the fluid domain D. In general the voxels do not fit the boundary and the discretization parameter M should be sufficiently large capture good level of geometric de- tails. Our domain is a discrete volume of 3D space throughout, a simpler alternative to embedding in latent space or the use of graph representation of data.
Clifford Algebra embedding. We fill in each voxel i.j.k E D with multivector P D G(3, 0, 0) (see Appendix A for notation). in which D C R represents the discrete grid of voxels in which the multivector field is defined and G(3, 0, 0) is the 3D Clifford Algebra. We call P.ys the multivector associated with the voxel specified by indexes j, k. We construct multivectors P to have a scalar component my vector component p and bivector component B.
4

---

Published as a conference paper at ICLR 2025
geametyy multivettr G/3.0.0)
vector
bivector
trivector
P
componente:
binary mask (1)
3D coordinates (3)
(3)
None (1)
An example of pressure geometry multivector P. It has scalar component (the binary mask mp). 3 vector components (the 3D coordinates p) and bivector components (the dual of the vectors normal points p).
Figure 2:
The scalar part binary mask mp, included inform the network about which voxels are filled and avoid ambiguity between the origin and empty voxels in the vector part, since for both we have that P1 P2 = P3
The vector part p represents the N -opoint point cloud p E R of coordinates in 3D space and it encodes information about the shape contour of the object.
The bivector B represents the plane orthogonal to the normal n defined for each point in In other words, B is the dual of i.e. B /an, in which is the normal vector perpendicular to the mesh points on the car surface and I3 = e1 A e2 A eg is the G(3, 0, 0) pseudoscalar. B is an oriented plane and hence it can be interpreted containing informa- tion about the surface of the object.
The trivector component is left blank.
Since the output pressure field ((x) is defined at each point we call P the pressure geometry multivector. An example pressure geometry multivector given Fig. The general form of the pressure geometry multivector is:
P mp + p B
my + p1c1 P262 PsC3 + B12012 + B1sC13 B23
scalar
bivector
For datasets that include other physical fields, we define other input multivectors. For example, if the velocity vector field v is known, we construct corresponding multivector V. The multivector V 'D G(3, 0, 0) is also defined over regular grid of voxels. We construct V to have scalar component m, and a vector component v. The vector component v corresponds to the K K-point point clouds its corresponding binary mask defined similarly to mp. Since the output velocity field (x)) is defined over each point of call V the velocity geometry multivector. Each velocity geometry multivector of the form:
IR3, with K > N, defined for points surrounding the car surface, and m,
= m. + +V=
My +u1 + 0262 + UyC3
(4)
sealor
vector
in which, similarly P,
The scalar part is a binary mask m
The vector part v represents the N N-point point cloud v E R3 of coordinates in 3D space over which the velocity field defined.
The bivector and trivector components are left blank.
An example of V is shown in Fig. The geometry multivectors P and V/ are representative of the geometry simply because they are themselves the geometry of the PDEs domain.
3D Clifford Geometry block. We define the 3D Clifford Geometry block to be the module acting volumes of multivectors with sequence of three 3D convolutional layers in Clifford Algebra.

---

Published as a conference paper at ICLR 2025
velocity geometry muliveeter 6:3,0,0
scalar
vector
bivector
trivector
componente:
binary mask (1)
3D coordinates (3)
None (3)
None (1)
Figure 3: An example of velocity geometry multivector V. It has scalar component (the binary mask m.)) and vector components (the 3D coordinates v).
In Clifford Algebra Networks, weights and biases are multivectors W, G(3, 0, 0), and convolu- tions are performed via geometric products:
C
(Cout) iikk
Σ Gin
ΣEE
plane
W (onnfood +tittmk+n
+ D D(oce)
(5)
sjk
172
where the range of the summation of I, m, n is specified by the kernel size and Can, Cout are the input and output channels, respectively. The 3D Clifford Geometry block takes in input single geometry multivector (e.g. P. V) and it outputs C' channels of multivectors. It serves four purposes:
grade mixing: multivectors P, V in input it only contains elements of a certain grade. Convolutional layers allow grades to mix and have full multivectors in 3D Clifford Algebra (i.e. with scalars up trivector components).
capturing local interactions: convolutions traditionally used to extract feature from data which are close to each other in space.
filling up the volume: fitting an irregular grid into a regular one requires a high-resolution grid, meaning that most of the initial input volume is sparse. Convolutions with a large enough kernel avoid this issue by filling up the volume.
increasing the number of channels in input to the Clifford FNO block.
We refer to the output of the geometry block processing the shape over which the pressure field is defined (i.e. the pressure multivector P) as Qp and to the output of the geometry block processing the shape over which the velocity field is defined (i.e. the velocity multivector V/) Qv.
Clifford ENO block. The 3D Fourier Neural Operator (FNO) in Clifford Algebra learns mul- tivector valued function ((Q): G(3, 0, 0) G(3, 0) described by the 3D Clifford convolution
theorem of Brandstetter et al. (2022) :
Q' tb = -7000
in which Ka 3R33 - G(3, 0, is the learnable filter of the FNO and F and F-1 are the Fourier and inverse Fourier transforms, respectively, with the Fourier transform in G(3, O, 0) applied pointwise over each real coefficient of Q and defined as:
Q(E) = F(Q)(E) = Qo + Q1e1 + Q2ec + Q3es + Q12612 + Q13e13 + Q23223 + Q123e123. (7)
Q is defined the sum of all the multivectors in output of the Geometry blocks. The codomain of ((Q) is also multivector valued, and each multivector in output of the 3D Clifford FNO, which we refer Q', is defined grid with the same resolution M of the inputs. The FNO captures global interactions within the geometry and maps the input multivectors from a geometrical to a physics domain.
3D Clifford Physics block. The 3D Clifford Physics block is analogous to its Geometry counter- part. differs from it since brings the C channels of multivector Q' in output the FNO down to 1. As estimate two different quantities, have two different blocks to output P' and V", the pressure physics multivector and velocity physics multivector, respectively, for which we set
(P') 00
(8)

---

Published as a conference paper at ICLR 2025
Table 1: The three steps of the Fengbo pipeline.
Inpux
Ourpur
Purpose
1. 3D Chifford Geomeury blocks
P (B) tut
(Q
local, upsmple, grade mixing
2. 3D Chifford FNO
Q L Q
global, PDE modelling
Cliffodd blocks
local, dowssample, grode maxing
(V'), = e1 + 1/22 + 4/36
(9)
where (-)x is the grade projector operator, which extracts the k-grade element out of the multivector. In short, Fengho models the PDE solution problem as a mapping E(-) of 3D (geometry) multivectors onto 3D (physics) multivectors in 3D Clifford Algebra G(3, 0, 0), i.c.
P = (P)
(10)
in which P [P:) i1 with Ng the number of input geometries in the dataset, and P = {P:) Np with Np the number output physical quantities to estimate. Ng and Np determine the number of Geometry and Physics blocks in Fengbo, respectively. The steps in the Fengbo architecture summarised in Table Additional insight on each block is provided in Appendix B.
EXPERIMENTS
4.1 DATASETS
ShapeNet Car. The ShapeNer Car dataset is subset of the larger ShapeNet 3D model repository consisting of thousands of realistic 3D car models employed a CFD simulation with constant inlet flow velocity. It contains 500 shapes for training and 111 for testing. For this dataset, N₂ = 2, N = 2, i.e. = V} (two inputs geometries) and p' (P', (two physical quantities to estimate, defined over the two different geometries).
Ahmed Body. The Ahmed Body dataset consists of CFD simulations with varying inlet flow velocity V'nn It contains 500 parametric variations of CFD simulations over simplified car models for training and 51 for testing. For this dataset, Ng = 1, N, 1, i.e. P = (P) (a single velocity field information
input geometry) and P' {P') (one physical quantity estimate, provided). The inlet velocity is a crucial component since the output pressure field range depends on it. We embedded as the trivector component P since has single component one direction, i.c. ei + Deg + Oeg. We do so by setting (P)s (mptin)12a in which mp is the binary mask.
4.2 METRICS
We assess the quality the pressure and velocity fields estimation through relative L2 norm (a percentage), defined as:
Cp (PPoo)o - (P')olle = 1Ø(x) 8(x)112
(11)
|o(x)]12
Cv ((Vr)): (V')lla [x(x) {xx)112
(12)
II(
(x)]2
in which è, & represent estimated pressure and velocity fields via Fengbo, extracted the grade-0 and grade-1 component of estimated physics multivectors P', V', respectively, while ó, 1 represent ground truth fields, extracted as the grade-O and grade-1 component of ground truth physics mul- tivectors P V respectively. The relative 1.2 norm has also been employed as loss function during training. Training details are discussed in detail in Appendix C. Code scripts can be found here, while trained model weights are available upon request.
4.3 RESULTS
Results are summarised Tables 23 for the ShapeNet Car and the Ahmed Body datasets, respec- tively. Fengho outperforms all variants of vanilla Fourier and Graph Neural Operators, well as

---

Published as a conference paper at ICLR 2025
5
(9)
(6)
Figure 4: (a) Ground truth pressure field (b) Fengbo's estimated pressure field and (c) their relative error for a test shape in the ShapeNet Car dataset.
Table Training and testing errors pressure and velocity prediction on the ShapeNet Car dataset. Fengho results have been obtained with a; 15, for pressure and with a, {1,50} for velocity, see Appendix C.
Model
Pressure
Velocity
training error
testing ertor
training crror
testing orror
MLP
13.0
5.12
PointNet Oi
7017a
11.0
4.94
Graph U. Naa & 2019
11.0
4.71
GraphSage Hamilion
10.5
4.61
MeshGraph Net et al.
2020
7,81
3.54
GNO (20200
Geo GND Lie 217M0
10.8
15.9
16.7
UNet Ronne bepter
12.5
12.8
FND Lr
9.65
9.42
GINO (encoder-deadin) La 2024
7.95
9.47
3.86
GINO (decoder) Li
6.37
7.12
Fengbo [ouss]
6.94
8.86
3.23
3.47
Table 3: Training and testing errors in pressure prediction on the Ahmed Body dataset.
MAGaNet
CINO
rdi,
Fengho [wurs]
training texting
9.88
126
9.30
UNet and Mesh GraphNet, and it yields comparable results GINO. For the ShapeNet Car dataset, for example, Fengbo is able to estimate the pressure field with a 0.6% lower relative L2 norm com- pared the GINO in its encoder-decoder (e-d) configuration, but with 1.6% higher error compared to its decoder-only (e) configuration. Fengbo does so, however, with 60% fewer trainable parame- ters, with a reduced computational cost which does not depend the degree of graph representa- tion of the input data, and being the only architecture reported able to do so while jointly estimating the scalar pressure field and the 3D velocity vector field. Fengbo achieves competitive accuracy thanks to this coupling of physical quantities, which allows to do so through simple convolutions on coarsely discretised meshes. This especially notable when compared more sophisticated architectures, including Geo-FNO Li et al. (2023). which degenerates when dealing with complex geometry, as shown in et al. (2024); Wu et al. (2024), despite it being precisely designed to learn to deform irregular domains onto a regular grid to be fed into the FNO. Models such as ONO Xiao jtt al. (2023) and OFormer [i et al. (2023), which are transformer. bassme,--aaed,, also become unstable when dealing with large meshes, as found in Wu et al. (2024).
(8)
(c)
Figure 5: (a) Ground truth velocity field (b) Fengbo's 's estimated velocity field and (c) their relative error for test shape in the ShapeNer Car dataset.
8

---

Published as a conference paper at ICLR 2025
(9)
Figure 6: (a) Ground truth pressure field (b) Fengbo's 's estimated pressure field and (c) their relative error for a test shape in the Ahmed Body dataset.
Interestingly, estimating the velocity vector field appears to be an easier task to tackle. This is likely due the significantly smaller variance of the velocity data as opposed to the sharp pressure variation over the car surface, as well as the fact that the velocity field (x(x) is defined over point cloud containing 10x more points as opposed to @(x). This is mirrored also the smaller gap between training and testing errors. The additional estimation of the velocity vector field does not imply a computational overhead, since the vector and pressure fields cxist naturally within the multivector-based formulation of the problem and they are both embedded in fixed-size volume. Note that this would not apply to wraph-based methods, in which larger cell count would mean a larger number of nodes, increasing drastically the computational complexity (see Table 4
Similar observations can be made for the Ahmed Body dataset. Fengbo outperforms all models reported, with the exception of some GINO configurations depending the choice of the radius of the Graph Neural Operator module. It worth mentioning that experiments in et al. (2024) could benefit from the joint estimate of the wall shear stress, a physical parameter which the version of the dataset we employed. We are positive that regressing also on wall shear stress in joint fashion, just like pressure and velocity for ShapeNer Car, could bring down the error of 10.7% we obtained on the test set with Fengbo. Note also how Fengbo attains a training error of just 8%, the lowest out of every other model reported, indicating how additional parameter optimisation could be performed and likely reduce overfitting to bring the error down even further.
missing in
Comparison of estimates with Fengbo and corresponding ground truth pressure fields are given in Figs and in Fig. 6 for the ShupeNer Car and Ahmed Body datasets, respectively. Note how errors in the pressure field are generally isolated points a more or less uniform region with relative error close to zero. We convinced that by simply smoothing the predicted field we could mitigate this issue and improve performance.
Scalar vector components channeis)
Figure 7: Intermediate outputs are interpretable physical quantities: Q', the output of the 3D Clifford FNO block, is processed by N, = 2 3D Physics blocks in parallel to obtain ((x) and (xx)) the pressure and velocity fields, respectively.
In Fig. 5 the ground truth and estimated velocity fields for a test case in the ShapeNer Car are reported. Note how the range of the relative L2 error in Fig. 5 is significantly smaller than the ranges in Figs. 4 Gc. This is likely due to the denser, larger point clouds over which (xx) is defined. Note also how larger errors are concentrated in the areas surrounding the outline of the car. Small discontinuities in the estimated field with respect to ground truth can be noticed in Fig. [p,
for example in the bottom right section: just like for d(x), we believe that smoothing the estimated field (x) further reduce the prediction error.

---

Published as a conference paper at ICLR 2025
An example of the interpretabilityy offered by Fengbo is given in Fig. 7 Q', the multivector in output of the 3D Clifford FNO module, processed by the the Clifford Physics blocks to obtain mul- tivector P and V' of which we extract the scalar part o(x) and the vector part (x()., respectively. Note that we are still dealing with full grade multivectors defined over the entire domain D, but for the sake of visualisation we only plot the scalar and vector component masked by m, and my respectively. As the velocity and pressure field are processed, is it possible to have visual intuition on how they are being transformed into the final estimate. As the quantities plotted are scalars and vectors throughout, they carry physical meaning and cannot be interpreted anything else but pres- sure and velocity fields, therefore we can claim that Fengbo is whitebox model. This concept of interpretable convergence analogous to that presented Pepe et al. (2024a) for protein structures and in Pepe al. (2024d) for camera poses.
Table 4: Comparison of different models. d is the maximum degree of the graph, D is the feature space dimensionality. *: See Appendix E.
Model
Range plobal
Complexity
Irregular Grid
Discretisation Convergent
PointNet(Qi et
((N)
PointNet+ er
1201
local-global
(N log N)
GNNSScarsllll et al
2000
local local
((Na)
CNNILeC
O(N)
X
UNet Roumeberrr el
2015
global
O(N)
X
Translormers
Varwami
radius r local local-glohal
O(N
Transolver
O + N
Galerkin
2021
global
O(ND2)
MeshGraphNet Pratt Mitee al
2020
local-global
O(Na)
GNO
global
O(Nd)
FNO La 20240
global
O(N log N)
X
Geo-FNO Lirel
global
O(N log N)
GINO al 2024
local-global
ON log N Nd)
Fengto [ours]
local-global
O(N log N)
X
g
Fengbo has computational complexity of O(N log N) (see Table 4) the embedding into 3D volume has complexity O(N), and the limiting component on the computational complexity is given by the Clifford FNO module, with complexity O log N). Moreover, Fengbo's accuracy is minimally impacted by smaller grid resolutions, making it robust to coarses discretisations and hence discretisation convergent (see Ablation Study in Appendix D).
CONCLUSIONS
We introduced Fengbo, Neural Operator pipeline able solve large-scale, 3D PDEs over complex shapes which sits entirely in 3D Clifford Algebra. With Fengbo, we combine the descriptive power of Neural Operators with the inductive bias and interpretability of networks in Clifford Algebra to obtain a compact pipeline that is able to estimate multiple physical quantities both accurately and at once, without extra computational overhead.
We reported results on the two 3D CFD datasets available, ShapeNer Car and Ahmed Body, yielding test error lower than most previously reported NO models. We are able do with model with about 60% fewer parameters and with reduced computational complexity compared to graph- and transformer--aat based models. Moreover, we also estimate jointly the velocity field, leveraging exclusively geometrical information of Q.p. Fengbo is thus lightweight, expressive and accurate pipeline entirely 3D Euclidean space.
Limitations. Due to limited computational resources, we could not test deeper, larger versions of our proposed architecture, which caps at million parameters. Fengho might struggle when applied on less informative datasets (c.g. fewer physical variables, lack of geometrical information) that do not allow for the construction of full-grade multivectors. Besides, most implementations of Clifford Algebra networks rely tensor representation of multivectors, negatively impacting the model training speed.
Future work. Future work might include testing larger Fengbo multiple datasets, projecting it down to 2D for or extending its use for different PDEs to estimate jointly multiple physical quantities defined over complex, irregular domains.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Ali Al-Aradi, Adolfo Correia, Gabriel Jardim, Danilo de Freitas Naiff, and Yuri Saporito. Extensions of the deep Galerkin method. Applied Mathematics and Computation, 430:127287, 2022.
Kamyar Azizzadenesheli, Nikola Kovachki, Zongyi Li, Miguel Liu-Schiafiini, Jean Kossaifi, and Anima Anandkumar. Neural operators for accelerating scientific simulations and design. Nature Reviews Physics, pp. 1-9, 2024.
Shuvayan Brahmachary and Nils Thuerey. Unsteady cylinder wakes from arbitrary bodies with differentiable physics-assisted neural network. Physical Review E, 109(5)005534,, 2024.
Johannes Brandstetter. Rianne van den Berg, Max Welling, and Jayesh K Gupta. Clifford neural layers for PDE modeling. arXiv preprint arXiv:2209.049.0 2022.
Shuhao Cao. Choose transformer: Fourier galerkin. Advances neural information processing systems, 34:24924-24940, 2021.
Giuseppe Carleo, Ignacio Cirac, Kyle Cranmer, Laurent Daudet, Maria Schuld, Naftali Tishby, Leslie Vogt-Maranto, and Lenka Zdeborová. Machine learning and the physical sciences. Re- views of Modern Physics, 91(4):045002, 2019.
Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piecialli. Scientific machine learning through physics- informed neural networks: Where we are and what's next. Journal of Scientific Computing, 92(3):88, 2022.
Jingyang Deng, Xingjian Li, Haoyi Xiong, Xiaoguang Hu, and Jinwen Ma. Geometry-Guide -uided Conditional Adaption for Surrogate Models of Large-Scale 3D PDEs on Arbitrary Geometries. 2024.
Chris Doran and Anthony Lasenby. Geometric algebra for physicists. Cambridge University Press, 2003.
Lco Dorst and Joan Lasenby. Guide to geometric algebra in practice. Springer Science & Business Media, 2011.
Michael Friswell and John Mottershead. Finite element modelling. Springer, 1995.
Hongyang Gao and Shuiwang Ji. Graph u-nets. In international conference machine learning. pp. 2083-2092. PMIR, 2019.
JingWei Gao, DongHui Wang, et al. Fast Reconstruction Method of 3d Flow Field Based on Enthaned--Enhaneed Pointnet+++. 2024.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying. Yinpeng Dong, Songming Liu,
Ze Cheng, Jian Song, and Jun Zhu. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, pp. 12556-12569 PMLR, 2023.
Eckhard Hitzer. Introduction Clifford's geometric algebra. Journal of the Society of Instrument and Control Engineers, 51 51(4):338 (1500033-500, 2012.
Christian Hockey, Yuxin Yao, and Joan Lasenby. Simplifying and Generalising Equivariant Geomet- ric Algebra Networks. The 9th conference on Applied Geometric Algebras in Computer Science and Engineering (AGACSE), 2024.
George Em Karniadakis, loannis Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang- Physics-informed machine learning. Nature Reviews Physics, 440, 2021.
Ali Kashefi. Kolmogorov-Amol PointNet: Deep learning for prediction of fluid fields on irregular geometries. arXiv preprint w:Xiv:240800295. 2024.

---

Published as a conference paper at ICLR 2025
Ali Kashefi, Davis Rempe, and Leonidas J Guibas. A point-cloud deep learning framework for prediction of fluid flow fields on irregular geometries. Physics of Fluids, 33(2), 2021.
Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Arizzadenesheli, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to PDEs. Journal of Machine Learning Research, 24(89):1-97, 2023..
Anthony Lasenby and Chris Doran. Physical applications of geometric algebra. Handout collection from a Cambridge University lecture course, 2001.
Anthony Lasenby and Joan Lasenby. Applications of geometrie algebra in physics and links with engineering. In Geometric Algebra with Applications in Science and Engineering, pp. 430-457. Springer, 2001.
Yann LeCun, Yoshua Bengio, et al. Convolutional networks for images, speech, and time series.
The handbook of brain theory and neural networks, (10) 1995.
Jian Li, Jing Yue, Wen Zhang, and Wansuo Duan. The deep learning galerkin method for the general stokes equations. Journal of Scientific Computing. 93(1):5, 2022a.
Zijie Li, Kazem Meidani, and Amir Barati Farimani. Transformer for partial differential equations' operator learning. arXiv preprint arXiv:2205. 2022b.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprins arXiv:2010.08895, 2020a.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differ- ential equations. arXiv preprint arXiv:2003.03453 2020b.
Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural oper- ator with learned deformations for PDEs on general geometries. Journal of Machine Learning Research, 24(388):1 2023.
Zongyi Li, Nikola Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Otta, Mohammad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadenesheli, et al. Geometry- informed neural operator for large-scale 3d PDEs. Advances in Neural Information Processing Systems, 36, 2024.
Phillip Lippe. Bas Veeling, Paris Perdikaris, Richard Turner, and Johannes Brandstetter. PDE- refiner: Achieving accurate long rollouts with neural PDE solvers. Advances in Neural Informa- tion Processing Systems, 36, 2024.
Tadeusz Liszka and Janusz Orkisz. The finite difference method at arbitrary irregular grids and its application in applied mechanies. Computers & Structures, 1((1-2):33-95, 1980.
Björ Lütjens, Catherine H Crawford, Campbell D Watson, Christopher Hill, and Dava Newman. Multiscale neural operator: Learning fast and grid-independent PDE solvers. arXiv preprint arXiv:2207.11417, 2022.
Farhad Nemati Taher and Abdussamet Subaşı. A fast three-dimensional flow field prediction around bluff bodies using deep learning. Physies of Fluids, 36(2), 2024.
Alberto Pepe, Sven Buchholz, and Joan Lasenby. Clifford Group Equivariant Neural Network Lay- ers for Protein Structure Prediction. In Northern Lights Deep Learning Conference, pp. 205-211. PMLR, 2024a.
Alberto Pepe, Sven Buchholz, and Joan Lasenby. GA-Rel.U: an activation function for Geo- metric Algebra Networks applied to 2D Navier-Stokes PDEs. In ICLR 2024 Workshop A14DiiffrentialEquatins Science, 2024b.
Alberto Pepe, Sven Buchholz, and Joan Lasenby. STAResNet: a Network in Spacetime Algebra to solve Maxwell's PDEs. arXiv preprint arXiv:2408. 13619, 2024c.
12

---

Published as a conference paper at ICLR 2025
Alberto Pepe, Joan Lasenby, and Sven Buchholz. CGAPoseNet+ GCAN: A Geometric Clifford
Algebra Network for Geometry-awan aware Camera Pose Regression. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 6593-6603, 2024d.
Nicholas Perrone and Robert Kao. A general finite difference method for arbitrary meshes. Com-
puters & Struciures, 5(1):45-57, 1975.
Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W Battaglia. Learning mesh- based simulation with graph networks. arXiv preprint arXiv:2010.03400 2020.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and patlern recognition, pp. 652-660, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas Guibas. Pointnet++: Deep hierarchical fea- ture learning point sets in a metric space. Advances in neural information processing systems, 30, 2017b.
Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Machine learning of linear differential equations using Gaussian processes. Journal of Computational Physics, 348:683 693,,, 2017.
Maziar Raissi, Paris Perdikaris, and George Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Bogdan Raonic, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bartolucci. Rima Alai- fari, Siddhartha Mishra, and Emmanuel de Bézenac. Convolutional neural operators for robust and accurate learning of PDEs. Advances in Neural Information Processing Systems, 36, 2024.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and computer-assisted intervention- MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings. part III 18, pp. 234-241, Springer, 2015.
Sujit Roy, Rajat Shinde, Christopher Phillips, Ankur Kumar, Wei Ji Leong, Manil Maskey, and Rahul Ramachandran. Clifford Neural Operators on Atmospheric Data Influenced Partial Differ- ential Equations. In 12th International Conference on Learning Representations, 2024.
David Ruhe, Jayesh K Gupta, Steven De Keninck, Max Welling, and Johannes Brandstetter. Geo- metric clifford algebra networks. In International Conference on Machine Learning, pp. 29306- 29337. PMLR, 2023.
David Ruhe, Johannes Brandstetter, and Patrick Foréé. Clifford group equivariant neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini.
The graph neural network model. IEEE transactions on neural networks, 20(1):61-80, 2008
Justin Sirignano and Konstantinos Spiliopoulos, DGM: A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375:1339-1364 2018.
Kiwon Um, Robert Brand, Yun Raymond Fei, Philipp Holl, and Nils Thuerey, solver-in-the-loww Learning from differentiable physics interact with iterative PDE-solvers. Advances in Neural Information Processing Systems, 33:6111-6133 2020.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Jared Willard, Xiaowei Jia, Shaoming Xu, Michael Steinbach, and Vipin Kumar. Integrating bassid modeling with machine learning: A survey. arXiv preprint UrXiv:2003.04919, 1(1):1-34, 2020.
Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: fast transformer solver for pdes on general geometries. arXiv preprint arXiv:2402.022 02366. 2024.
13

---

Published as a conference paper at ICLR 2025
Zipeng Xiao, Zhongkai Hao, Bokai Lin, Zhijie Deng, and Hang Su. Improved operator learning by orthogonal attention. arXiv preprint arXiv:2310. 1ee::2280.1487,,, 2023.
Xinyao Zhang and Junzhe Cao. Novel Point Cloud Deep Learning Method for Predicting Variable Airfoil Flow Fields. In 2024 43rd Chinese Control Conference (CCC), pp. 8381-8386, IEEE, 2024.
A CLIFFORD ALGEBRA FUNDAMENTALS
A.1 DEFINING A SUBALGEBRA
Clifford algebra of size can be defined over a scalar field and a set set independent basis vectors (e..i....... We indicate generic closed subalgebra with G(r, r, or alternatively G'(p. r), with n = p + + r. A closed subalgebra G(p, q: r) has p basis vectors that square to 1, g basis vectors that square -1 and basis vectors that square to O.
A.2 THE GEOMETRIC PRODUCT
Elements a GA are called multivectors. Elements of any type can added or multiplied to- gether. Each element has grade associated with it. By grade define the dimension of the hyperplane an object specifies. e.g. scalars are grade O, vectors are grade 1, bivectors are grade 2, etc. Clifford algebra is also known Geometric algebra because of the geometric product. The geometric product between two vectors is given by
(13) in which the scalar (or inner) product a b is the usual scalar product of linear algebra equal to the cosine of the angle between a and while the wedge (or outer) product ^ produces bivector (e.g. an oriented plane). The geometric product between vectors is hence the sum of scalar and bivector, that have different grade. Any multivector of a unique grade that can be defined as A A A... A is called a blade.
ab ba-b+aAb
The reversion operator for multivector is given by Å. The reverse of a scalar is equal to the scalar itself and the reverse of a vector is equal to the vector itself. For a multivector, we have that
(AB) BA
(14)
(A + B) Á + B
The general rule to reverse a r-blade is given by
A (-1) =tr-1) A,
(15)
The geometric product between multivector and its reverse gives the squared magnitude: |A|² (AA)o. The reversion operator can be used to define the inverse of multivector as
A-1
(16)
AA
can be easily shown that A-1A when AA is scalar.
The grade projector operator is denoted by (A)r, where r is the grade want to extract from A. This comes from the fact that a multivector in an n dimensional algebra can be written as
A =
(A)
(17)
0=0
The dual of multivector is defined as
A1
(18)
where I, is called the pseuloscalar, defined as In e1 A e2 A... 6x- The pseudoscalar is the highest grade element a GA. The product grade-n pscudoscalar In and grade-r multivector A, is grade-(n r) multivector, and is termed the duality transformation. The pseudoscalar interchanges inner and outer products:
A, (B. (A, B, (a))r-(---0 (x(-)) = (A.BgTa)n-(4aa (A,Ba) A B. (19)
14

---

tb, where Published as a conference paper at ICLR 2025
A.3 CLIFFORD ALGEBRA OF THE PLANE AND SPACE
The Clifford algebra of the Euclidean plane is denoted by G,2,0,0), with two basis vectors 61, C2- Being
dimensional GA (since p + +
0 +0
n), it is spanned by 2
elements, namely a scalar, two vectors e and the bivector eje2 = -1 62. G 0, 0) includes the concept of complex numbers, since the pseudoscalar of this algebra squares to -I, since 1 = ei 622612 = (e16)) = ((1
e1 62 e1e2 112
= -1. A scalar plus
bivector can be seen as representation of complex number, since Z
Ib
is the imaginary unit. Similarly, if X = ac, + be2 = e₁ (a bl) e,
Adding a third basis vector E3 we form G(3, 0, 0). the GA of Euclidean space, which is what employed in this paper. It has 2 8 elements, scalar, three vectors, three bivectors (612, C23, 61s) and one trivector (6123 e1 A 62 A 63 Iz, the pseudoscalar). The GA of space includes quaternion algebra, since quaternion q al + bj + ck can be represented as a multivector w. a212 + be13 + C623.
B IMPLEMENTATION DETAILS
Details of the Fengbo architecture are shown in Fig. 8 The 3D Clifford Geometry block (Fig, Sap consists of 3D convolutional layers with kernel size 5 5 x 5. The first 2 convolutions are followed by a group normalisation layer and a GeLU activation function. The block yields multivectors with progressively increasing number of channels Cg f1, 2,4)
Cirn
(a) 3D Clifford Geometry block
(b) 3D Clifford Physics block
ING LOLL
PAO
(c) 3D Clifford FNO block
Figure The three components of the Fengbo architecture.
An example of the geometric interpretability of the intermediate outputs of Fengbo is given in Fig. Multivector P. containing a scalar, vector and bivector component, is processed by the 3D Clifford Geometry block obtain the mucliveel-mulliveetoo Q. We employ grayscale colormap for scalar quantities, i.e. scalar and trivector components, and jet and rainbow colormaps for the vector and bivector components, respectively. The input shape multivector /' built from the dataset is scattered within the [- volume, bounded by the tanh activation function. Elements of different grades are mixed, as can be noticed from the appearance of trivector components. The sequence of convolutions makes the 3D multivectors progressively denser. The last multivector Qp, with 4-channels, unbounded due the lack of an activation function and fed into the 3D Clifford FNO. Each channel shows how different grade elements in the volume cluster to form different shapes, more or less aligned a certain direction. While from the original car shape, these blobs indeed represent scalar, vectors, bivectors and trivectors in 3D space: the vector components shown, for example, cannot be interpreted as anything else than coordinates 3D point clouds precisely because of our choice of embedding. Similar considerations
be made for V and Qv.
Fig shows the 3D Clifford Physics block. It contains the same layers as the 3D Clifford Geometry block, but with decreasing number of channels Cp {2, 1, 1}. ), shown in Fig. 7 and different meaning attached to the multivector representation, where the scalar and vector part represent the pressure field d(x) and the velocity field (x).
15

---

Published as a conference paper at ICLR 2025
sivector bivector vector scalar
Figure 9: Intermediate layers outputs from P to Qp within the 3D Clifford Geometry block for test sample in the ShapeNet Car dataset.
Fig shows the 3D Clifford FNO block. It includes F 2 residual blocks, 4 input channels, 4 output channels, C hidden channels and m = 8 Fourier modes for each of its 3 dimensions. Each layer is followed by group normalisation layer and GeLU activation function.
Pipeline Generalisability. Fengbo operates entirely within Clifford Algebra, which is naturally suited for multidimensional treatment (reatmen[[aasenyy & Lasenby 2000); Lasenby & Doran (2001): Do- ran & Lasenby] (2003); Hitzer (2012), and extensions of a same architecture in Clifford Algebra to higher dimensions has been widely documented in the literature one of the main advantages of these types of networks Brandstetter et al. (2022): Ruhe et al. 2024 2023); |Pepe et al. (2024c). The implementation of 2D equivalent our 3D Fengbo pipeline straightforward,
detailed in Table 5] By projecting the convolutional operations from 3D to 2D and operating within either G(2, 0) G(O, 2, 0), achieve fully analogous approach for 2D problems with targets that include scalar fields or 2D vector fields. Besides, just like Fengbo 3D is not limited to the task of 3D flow estimation, Fengbo 2D is not limited to 2D flows, but it can be extended to any 2D PDE that establishes a mapping from the geometry to the physics of the problem.
Table 5: Comparison of Fengbo's s 3D and 2D configurations.
Aples
Iour Stge
Ureeem Rea
Tirans
the
Vocier loks
We can consider the 2D Fengbo as a simpler subcase of the 3D case, since its 3D implementation presents several more challenges, namely:
6-dimensional tensors in 3D, with shape X D, where B is the batch size, the number of channels, M is the grid resolution and D is the algebra dimensionality opposed to 5-dimensional tensors in 2D, with shape B x C x M x M x D, which require significantly less memory and allow for larger model sizes.
Larger algebra dimensionality, D 2 8 elements in G(3, 0, 0), namely scalar, trivector, 3 vectors and bivectors, as opposed to the 2D case with D = 4, with scalar, 1 bivector and vectors. This has implications in the sparsity the input tensors and in their memory requirements, which negatively impact convergence.
Significantly higher computational complexity. since it stays the same for both cases, namely O log N), but with N 3 for the 3D case and N M2 for the 2D case. Moving to 2D would allow for a larger discretisation that can preserve a higher level of detail at a fraction of the computational cost.
Moreover, handling 3D datasets with the proposed pipeline inherently includes the capability to process 2D datasets oi et al. (2020a): Wu et al. (2024). This is shown in Table( When processing instances in ShapeNet Car and Ahmed Body,
1) sample points from the unstructured meshes; 2) discretise the irregular point clouds onto regular grids and 3) embed them in multivector form. Datasets like AirFRANS, which also contains unstructured meshes, would be processed in the same way. All the remaining 6 datasets fall into data structures which are intermediate steps of the pipeline established with Fengbo: point clouds, like the Elasticity dataset, can be directly discretised onto
16

---

Published as a conference paper at ICLR 2025
regular grids and embedded into multivector form, regular grids of the Plasticity, Airfoil and Pipe datastes can be simply embedded as multivectors, as already demonstrated in Brandstetter et al. (2022) ; Pepe et al. (2024b), and structured meshes of the Navier-Stokes and Darcy datasets can be processed like their unstructured counterparts, demonstrating Fengbo's o generalisability to 2D cases.
Table 6: Data representations and their processing steps with the Fengbo pipeline.
Unstructare Metles
Polut Elasticty (DD)
Grld
Structured Narter Stokes, Darcy (2D)
ShapeNer Car, Aboee Body (DD) (DD)
Plasniciy, Airfol. Piee (DD
Sample points
Discretise onto regular grid 3. Embed in multivector form
C TRAINING DETAILS
Fengho was trained on 3 NVIDIA A100 GPUs with 40GB RAM. It was trained for 100 epochs with batch size of 3, for total of approximately 24 compute hours. We employed the Adam optimiser to update the model's weights with default parameters of 31 = 0.9, 32 0.999. We adopted a learning rate of 10 reduced on plateau by factor of 2 with patience on the validation loss set to 8 epochs. The loss we minimised for the ShapeNet Car dataset is
== aLp BLv + 11 - ر||
(20)
with parameters a 5, 5=
= picked empirically to weight the pressure component more, since was harder regress. A similar loss was employed for the Ahmed Body dataset:
== aLp + B||r - f1l1 + 1| 611
(21)
but with a = 1,3 = and r being the Reynolds number embedded as the trivector component of the output, i.e. (R')s (mpr)e123, where R' refers the second output of the network, P' being the first. No velocity information is provided. The L1 norm term on pressure was added to further penalise large deviations of á, the estimated pressure field with respect to ground truth.
We employed elastic net regularisation, with \ 10-5 lasso regularisation coefficient and X2 = 10-4 ridge regularisation coefficient, to encourage group selection of correlated features and reduce overfitting, which we found to be significant over such small training sets. For training, input geometries are normalised in the range [-1, 1J. vector velocity fields are normalised in the range [0, 1] and pressure fields are unit normalised by subtracting their mean and dividing them by their variance. Test metrics are measured over denormalised quantities yield physically meaningful errors.
D ABLATION STUDY
Table Ablation on the impact of M.
ShapeNet Car
Ahmed Body
Grid Size M
Pressure
Velocity
Pressure
training
testing
training
testing
training
testing
40
6.69
12.8
5.58
6.11
10.6
15.1
50
7.38
11.2
5.99
5.70
10.4
13.9
60
6.42
10.1
5.17
5.26
9.17
12.3
70
7.43
10.5
5.97
5.61
8.10
11.7
80
6.94
8.86
5.56
5.10
8.00
10.7
Pipeline Scalability. We study the impact of four components of the Fengbo pipeline, namely the grid size M. the number of hidden channels in the 3D Clifford FNO module C,, the number of blocks in the FNO F and the number of modes in the FNO m.
17

---

Published as a conference paper at ICLR 2025
The impact of grid size Fengbo's accuracy is shown in Table We train and test volumes with the same resolution. It can be noted that, by reducing the number of voxels by 87.5% (from M 80 to M 40), i.c. significantly reducing the level of details in imput shape, Fengbo still yields a test error only 4% higher for the ShapeNel Car dataset and 4.5% higher for the Ahmed Body dataset. We hence claim discretisation convergence.
Since the 3D Clifford FNO is Fengbo's s key component, we study the impact of its parameters. In Fig. T0 report the ablations over the number channels C within the FNO for the datasets and for the relative L2 norm over pressure and velocity fields o, sh, respectively. For the velocity field of ShapeNet Car we tested two combinations of the weighting coefficients of the loss function a, 9. The grid size is fixed to M = 80 and the number of blocks within the FNO is fixed to F = 2, while (5, 10, 15, 20, 25).
we test
Rolstive L2 norm (ShapoNet Car)
10.0
Relative La nom [Ahmed
9.0
11.6
11.2
9.2
11.0
9.0
10.8
8.8
7.5
Hidden
Channels
(a) Relative Le norm on pressure, ShapeNet Car
(b) Relative Lg norm on pressure, Ahmed Body
dataset versus C.
dataset versus C.
Relative 1, nemm (Shapetee Can
Relative L, norm (Shapetiet Car)
6.0
5.5
5.8
5.0
5.4
4.5
5.2
5.0
3.5
5.0
7.5
10.0
12.5
15.0
17.5
20.0
22.5
25.0
5.0
7.5
10.0
12.5
15.0
17.5
20.0
22.5 25.0
# Hidden Channels
Hidden Channels
(c) Relative La norm velocity, ShapeNer Car dataset versus C. (a, 3) 15
(d) Relative La norm velacity, ShapeNet Car dataset versus (), (1,50)
Figure 10: Ablation the number of hidden channels C.
In all four presented scenarios, a higher number of hidden channels (i.e. a wider network) yields steep decrease in the eπrоr at testing stage, proving how larger version of Fengbo to that presented the main body of this manuscript (C 25) could further improve the quality of the PDE solution and demonstrating its scalability with respect to C.
In Fig. II we study the impact of the number of FNO blocks for the same four cases above. We tested 1, 2, by keeping C = 15 and M = 80. Also in this scenario, deeper network corresponds to more accurate estimation and hence scalability with respect to F. Note, in Fig. 11a lower absolute minimum for relative La norm over for the ShapeNet Car dataset of 8.25% with F = 4.
We then studied the impact of the number of Fourier modes of the FNO. We tested {3, 6,8, 10, 12, 14) by keeping F = and = 20. this case results are less uniform across the four test cases, but can conclude thar larger number of modes often corresponds similar not worse performances, as already pointed out in BBrandttetter et al. (2022).
The effect that these ablations have the number of models parameters and size reported in Fig. [3] Note how the ablations on M are missing since they do not affect the model dimension. The model size scales exponentially with respect C,m, and linearly with respect to F. Fourier
18

---

Published as a conference paper at ICLR 2025
[Shapefiet Can)
11.5
9.6
Relative ia norm - (Ahmed
9.2
11.3
Relative L porm
8.8
8.6
11.2
8.2
11.1
1.0
1.5
2.0
2.5
3.0
3.5
1.0
3.0 Number INO blocks F
2.0
2.5
3.5
4,0
Number INO blocks F
(a) Relative L2 norm on pressure, ShapeNer Car dataset versus F.
(b)
Relative L2 norm pressure, Almed Body
dataset versus F. {a, 3) (5,1).
Relativ nomm 11hapeeett Can)
Relative,, L, narm. iShapehet Car)
4.4
5.0
1.0
1.5
2.0
2.5
3.0
3.5
2.0
3.0
3.5
4.0
Number FNO blocks -F
Number of FNO blocks
(c) Relative La norm on velocity, ShapeNer Car dataset versus F.
Relative L2 norm on pressure, Aluned Body (d)
dataset versus F. {a, 8} (1,50).
Figure 11: Ablation the number of FNO blocks F.
Relative L3 norm [Shapelte: Can
11.4
11.3
9.7
11.2
11.1
9.5
La nomm
11.0
10.9
9.3
10.8
10.7
12
14
Fourler
(a) Relative Le norm on pressure, ShapeNer Car dataset versus
(b) Relative L2 norm on pressure, Ahmed Body dataset versus
Relatiell L, noo ShapeNet Carl
Relatiee,, L, nome IShapeMet Carl
4.10
5.15
8.05
5.10
5.05
3.95
3.90
3.85
4.90
10 Number Fourier modes
12
10
12
14
Number Fourier modes
(c) Relative La norm on velocity, ShapeNet Car dataset versus m. {a, (G 1}.
(d) Relative L2 norm on pressure, Ahmed Body dataset versus F. (a, (1,50).
Figure 12: Ablation the number of Fourier modes
19

---

Published as a conference paper at ICLR 2025
modes have the biggest impact the model parameters, with 14 corresponding to x13 increase with respect to the Fengbo presented the main body of the text, without benefiting the test emrr. The number of blocks F, on the other hand, corrssonnds to relatively milder increase in the model size while still providing a substantial improvement in performance. This proves our claim in the Limitation section, i.e. that deeper network can likely correspond more robust and accurate predictions than those shown in Table 2
muniraa 201
109
370
205
115
113 80 41880n Curabio
il bo00-1
#uunce fcate nocern
(a) Ablation on C.
(b) Ablation on F.
(c) Ablation
Figure 13: Number parameters (M) and model size (MB) as a function of C, R, m.
Lastly, the impact of the weighting coefficients of the loss function Fengbo's 's accuracy is shown in T We fix the resolution M 80 and vary the weight attributed different physical quantities in the loss function. 3 weights velocity for the ShapeNer Car dataset and the Reynolds number for the Ahmed Body dataset, while a weights number does not contribute in significant way to the estimation of pressure in the Ahmed Body dataset. From Table X we can conclude that the high accuracy accuracy of Fengbo stems also due to the joint estimation of variables, e.g regressing also v can better constraint the values of é can assume and vice versa. a, 3 can be thought as two parameters that mix quantities to be regressed and that be tuned based the specific requirements of problem to be tackled, e.g. which quantity we wish to prioritise.
pressure in both. Note how Reynolds
Table 8: Ablation the impact of a,
ShapeNet Car
Ahmed Body Pressure
a
3
Pressure
Velocity training
training
testing
testing
training
testing
1
0
8.53
9.21
8.60
11.8
1
1
9.07
9.32
7.23
4.39
8.00
10.7
2
1
8.03
9.30
6.40
4.56
8.23
10.9
5
6.94
8.86
5.56
5.10
7.64
10.9
10
1
5.38
9.12
4.28
5.48
9.31
11.9
0
1
4.90
4.03
I
2
9.37
9,50
4.09
4.12
8.42
11.9
1
5
7.71
9.83
3.98
3.82
8.34
11.8
1
10
9.37
10.1
3.82
3.60
8.26
11.8
1
20
10.5
10.8
3.37
3.59
1
50
8.42
11.4
3.23
3.47
E NOTES ON COMPUTATIONAL COMPLEXITY
As outlined in Section 2, we focused primarily Neural Operators over Transformers due to the latter's significantly higher computational complexity, namely O(N2). Transformers, albeit offer- ing improved performances, introduce substantial challenges in terms of resource requirements and scalability, As proof of that, we offer an analysis the theorical complexity of the Fengbo model and compare it with the current state-of-the-att in Transformer-based solvers, the Transolver archi- tecture Wu et al. (2024), which to the best of our knowledge is both the most accurate and the least computationally expensive Transformer architecture designed to solve PDEs. Albeit mostly vali- dated over 2D problems, the Transolver has also been tested over one 3D dataset, namely ShapeNet Car.
20

---

Published as a conference paper at ICLR 2025
More specifically, authors of Wu et al. (2024) designed an ad hoc attention layer, the Physics- Attention layer, that operates on slices of the PDE domain Zg- The reported computational com- plexity of such layer O + S2C', in which N is the number of meshes, S is the number of slices into which the domain is partitioned and C' is the number of hidden channels of the model. The authors claim quasi-linear complexity with respect to N. However, the overall complexity is heavily dependent on the choice of the model parameters S and C, and for large values of S and C, which is the setting for most of the experiments in Wu et al. (2024). it becomes sub-quadratic. We compare the model complexities for the 3D and the 2D case.
3D Case. Model complexities for the 3D case are shown in Fig. 14 For the ShapeNer Car, specif- ically, the reported parameters are as follows: N = 32000 meshes, S = 32 slices into which the car surface partitioned and = 256 channels. This places the computational complexity of the Transolver architecture for the ShapeNer Car dataset at the green marker shown in Fig. 14a
On the other hand, as shown in T we report a computational complexity of O(N log N) for the Fengbo pipeline. In our approach, N is the dimension of the 3D regular grid, hence N = M with M being the grid resolution. This places the computational complexity of the Fengbo architecture for the ShapeNet Car dataset at the yellow marker shown in Fig. 14a
Such design choices yielded results shown in Fig. 14b the Transolver model attains a 1.4% de- crease relative Le error over Fengbo, but at computational complexity of two orders of magnitude larger. It is also worth noting that the experiment setting for Transolver followed the implementa- tion of 3D-GeoCA Deng al., (2024), which takes 789 samples for training and 100 samples for testing. On the other hand, we followed the approach of GINO et al. (2024), in which we retain 611 watertight meshes and employ 500 samples for training and 100 for testing, meaning that the Transolver, besides being computationally more complex, was also trained on 57. 7.. more samples.
pict ollg N and NSC + various values
10F
Fengbo
Transolver
Compl.
O
O
10F
L2 (Ø)
8.86
7.45
000
La (1) 3.47
2.07
10F
(b) Comparison of Fengbo and Transolver in terms of computational complexity (yellow and green marker in the figure, respectively) and test Le norm for the pressure and locity field over the ShapeNer Car dataset. Pressure L2 obtained with fa, B} 15, 1/. Velocity L2 obtained with fa, 3 = {1, 505
10
(a) Computational complexities of Fengbo (N log N) and of Transolver (NSC C). for different values of S. Fengbo operates at N1, while the Transolver operates N₂ in its 3D setting. Linear and quadratic complexity curves added for ref- erence.
Figure 14: Computational complexity comparison of Fengbo and Transolver Wu et al. (222)) for the 3D case.
A similar claim can be made for the Ahmed Body dataset, not analysed in Wu et al. 2024 but whose complexity can still be studied. Assuming one point per mesh, i.e. N = 100000, and the same parameters configuration employed for the ShapeNer Car dataset, i.e. (S.C) the resulting complexity of the model also reaches O(10)), corresponding to to the pink marker in Fig. [4a] With Fengbo, the grid resolution is kept unchanged for the Ahmed Body dataset, which yields identical complexity to the ShapeNet Car dataset, demonstrating its robustness to larger mesh size.
{32, 256),
2D Case. We compare the ablations presented in Appendix A of this manuscript with those pre- sented in Appendix C of Wu et al. (2024). We compute the corresponding Fengbo complexity as the grid size varies, i... M {40, 50, 60, 70, 80} and the corresponding Transolver complexity as N and C vary across datasets and as the number of slices S vary employed across ablations, namely S (1, 8, 16, 32, 64, 96, 128, 256, 512, 1024). This is summarised in Table 9 We then plot the
21

---

Published as a conference paper at ICLR 2025
Computational Complexity versus promotion for different datasets
Tremiie Canicyy-
Frx. Traassee
2x 10
S
Vol Enity Fangto
Promotion
10"
M
10
107
10
Computational Complexity
Figure 15: relative L2 norm promotion versus computational complexity for Fengbo and Tran- solver's ablations.
relative L2 norm promotion versus the resulting complexities for six 2D datasets (Darcy, Elasticity, Plasticity, Airfoil, Navier-Stokes and Pipes) and two 3D datasets (ShapeNet Car and Ahmed Body). The promotion defined as P, = Li/C*, with L, the relative L2 norm reported for the ith ablation and L* the overall minimum L2 norm reported. We do different datasets might present very different ranges of L.
This is shown in Fig. 15 in which the markers represent different ablations and the curves the resulting interpolations. Note how, even when compared to 2D datasets, Fengbo still operates at one order of magnitude below Transolver for the Elasticity dataset and two orders of magnitude for the remaining five datasets. On top of that, the computational cost to lower L to the optimal value is also significantly lower for the Fengbo pipeline. Additionally, if Fengbo were to be tested in 2D scenario, the value of N would likely be much smaller than 106.
Table 9: Complexity comparison for different datasets
Darcy
Elashcity
Plasticity
AirFoil
Navier
Pipes
Shape/Net Car, Ahmed Body (Fengbo)
7225
976
3131
11271
16641
4096
5
64
64
64
64
64
32
128
128
128
128
128
256
Complexity
O
O(10²)
O(0*)
O(10*)
O(10*)
O(10*)
O
22


Reviews:
[{"review_id": "lLqYdK6tkn", "reviewer": "Reviewer_gPtN", "summary": "The paper introduces Fengbo, a novel computational pipeline designed to solve 3D partial differential equations (PDEs) specifically for applications in computational fluid dynamics (CFD). Utilizing Clifford Algebra, Fengbo employs an architecture that consists solely of 3D convolutional and Fourier Neural Operator (FNO) layers, effectively modeling the PDE solution process as a clear mapping from geometric representations to the underlying physics of the problem. Despite its relatively simple architecture with only 42 million trainable parameters, Fengbo demonstrates competitive accuracy, outperforming five out of six models previously proposed in the literature for the same dataset. The architecture achieves this with reduced computational complexity compared to graph-based methods while estimating both pressure and velocity fields. A notable feature of Fengbo is its transparency; the output of each layer can be visualized as objects and physical quantities in 3D space, thereby classifying it as a \"whitebox\" model. By integrating geometry with physics, Fengbo offers an efficient, interpretable, and physics-aware solution for addressing complex PDEs in CFD applications.", "questions": "1. Please compare with more recent developed methods\n2. Please conducts more experiments on different datasets or PDEs.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper presents a novel approach to solving 3D partial differential equations (PDEs) in computational fluid dynamics (CFD) through the use of Clifford Algebra. The use of a pipeline that incorporates only 3D convolutional and Fourier Neural Operator (FNO) layers within a Clifford Algebra framework is a creative combination of mathematical structures and neural network architectures. This uniqueness not only offers a fresh perspective but also has the potential to influence future research in both machine learning and PDE solving. The ability to visualize outputs as physical quantities in 3D space transforms the model into a \"whitebox\" system. This enhances the interpretability of complex models, which is increasingly important in scientific computing, as it allows for better understanding and trust in the results produced by neural networks.", "weaknesses": "1. Lack of Comprehensive Benchmarking\n2. Absence of Generalization and Scalability Discussion\n3. Potential Overlook of Limitations", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SRVuuEwkhL", "reviewer": "Reviewer_TsLb", "summary": "This work is essentially the combination of \"Clifford Neural Layers for PDE Modeling\":\n\nhttps://arxiv.org/abs/2209.04934\n\nand FNO:\n\nhttps://arxiv.org/abs/2010.08895\n\nwith the extension to computational fluid dynamics (CFD). 3D test cases are considered in this work, with the goal of prediction of the pressure and velocity fields. The complexity of the algorithm, error analysis, and visual comparison between the ground truth and prediction were conducted in this work.", "questions": "--> In Table 2, for FNO, the test error is lower than the train error, how is this possible?\n\n--> In Eqs. 20 and 21, the loss function is a combination of the relative $L^2$ norm with the absolute $L^1$ norm. Mathematically, it does not seem reasonable. How do you justify that?\n\n--> I had some concerns, listed in Weakness. Please address them. Thanks.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "High quality of writing and figures. Details explanations. A successful extension of Clifford Neural Layers for PDE Modeling to the Navier-Stokes equations for molding fluid dynamics.", "weaknesses": "--> The novelty is limited. This is simply just another application of the Clifford Neural Layers for PDE Modeling paper.\n\n--> In the literature review, the classes of PointNet and PointNet++ for deep learning of CFD have been missed. I suggest that the authors take a look and search on Google Scholar to find those articles and perhaps discuss them. Note that PointNet is suitable for unstructured grids and much lighter than graph neural networks since there is no connectivity between nodes.\n\n--> I disagree with the claim of this manuscript saying that their proposed method is appropriate for irregular grids compared to graph neural networks or PointNet because they still convert irregular grids to Cartesian grids and this definitely introduced errors no matter how much you \"carefully\" convert these data.\n\n--> Following my previous comment, I believe the information listed in Table 4 is misleading. FNO can be used for irregular geometries if one uses geometric transfer. See the following paper:\n\nhttps://www.jmlr.org/papers/v24/23-0064.html\n\nOn the other hand, the proposed method is not inherently designed for irregular geometries, similar to CNNs and FNOs.\n\n--> As a minor comment, it is better to write L2 as the $L^2$ norm", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g5Qm5yECEj", "reviewer": "Reviewer_E7Y6", "summary": "The paper presents a new deep learning pipeline to predict the velocity and pressure fields in 3D CFD simulations. The algorithm relies on the use of Clifford algebra layers, a mathematical construction which enables the processing of n-dimensional multivector fields. First, the input fields are upsampled and mixed to translate local to global features. Then, the global information is processed with a FNO-like frequency learning algorithm in a regular voxelized domain. Last, the processed information is decoded to the desired outputs: pressure and velocity fields. This pipeline is tested in the ShapeNet Car and Ahmed Body benchmark datasets and compared with other state-of-the-art architectures.", "questions": "* Line 245: Why is the bivector component left blank in the case of the velocity? The surface information is specially important for the velocity field as it determines the boundary layer dynamics, crucial for drag/lift analysis in aerodynamics. \n* The method's claim of being a white-box model may be overstated. While the final layers may provide some intuitive information during the decoding stage, the true learning of the underlying physics primarily occurs in the 3D Clifford FNO block, which still operates within a latent space. Furthermore, how practical is the interpretability of the final layers beyond offering intuition about the predictions? Have the authors observed any relevant phenomena, such as different energetic modes of the pressure and velocity field across each decoding stage? Without any analysis of this regard, the interpretability claim remains largely qualitative rather than quantitative and has little practical use.\n* Table 2: Why MeshGraphNets are not tested for the ShapeNet car dataset? The original paper predicts both the pressure and the momentum field of the fluid.\n* Line 388: \"[...] being the only architecture reported able to do so while jointly estimating the scalar pressure field and the 3D velocity field\". Any of the reported architecture can be trivially modified to include an additional output (the velocity field), so I don’t see this as a specific advantage of the proposed methodology.\n\nFinal comment: The presented paper proposes almost the same methodology as GINO (\"Geometry-Informed Neural Operator for Large-Scale 3D PDEs\" paper) but changing the GNO layers with Clifford algebra layers. From an accuracy and novelty perspective, the results offer only incremental improvements. While the interpretability of the final layers might provide some vague intuition, the paper fails to extract any substantial insights from an engineering or mathematical perspective. The only real novelty of the paper is the reduction in number of parameters, which in my view is insufficient for the standards of this venue. For these reasons, I'd rate this paper as marginally below the acceptance threshold.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "* The Clifford algebra architecture is an interesting inductive bias and is generalizable to more complex multidimensional fields. \n* The method has less trainable parameters compared to other techniques.\n* The last blocks of the algorithm can be interpreted as velocity and pressure fields, so one can have a visual intuition of how the network learns the final  prediction stage.", "weaknesses": "* The method is limited for steady-state simulations, and only tested for low density/viscosity fluids.\n* The performance of the method is very similar in error compared to existing techniques.\n* The voxelization of the space might be very inefficient with more complex geometries.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u6CsGPm3lT", "reviewer": "Reviewer_sKmQ", "summary": "The paper introduces Fengbo, a neural operator pipeline that uses Clifford Algebra to solve 3D PDEs in computational fluid dynamics. Fengbo leverages 3D convolutional and Fourier Neural Operator layers within a Clifford Algebra framework to map 3D geometries to physical fields, such as pressure and velocity. It demonstrates competitive accuracy on CFD datasets with fewer parameters and lower computational complexity, while offering interpretability.", "questions": "Please refer to weaknesses section for questions.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* The paper introduces a novel approach by embedding the entire architecture in Clifford Algebra, which allows for a unified treatment of geometric and physical data, enhancing model interpretability and preserving geometric relationships.\n\n* This model provides white-box interpretability by representing intermediate outputs as multivectors, which correlate with physical quantities in 3D space.", "weaknesses": "* No comparison with the most advanced deep learning-based methods (e.g. transolver, etc.).\n\n* The main results in the paper show better results with fewer parameters, but not the best performance, and it would be better if the performance could be compared with the same parameter Settings. \n\n* The paper was validated on a limited dataset, and it is hoped that it can be validated on more diverse datasets and tasks (e.g., point cloud, structured mesh, regular grid, etc.), which can be referred to transolver's experimental design.\n\nMinor comments:\n* L257\"... were the range of the summation of l,m,n isspecified by the kernel size and cin,cout are the ...\" has some grammar and typo issues in line 127.\n\n* The definitions of metrics in lines 346 to 363 (formulas (11),(12)) are inconsistent (groud truth and estimated results are used in the denominators respectively).\n\nI would consider revising my rating if the author is able to address my questions and effectively improve on the areas I identified as weaknesses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AJuGn1tKm9", "reviewer": "Reviewer_FKFg", "summary": "This submission targets the learning of 3D flow fields together with pressure distributions by using Clifford algebra. This approach has been proposed in previous work, and the submission at hand extends its implementation, and add geometry and physics blocks that seem to primarily aim for up- and down-sampling.\n\nAs the paper is largely extending previous work, it does not include and evaluate simpler cases, but directly focuses on 3D flows. Results are shown for flows around obstacle geometries from ShapeNet cars, and the Ahmet body. \n\nThe results are mixed: in some cases the proposed architecture seems to perform well, but is outperformed by previous work in others. Especially the classic Unet still seems to do a fairly good job, and probably has a much lower computational workload (and simpler implementation).\n\nOverall, I like the direction of the paper: 3D flows are definitely a challenging topic, and important for practical applications. At the same time, the proposed method does not seem overly convincing to me: it is very tailored towards 3D flows, and seems somewhat incremental given the previous work on Clifford based GNNs. With the results in the paper, I would be hesitant to try this approach, and correspondingly, I also find it difficult to really argue for accepting this paper to ICLR. I think with the current, somewhat narrow scope on 3D pressure (+velocity) the submission would be better suited for a more specialized conference or journal.", "questions": "What is the bottleneck that caps the model size at 42M parameters? This does not seem overly large for 3D problems.\n\nMinor, but why are the sizes of the other models in table 2 not listed? How many parameters did they have?\n\n(Very minor recommendation, it's a good idea to give intuition with figures like fig. 2 about the Clifford algebra setup, but figure 3, for example, did not add much information.)", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "I think the paper has the following strong points:\n- it targets non-trivial flow scenarios in three dimensions\n- the ShapeNet cars and especially the Ahmet body are interesting use cases\n- a nice range of baselines models is compared to\n- the underlying theory is complex", "weaknesses": "At the same time, the submission has weak spots:\n- the approach seems to be specialized to 3D flows, and I don't see how it would naturally extend to other problems\n- the gains in terms of accuracy seem to be mild, which is a pity given the complexity of the approach\n- the Clifford algebra comes from previous work, and I have to admit that I don't find it intuitive to work with\n- the properties of the baselines are not fully clear (e.g., parameter counts are missing)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)