Geometry Aware Operator Transformer as an efficient and accurate neural surrogate for PDEs on arbitrary domains
Authors: Shizheng Wen
Arsh Kumbhat
Levi Lingsch
Sepehr Mousavi
Yizhou Zhao
Praveen Chandrashekar
Siddhartha Mishra
Venue: NeurIPS 2025 poster ()
Keywords: Partial Differential Equations
Surrogates
AI for PDEs
Neural Operators
Transformers
Computational Efficiency
Scalability
Arbitrary Domains
Primary area: machine_learning_for_sciences
TL;DR: We propose an accurate and efficient neural operator architecture for learning PDE solutions on arbitrary domains. We demonstrate its effectiveness across a variety of challenging benchmarks, including large-scale 3D CFD problems.

Abstract: The very challenging task of learning solution operators of PDEs on arbitrary domains accurately and efficiently is of vital importance to engineering and industrial simulations. Despite the existence of many operator learning algorithms to approximate such PDEs, we find that accurate models are not necessarily computationally efficient and vice versa. We address this issue by proposing a geometry aware operator transformer (GAOT) for learning PDEs on arbitrary domains. GAOT combines novel multiscale attentional graph neural operator encoders and decoders, together with geometry embeddings and (vision) transformer processors to accurately map information about the domain and the inputs into a robust approximation of the PDE solution. Multiple innovations in the implementation of GAOT also ensure computational efficiency and scalability. We demonstrate this significant gain in both accuracy and efficiency of GAOT over several baselines on a large number of learning tasks from a diverse set of PDEs, including achieving state of the art performance on three large scale three-dimensional industrial CFD datasets. Our project page for accessing the source code is available at https://camlab-ethz.github.io/GAOT.

Geometry Aware Operator Transformer As An Efficient And Accurate Neural Surrogate For PDEs On Arbitrary Domains
Shizheng Wen'
Arsh Kumbhat
Levi Lingsch 1.2
Sepehr Mousavi L3
Yizhou Zhao 4
Praveen Chandrashekar
Siddhartha Mishra 1.2
Seminar for Applied Mathematics, ETH Zurich, Switzerland
ETH AI Center, Zurich, Switzerland
Department of Mechanical and Process Engineering, ETH Zurich, Switzerland
School of Computer Science, CMU, USA
Centre for Applicable Mathematics, TIFR, India
Abstract
The very challenging task of learning solution operators of PDEs on arbitrary do- mains accurately and efficiently is of vital importance to engineering and industrial simulations. Despite the existence of many operator learning algorithms to approx- imate such PDEs, we find that accurate models are not necessarily computationally efficient and vice versa. We address this issue by proposing geometry aware oper- ator transformer (GAOT) for learning PDEs on arbitrary domains. GAOT combines novel multiscale attentional graph neural operator encoders and decoders, together with geometry embeddings and (vision) transformer processors accurately map information about the domain and the inputs into robust approximation of the PDE solution. Multiple innovations in the implementation GAOT also ensure computational efficiency and scalability. We demonstrate this significant gain in both accuracy and efficiency GAOT over several baselines on a large number of learning tasks from diverse set of PDEs, including achieving state of the art perfor- mance on three large scale mtree-dimensiones industrial CFD datasets. Our project page for accessing the source code is available at
Introduction
Partial Differential Equations (PDEs) are widely used mathematically model very diverse natural and engineering systems [17]. Currently, numerical algorithms, such as the finite element and finite difference methods, are the preferred framework for simulating PDEs [41]. However, these methods can be computationally very expensive, particularly for the so-called many-query problems such as uncertainty quantification (UQ), control, and inverse problems. Here, the solver must be called repeatedly, leading to prohibitive costs and providing the impetus for the design of fast and efficient surrogates for PDE solvers [34].
To this end, ML/AI based algorithms are increasingly being explored as neural PDE surrogates. In particular, neural operators [24, 6]. including those proposed in [27, 28, 32, 42, 22], which learn the PDE solution operutor from data, are widely used [4]. As many of these neural operators are restricted PDEs on Cartesian (regular) grids, they cannot be directly applied to most engineering
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

and industrial systems, which are set on domains with complex geometries, imposing pressing need for neural operators for learning PDEs on arbitrary domains (point clouds).
In this context, a variety of options have recently been proposed, including domain mask- ing for FNO and CNO [42], replacing FFT in FNO with direct spectral evaluations [30], augmenting FNO with learned diffeomorphisms [26] and mapping arbitrary point cloud data between the input domain and latent regular grids with learned encoders/decoders, while
processing on the latent grid with FNO [29] or transformers [1, 50]. Alternatives such
as end-to-end passage-passing based graph neural networks [38, 18, 44, 43, 8, 16, 36] or end-to-end transformer based models [49, 33, 20, 48, 45] have also been proposed.
thorough comparison of the existing models, not
just in terms of accuracy but also computational efficiency and scalability, is necessary to evaluate whether these models are yet capable enough to
Accocact
Elfckocc
act as surrogates for highly successful finite ele- ment methods for engineering simulations [41]. As one of the contributions in this paper, we per- formed such comparison (see Sec.3 for details) to find evidence for an accuracy- accuracy-eficienncy trade- off. i.e., accurate and robust models, such the message passing based RIGNO of [36] are not necessarily computationally efficient scalable in terms training throughput and inference la- tency. On the other hand, more efficient models such as GINO [29] are not accurate enough (see the accompanying Radar Chart in Fig. 1). Given this observation, our main goal in this paper is propose an algorithm learn PDE solution operators arbitrary domains that is accurate, computationally efficient, and can be seamlessly scaled to real-world industrial simulations.
Tew Sx.
Labey
34aada (2300
Figure I: Normalized performance of GAOT and baselines across eight axes, covering accuracy (Ace.), robustness (Robust), throughput (Tput), scalability (Scal.) on time-dependent (TD) and fime-independent (TI) tasks.
To this end, we propose Geometry Aware Operator Transformer (GAOT, pronounced goat) as a neural surrogate for PDEs on arbitrary domains. While being based on the well-established encode-procett process-decode paradigm [38], GAOT includes several novel features that are designed to ensure both computational efficiency and accuracy, such as
Our proposed multiscale attentional graph neural operator (MAGNO) as the encoder between inputs on an arbitrary point cloud and coarser latent grid, designed to enhance accuracy through its multiscale information processing and attention modules.
Novel Geometry embeddings in the encoder (and decoder) that provide the model with access to information about the (local) domain geometry, greatly increasing accuracy.
A transformer processor that utilizes patching (as ViTs [12]) for computational efficiency,
A MAGNO decoder, able to generate neural fields, with the ability approximate the underlying solution at any query point in the domain.
A set of novel implementation strategies to ensure that the computational realization of GAOT efficient and highly scalable.
Combining these elements allows GAOT to treat PDEs on arbitrary domains in a robust, accurate and computationally efficient manner. We demonstrate these capabilities by.
Extensively testing GAOT on 28 challenging benchmarks for both time-independent and time-dependent PDEs of various types, ranging from regular grids to random point clouds to highly unstructured adapted grids, and comparing with widely-used baselines to show that GAOT both highly accurate as well as computationally efficient and scalable, see Fig.
The efficiency and scalability of GAOT is further showcased by achieving state of the art (SOTA) performance on the large scale three-dimensional industrial benchmark of DrivAer- Net++ dataset for automobile aerodynamics [14]. We also test GAOT and demonstrate its superior performance the GINO bascline on two further 3D industrial scale datasets. i.c.,

---

DrivaerML dataset for automobile aerodynamics and a NASA-CRM dataset for aerospace applications.
Through extensive ablations, we also highlight how the novel elements in the design of GAOT such as multiscale attentional encoders and geometry embeddings crucially contribute to the overall performance of our model.
2 Methods.
Problem Formulation. We start with a generic iime-independent PDE,
Díc, u) f. D R", B(u)= = ub. x E AD,
(I)
with u D R., the PDE solution, es the coefficient (PDE parameters), f is the forcing term, u6 boundary values and D and B are the underlying differential and boundary operators, respectively. Denoting as XD. a function (e.g. indicator or signed distance) parameterizing the domain D, we combine all the imputs to the PDE (1) together into a = (c, f. ub, XD), then the solution operator maps inputs into PDE solution with u = Sa. The corresponding operator learning task is to learn the solution operator 8 from data. To this end, let μ be an underlying data distribution. We sample i..dd inputs (+) ~ μ. for 1 S S M and assume that we have access to data pairs (at), u()) with ( = (a Thus, the operator learning task is to approximate the distribution S#y from these data pairs. In practice, we can only access discretized versions of the data pairs, sampled collocation points (which can vary over samples).
Similarly denoting a generic time-dependens PDE as,
ut + D(c, u) = 0, Vx D E T] m00)= uo. D,
with, u D 0, T] RT, the PDE coefficient and uo the initial datum and the underlying (spatial)
differential operator D. Clubbing the inputs to the PDE (2) into a = (c, uo, "0, XD), the corresponding solution operator St, with u(t) Se(a) for all [0, T). maps the input into trajectory of the solution. The operator learning task consists of approximating (8,) #y from data pairs (al), ud (t))
for E [0, T()) and S i S M with samples a, drawn from the data distribution Js. However in practice, we only have access data, sampled on discrete set of spatial points per sample well as only on discrete time snapshots {( E (+)] and have to learn the solution operator from them.
The overall architecture of GAOT is depicted in Fig. 2. For simplicity of exposition, we start with the -nneeindeeendent case, where given inputs a(x;) on the point cloud solution of the PDE (1)
GAOT Model Architecture.
DA { D, for 1 < j S J, GAOT provides approximation any query point x E D. At high level, GAOT follows the encode-proced dreced paradigm of [38]. In the first step, an encoder transforms the input on the underlying point cloud Da a latent point cloud D C Rdd The resulting spatial tokens are then processed by processor module to learn useful representations and its output is remapped the original domain D via the decoder, which allows evaluation at any query point E D.
Choice of Latent Domain. As depicted in SM Fig. B.1, the latent domain D (to which the encoder maps) can be chosen in three different ways, namely i) regular (structured) grid stencil, consisting of equispaced points on Cartesian domain (see also Fig. 2) ii) randomly downsampling the underlying point cloud Da or iii) a projected low-dimensional representation, where high-dimensional domain is projected to a lower dimension (for instance using tri-plane embeddings in 3-D [10]) and a regular grid is used in the nower-dimensions domain. GAOT is a general framework where any of these latent point cloud choices can be employed for D.
Encoder. Given input values (Irz) on the underlying point DA. the encoder aims transform it into latent features we (y) at any point y E 'D on the latent point cloud. Using a graph-neural operator (GNO) encoder as in GINO [29] would lead to,
nl
We (
a og K(, In, ((z))(a)(
(3)
k=1
3

---

Encoder
@ Concoterate Uneer
Weggred Bunn
Physical Fied
Processor
AGNO
GEmb
Trorossig
AGNO
Tolee Pitennaattioon
GEmb
Embediking
AGNO
Gecmetry-Auore
Transfomer Block
liver
Playoral
Transfomer Block
Decoder
MAGNO
AGNO
GÉmb
Transforer Block
AGNO
GEmb
Transforer Block
Linear
Figure Schematic of the GAOT with an equispaced latent token grid. The encoder uses a multiscale attentional graph neural operator (MAGNO) aggregate the input data into geometry-aware tokens. A vision transformer (ViT) block with residual connections processes tokens, enabling global exchange of information. A MAGNO decoder identifies the nearest tokens around given query point to decode the final field.
with MLPs K and and the sum above taken over all the ny points 23k E DA such that ly - Ix S 0, where ag are some given quadrature weights. In other words, GNO accumulates information from all the points in the original point cloud that lie inside ball of radius r, centered at the given point y in the latent point cloud, and processes them through a kernel integral.
for some hyperparameter
Our first innovation is based the realization that this single-scale approach might be limiting the overall accuracy. Instead, we would like to introduce mechanism to integrate multiscale information into the encoder. To this end and as shown Fig. 2, choose I'm = Smao for some base radius To mi to modify GNO (3) by,
and scale factors S,,, for m
oh K" a(x))))(((()))
for any fixed scale and with MLPs K" s. The above sum is taken over all the ne points ax D such that ly xxl S Tm- To choose the quadrature weights ag we propose attention based choice.
exp(e
(W y. W
0%
Vd
exp(eX)
with Wm. Wmh E dddmm are query and key matrices respectively, completing the description of the attentional graph neural operator (AGNO) (4) at each scale m.
Geometry Embeddings. The only geometric information in the afore-described encoder is provided by the coordinates of the underlying points. This alone does not convey the rich geometric information about the domain that can affect the solution of the underlying PDE (2). Hence, we need to embed further geometric information into the model. Deviating from the literature where geometric information is provided either by appending them as node and edge features on the underlying graphs (18) or by encoding a signed distance function [29], we propose to use novel geomeiry embeddings to encode this information. To this end and as described in SM Sec. B.3, we can rely local statistical embeddings for each point y as all the neighboring points Fx in DA with [y - F&l S I'm have already been computed in the AGNO encoder. From these points, we can readily compute statistical

---

descriptors such as i) number of neighbors 24 E DA, in the ball Br.,, (v), ii) the average distance Davg Σx= y sul. iii) the variance of this distance Dvar. with respect to the average Davg: iv) the centroid offset vector Ay = 1 (# - y) and v) a few principal component (PCA) "
features of the covariance matrix of y I& to calculate the local shape anisotropy. These statistical descriptors, for each scale m and each point E D are then concatenated into single vector Zy. normalized across components to yield zero mean and unit variance and fed into MLP provide the embedding g"(()) Alternatively, geometry embedding using PointNet models [39] can also be considered.
MAGNO. As shown in Fig. 2. the scale-dependent AGNO W. (4) and the geometry embedding g²t, at each scale m, can be concatenated together and passed through another MLP to yield a scale-specific latent features function (y). Next, we need to integrate these features across all m scales. Instead of naively summing these scale contributions, we observe that different scales might contribute differently for every latent token to the encoding. To ascertain this relative contribution, we introduce a (small) MLP V'm and weigh the relative contributions with a softmax and combine them into the multiscale attentional graph neural operator or MAGNO encoder by setting,
Vy E D, Bm( = exp(wm (w))
we(y)
(m
(6)
exp(vme(x))
Transformer Processor. The encoder provides a set of geometry aware tokens we (Be), for all points ye E D. with < L S L, in the latent point cloud. These tokens are further transformed by processor. As shown in Fig. 2, we choose a suitable transformer based processor. While postponing details the processor architecture SM Sec. B.4, we summarize points {ye} lie on a regular grid (either through a structured stencil or a projected low-dimensional one), we patch-based vision transformer ViT ([12) and [22, 35] for PDE operator learning) for computational efficiency. The equispaced latent points are combined into patches and the tokens in each patch are flattened into single token embedding which serves as the input for a multi-head attention block, followed by feed forward block. RMS normalization is applied to the tokens before processing. Either sinusoidal absolute position embeddings or rotary relative position embeddings are used to encode token positions. If the latent points are randomly downsampled from the original point cloud, there is no obvious way to patch them together. Hence, a standard transformer [47], but with RMS normalization, can be used. Additionally, we employ multiple skip connections across transformer blocks (see Fig. 2). The transformer processor transforms the tokens (e(e)) into processed tokens, that we denote by (Ve)), for all < 1 < L.
choices here. If the latent
Decoder. Given any query point x E D the original domain, the task of the decoder GAOT is to provide w(xz), which approximates the solution u of the PDE (1) at that point. To this end, we simply employ the MAGNO architecture in reverse. By choosing a base radius fu and scale factors 8m. set of increasing radii f... fo selected to define set of increasing balls B, (x) around the query point æ (Fig. 2). A corresponding AGNO model is defined by replacing y + x. 2g + ye and - up in (4), with corresponding attentional weights computed via (5). In parallel, geometry embeddings over each ball B;,,, (z) are computed to provide statistical information about how the latent points ye distributed in the neighborhood of the query point These AGNO features and geometry embeddings are concatenated and passed through a MLP to provide w(x) which has the desired dimensions of the solution of the PDE (1). We denote the GAOT model as Sa with the output w = (a)a). for the inputs a to the PDE (1). It is trained to minimize the mismatch the underlying operator 8, i.e, the parameters are determined minimize loss L(8(a), 8g(a)), over all input samples a', with L being either the absolute or mean-squaie errors.
Extension to time-dependent problems. To learn the solution operator 8 of the time-dependent PDE, we observe that the Sn can be used to update the solution forward in time, given the solution at any time point u(t) by applying ult + r) = S, (t)) Thus, for any time I, given the augmented input al) = (c, u(()), with e being the coefficient in the PDE (2), we need GAOT to output u(t + +r), for any - 2 0. To this end, we retain the architecture of GAOT, as described for the time-independent case above, and simply add the current time and the lead-time 7 as further inputs to the model.
5

---

Table 1: Benchmark results on time- dependenndd and -ndeeenneenddntt datasets. Best and 2nd best models are shown in blue and orange fonts for each dataset.
Dataset
Median relative L error [%]
Iimeennndpendent
GAOT
RIGNO-18
Transolver
GNOT
UPT
GINO
Poisson-C-Sines
3.10
6.83
77.3
100
100
20.0
Poisson-Gauss
0.83
2.26
2.02
88.9
48.4
7.57
Elasticity
1.34
4.31
4.92
10.4
12.6
4.38
NACA0012
6.81
5.30
8.69
6.89
16.1
9.01
NACA2412
6.66
6.72
8.51
8.82
17.9
9.39
RAE2822
6.61
5.06
4.82
7.15
16.1
8.61
Bluff -Body
2.25
5.76
1.78
44.2
5.81
3.49
Time-De Senendeeenddet
GAOT
RIGNO-18
GcoFNO
FNO DSE
UPT
GINO
NS-Gauss
2.91
2.29
41.1
38.4
92.5
13.1
NS-PwC
1.50
1.58
26.0
56.7
100
5.85
NS-SL
1.21
1.28
13.7
22.6
51.5
4.48
NS-SVS
0.46
0.56
9.75
26.0
4.2
1.19
CE-Gauss
6.40
6.90
42.1
30.8
64.2
25.1
CE-RP
5.97
3.98
18.4
27.7
26.8
12.3
Wave-Layer
5.78
6.77
11.1
28.3
19.6
19.2
Wave-C-Sime
4.65
5.35
13.1
5.52
12.7
5.82
More precisely, the time-dependent version of GAOT is the form S0 a())), where a(t) takes values at points sampled in D. Following [36], the map Sg can be used to update approximate solution of PDE (2) time by following very general time-stepping strategy:
St T,)(t)) = vu(() 58, (ax a(t).
Here, choosing the parameters (7, 8) appropriately leads to different strategies for time stepping:
y 0, directly approximates the outpul of the solution operator time + 7: 1,
yields the residual of the solution at the later time, with respect to the solution at current time; 1 is equivalent approximating the fime-derivative of the solution. GAOT provides the flexibility to use any of these time-stepping strategies. We also use the all2all training strategy [22] leverage trajectory data for time-dependent PDEs.
Efficient implementation. As our goal is to ensure accuracy and computational efficiency, we have designed GAOT with ability for large-scale computations in mind. We started with the realization that the heaviest burden of the computation should fall on the processor. The encoder and decoder are often responsible for memory overheads these modules entail sparse computations on graphs with far more edges than nodes, making the computations largely edge-based and leading to high (and inefficient) memory usage. Moreover, many PDE learning tasks on arbitrary geometries, underlying domain (and the resulting graph) varies significantly between data samples, making load balancing very difficult.
To address these computational challenges, we resorted to i) moving the graph construction outside the model evaluation by either storing the graph, representing the input point cloud, in memory for small graphs on disk for large graphs and loading them during training with efficient data loaders ii) sequentially processing each input in given batch for the encoder and decoder, while still batch processing in the transformer processor, allowing us to reduce memory usage while retaining efficiency and iii) needed for very large-scale datasets, we use an edge-dropping strategy to further increase the memory usage of the encoder and decoder. These innovations are essential to ensure batch training and underpin the efficiency GAOT, even when input geometries vary significantly. A more detailed discussion on these novel implementation tricks is provided in SM E.2.
3 Results.
Datasets and Baselines. We start by testing GAOT of a challenging suite of 15 datasets for PDEs with input/output data arbitrary point clouds in two space dimensions, see SM Secs D and G

---

Moddd
(a) Grid vs. Throughput
(b) Model vs. Throughput
(c) Transfer Learning
Figure 3: Training throughput (samples/s) with increasing input grid size (a) and model size (b) for proposed GAOT, GINO, RIGNO and Transolver. (c) Transfer learning performance of GAOT on unseen bluff body shapes (See SM Sec. E. for dataset details). FT (fine-tuning) adapts a pretrained GAOT model from Table I, while TFS denotes training from scratch. FT consistently outperforms TFS across varying numbers of task-specific training samples.
for a detailed description of the datasets and for visualizations, respectively. For time- lime-independent PDEs, in addition to the elasticity benchmark of [26], we consider two Poisson equation datasets: Poisson-Gause Gauss, defined on random points in square domain, and Toisson-C-Sines we propose, containing rich multiscale solutions on a circular domain. In addition, we propose new datasets comprising compressible flows past objects, both airfoils as well as bluff bodies. These datasets have significant variation in domain geometry and flow conditions (Mach numbers ranging from subsonic to supersonic, varying angles of attack etc.) and are tailor-made for testing neural PDE surrogates on arbitrary domains in two space dimensions. For dimeddependent PDEs, we test on the challenging datasets considered recently in [36], composed of 8 operators corresponding to the compressible Euler (2), incompressible Navier-Stokes (4), and acoustic wave equations (2). These dime-dependent operators include complex multiscale solutions with shocks and other sharp traveling waves which can interact, reflect and diffract making them hard to learn. We test GAOT on these datasets and compare them with several widely used neural operators for PDEs arbitrary domains including those based on message passing (RIGNO [36]), Fourier Layers (GINO [29], GeoFNO [26], FNO DSE(30]) and Transformers (Transolver [49], UPT [1] and GÑOT [20]).
new dataset
Accuracy and Robustness. In Table 1, we present the relative test errors for the above datasets to observe that GAOT is very accurate on all of them, being either the best (10) second-best (4) model on 14 of them. On average, over the indeinndeenddnt datasets, GAOT almost 50% more accurate than the aecond-best performing model CRIGNO-18) while on time-dependent datasets, it is slightly more accurate than the second-bess best performing model (RIGNO-18). What even more noteworthy is the robustness of the performance of GAOT over all the datasets. As seen from Table 1, the accuracy of GAOT is uniformly good over all the datasets and does not deteriorate on any of them. On the outlier datasets. This robustness
other hand, all the baselines show significantly poor performance robustness score (see SM Sec. E.3) to find that GAOT is almost three times more robust the time- GAOT and RIGNO-18 18 are as robust as each other on the time- dependent datasets.
can be quantified in terms imme-independent datasets as the second-bes. model 18) while
Computational Efficiency and Scalability. It is worth reiterating that the computational efficiency of MI. model is a significant marker of overall performance. We test efficiency in terms of two critical quantities, the train- ing throughput and the inference latency. For given input and model size, training throughput measures the number of samples that model can process during training (for- ward pass, backward pass and gradient update) per unit time (in seconds) on a given compute system (GPU or CPU). The higher the training throughput, the faster the model can be trained. On the other hand, the inference latency is the amount of time it takes for model to infer a single input. We present the throughput and latency for GAOT and three selected basclines (RIGNO-18 18 for Graph-based, GINO for FNO-based and Transolver for Transformer-bast
Table 2: Comparison of model size (Params.), throughput (samples/s), and latency (ms) across GAOT and represen- tative baselines.
Model
Params. (M)
Tput.
Latency
GAOT
5.62
97.5
6.966
GINO
6.04
60.4
8.455
RIGNO-18
2.69
12.74
Transolver
3.86
39.5
15.29
7

---

Input Mesh
GT Pressure
Pred Pressure
GT WSS-x
Pred WSS-
Figure 4: Comparison of predicted and ground -ruth (GT) results for the pressure and wall shear stress in the x-direction (WSS-x) on the DrivAerNet++ test sample H_S_WWS_WM__17 172.
models) in Table 2 for learning tasks (such as the bluff-body dataset for compressible flow) where the domain geometry varies throughout the dataset. These experiments are conducted on one NVIDIA GeForce RTX 4090 GPU with float32 precision. We see from this table that GAOT has the highest training throughput and the fastest inference latency, being almost 50% and 15%, respectively better than the second-mos -ost efficient model (GINO).
How the training throughput of model scales with increasing input and model size, is absolutely crucial for evaluating whether it can be used to process large-scale datasets (input scalability) or whether it can serve backbone of foundation models (model scalability) which require large model sizes [22]. To evaluate the scalability of different models, we plot how the training throughput changes input size and model size (Fig. 3 (a, b)) are increased find that GAOT scales much more favorably than the baselines with respect to both input and model size. In fact, models like Transolver and GNOT scale very poorly, making it impossible for us to train them for the large-scale time-dependent datasets with all2all training, which requires handling large volumes of data for large input sizes. Hence, we omit them in the accuracy results for time-dependent datasets in Tab. 1. The results for both accuracy and efficiency across a range of metrics for GAOT, RIGNO, GINO and Transolver are summarized in SM Tab. E.4 and visualized in the Radar chart Fig. 1, This demon- strates that GAOT ensures both accuracy (robustness) and computational efficiency (scalability) at the same time, while being the best model performing model on both sets of metrics.
Industrial scale 3D datasets. Given the high accuracy and excellent computational efficiency and scalability of GAOT, we showcase its abili- ties further on three challenging three-dimensional
Table 3: Error metrics of MSE (x10-2) and Mean AE (x10-1) for Pressure and Wall Shear Stress on the Driv dataset.
large-scale benchmarks for industrial simulations.
Pressure
Wall Shear Stress
We start with the DrivAerNet++ dataset of [14]. In this benchmark, the data consists of high-fidelity CFD simulations across 8K different car shapes which span the entire range of conventional car de- sign. The underlying task is to learn steady-state surface fields (See Fig. 4) such as the pressure and wall shear stress, given the input car shape and flow conditions. The data has approximately 500K points per shape, making the overall training extremely compute intensive. Thus, only scalable models can currently process this learning task. We test GAOT on this challenging 3D benchmark and report the RMSE and MAE test errors for the pressure and wall shear stress in Tab. 3. Compared to basclines results taken from the leaderboard of the Driv AerNet++ challenge [10], we see that GAOT significantly improves on the state-of -f------tttttttt (see also Fig. 4). This improvement is most visible in the MAE for wall shear stress where GAOT is ca. 30% more accurate than the second-best model (TripNet), which currently sits atop the leaderboard for wall shear stress predictions. We recall that GAOT's decoder endows with neural field properties. We showcase for the DrivAerNet++ dataset by training GAOT on randomly selected set of less than 10% of the total input points (per batch) and then testing on the original car surface point cloud by querying the desired points through GAOT's decoder. Although not as accurate as training GAOT with full input, we observe from Tab. 3 that this neural field version of GAOT has comparable accuracy to some of the baselines which have been trained with 10x more input points, further demonstrating the flexibility and accuracy of
Model
MSE
Mean AE
MSE
Mean AE
GAOT
4,2694
1.0699
8,6878
1,5429
FIOComvNet
4,9900
1.2200
9,8600
2,2200
TripNet
5,1400
1,2500
9.5200
2,1500
RegDGCNN
8,2900
1.6100
13.8200
3.6400
GAOT (NeurFiiek))
12,0786
1.7826
22.9160
2.5099
8

---

Table 4: Comparison of GAOT and GINO across two benchmarks with MSE (x10-2) and Mean
AE (x10-)). Cp: Pressure Coefficient, WSS: Wall Shear Stress, P. Pressure, Cf: Skin Friction
Coefficient, DML: DrivaerML dataset, CRM: NASA CRM dataset.
Cp (DML)
WSS (DML)
P (CRM)
Cf (CRM)
Model
MSE
Mean AE
MSE
Mean AE
MSE
Mean AE
MSE
Mean AE
GAOT
5.1729
1.2352
16,9818 28.4832
2.1640 2.7330
7.7170 10.5688
1.6014 1.7450
16,1091 21.1789
2.2305 2.4240
GINO
8.8124
1.5238
NASA CRM
MASK CAN
NASA CRM Pred
Trboeeki..rqu
Dewerk GT
Inhorki Peal
Figure 5: Comparison of predicted and ground-truth (GT) results for the pressure on the test sample of DrivAerML and NASA-CRM.
GAOT, Further assessments of the neural field property of GAOT are provided in SM E.8. A natural follow-up question in this regard is whether training at full (high) resolution provides significant advantage over training at low resolution and inferring at high resolution Recent works such as [15] have investigated this question in the context of the DrivAerNet++ dataset. We explore this issue in SM E.9 and demonstrate that GOAT trained at full resolution is significantly (almost twice) more accurate than competing models trained low resolution and inferred at full resolution, see SM Table E.8.
Next, we consider the very challenging Driv AerMI. dataset of [3] (see Fig. 5), where the learning task is exactly the same as in Driv ArimAtrnet++, i.c., predicting surface fields on the car, given its shape as the input. However, unlike +rivAernet++ which was based on coarse RANS simulations. Driv DrivAssML's ground truth is based on highly accurate LES simulations. This enables the incorporation of much more fine-scale physical effects in this dataset. The learning problem becomes harder, not just in terms of the challenging underlying physics, but also the fact that the number of points on the car surface is now 9M, instead of 500K for DrivAermet++. Thus, only highly scalable models can deal with this extreme resolution. Consequently, we are only able to test GAOT and GINO for this dataset and report the results in Table 4 to observe that GAOT is significantly (almost twice wall shear-stress) as accurate as GINO on this dataset.
Finally, we consider an industrial- scale dataset, recently proposed in [7], where the learning task (sec Fig. 5), is to predict surface pressure and the skin friction coefficient, given the shape of full aircraft. The ground truth is generated with RANS simulations using Spalart-Allmaras turbulence model, and the results with GAOT and GINO are reported in Table 4, showing that GAOT significantly outperforms GINO this large-scale industrial dataset.
Generality, Generalization and Scaling. We highlight GAOT's flexibility with respect to the point distributions that it can handle by testing it on PDEs with regular grid inputs, as suggested in [36]. To this end, we considered 7 additional datasets and present the test errors in SM Sec. F.5. to find that GAOT is highly accurate even on regular grids and is either more accurate or comparable to the highly expressive GNN-based RIGNO, while being more accurate than widely used neural operators such as FNO and CNO. A key requirement in operator learning [24, 6] the ability of the model to generalize (at test time) to input and output resolutions that are different from the training resolution. As GAOT can be readily evaluated at any query point, we showease this aspect of GAOT in SM Sec. E. by plotting the test errors for a sequence of resolutions, different from the training resolution, for the Poisson-Gaus and resoutresol settings, even to grids with 10x more input points than the training resolution. Another test of the generalizability of a model is its ability to perform well out-of -fd---diistiiuuinnn either zero-shot or when it is fine-tuned with few in-distribution samples for the new learning task. To
benchmark, to find that GAOT generalizes very well both the sub-

---

test this aspect, we consider the datasets for compressible flow past bluff bodies and train a GAOT model on a set of bluff body shapes and then test on shapes that were not the training set. Then, the model is fine-tuned with a few task- spec--seeificc samples and the results are shown in Fig. 3 (c). We observe that our model performs very well in a few-shot transfer learning scenario, with the fine-tuned model providing an almost order of magnitude gain in accuracy over the model, trained from scratch. Finally, in SM Sec. E.6, we demonstrate that GAOT scales with both model and dataset size, with scaling with respect to dataset size, also illustrated in Fig. 3 (c)
Why does GAOT work so well To answer this question, we have performed extensive ablation studies in SM Sec.F to observe that i) the MAGNO encoder/decoder is clearly superior to message- passing based encoders/decoders, ii) choosing a regular equispaced latent point cloud performs significantly better than either downsampling on the original point cloud using projected low- dimensional regular grid, iii GAOT is highly robust to the size of its latent grid, iv) a time-derivative marching strategy, L., setting ? 1,
in (7) is superior to other choices of y, 6, v) using a statis- tical geometric embedding performs significantly better than either not using additional geometric information using a pointnet process geometric information vi) incorporating multiscale features in the MAGNO encoder/decoder provides a significant gain in accuracy when compared to using just a single scale GNO encoder/decoder as in GINO and vii) The power of GAOT does not just stem from its VIT processor, but also from its MAGNO encoder/decoder (SM E.5), acting in tandem. These results justify the choices that we have made in designing GAOT and selecting the relevant model components, while also revealing how these innovative features underpin GAOT's accuracy. However, as argued before, this accuracy might come at the price of computational inefficiency. But, as we have demonstrated above, GAOT is also the most efficient model and does not have to pay the securacy-efficinnee trade-off. The reasons behind this boil down to the tricks used in efficiently implementing GAOT, which are discussed at length in SM E.2 and E.
4 Discussion
Summary. We present GAOT, new neural operator for learning PDE solutions on arbitrary domains. It is based on a novel multiscale GNO encoder/decoder, combined with geometric embeddings that convey statistical information about the local domain geometry. and a (vision) transformer based processor. The model is designed to handle any point cloud input and provide the output at any query point in the underlying domain. Several innovative strategies have been used to make the implementation of GAOT computationally efficient and scalable. We test GAOT large number of challenging datasets for a variety of time-dependent and time-independent PDEs over diverse wooddimensional domain geometries to find that GAOT is significantly more accurate, robust and computationally efficient in terms of training throughput and inference latency, over a large set of baselines. We further demonstrate the potential of GAOT by presenting its SOTA performance on three large-scale three-dimensional datasets of industrial simulations in the automobile and aerospace sectors. These results demonstrate that GAOT can be a powerful and scalable neural operator with wide-spread applications. They also showcase the main advantage with an efficient and accurate neural operator such as GAOT, i.e, its inference time is many orders of magnitude faster than the runtime of a classical numerical PDE solver. We quantify this speedup in SM 1. to find that GAOT can be anywhere between 4 to 9 orders of magnitude faster to run than classical PDE solvers, while retaining accuracy.
Related Work. As discussed before, there are 3 broad classes of models for learning PDEs on arbitrary domains namely i) end-to-end message-passing based frameworks exemplified here with RIGNO, which significantly improves upon models such as (multiscale) MeshGraphNets [38, 18) ii) Transformer based frameworks such as Transolver [49] GNOT [20] and UPT [1] and iii) frameworks, based on GNO encoders/decoders and FNO processors as in GINO [29]. GAOT differs from all these approaches by a) not using graph- based message passing, b) only employing transformers in the processor c) using a transformer, rather than FNO as a processor and significantly augmenting the GNO encoder/decoder by multiscale features, attention based quaddruuar and geometry embeddings. It is precisely these choices, along with a highly efficient implementation, that allows GAOT to significantly surpass GINO, RIGNO, and Transolver in both accuracy and efficiency.
10

---

Acknowledgements
The contribution of Siddhartha Mishra to this work was supported in part by the DOE SEA-CROGS project (DE-SC-0023191), The authors thank Mohamed Elrefaie and Prof. Faez Ahmed (MIT) for their inputs regarding the Carbench datasets and results.
References
[1] Benedikt Alkin, Andreas Fürst, Simon Lucas Schmid, Lukas Gruber. Markus Holzleitner, and Johannes Brandstetter. Universal physics transformers: A framework for efficiently scaling neural operators. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024.
[2] Benedikt Alkin, Richard Kurle, Louis Serrano, Dennis Just, and Johannes Brandstetter. Ab-upt for automotive and aerospace applications, 2025.
[3] Neil Ashton, Charles Mockett, Marian Fuchs, Louis Fliessbach, Hendrik Hetmann, Thilo Knacke, Norbert Schonwald, Vangelis Skaperdas, Grigoris Fotiadis, Astrid Walle, et al. Dri- vaerml: High-fidelity computational fluid dynamics dataset for road-car external aerodynamics. arXiv preprint arXiv:2408. 19969, 2024.
[4] Kamyar Azizzadenesheli, Nikola Kovachki, Zongyi Li, Miguel Stu-Schiafiii,, Jean Kossaifi, and Anima Anandkumar. Neural operators for accelerating scientific simulations and design. Nature Reviews Physics, pages 1-9, 2024.
[5] Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 22669226679 2023.
[6] Francesca Bartolucci, Emmanuel de Bezenac, Bogdan Raonic, Roberto Molinaro, Siddhartha Mishra, and Rima Alaifari. Representation equivalent neural operators: a framework for alias- free operator learning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[7] Philipp Bekemeyer, Nathan Hariharan, Andrew M Wissink, and Jason Cornelius. Introduction of applied aerodynamics surrogate modeling benchmark cases. In AIAA SCITECH 2025 Forum, page 0036, 2025.
[8] Johannes Brandstetter, Daniel Worrall, and Max Welling. Message passing neural PDE solvers. In International Conference Learning Representations, 2022.
[9] Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, et Efficient geometry- aware 3d generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16123-16111 16133, 2022.
[10] Qian Chen, Mohamed Elrefaie, Angela Dai, and Faez Ahmed. Tripnet: Learning large-scale high-fidelity 3d car aerodynamics with triplane networks, 2025.
[11] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
[12] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
[13] Thomas D Economon, Francisco Palacios, Sean R Copeland, Trent W Lukaczyk, and Juan J Alonso. Su2: An open-source suite for multiphysics simulation and design. Aiaa Journal, 54(3):828-846 2016.
11

---

[14] Mohamed Elrefaie, Florin Morar, Angela Dai, and Faez Ahmed. Drivaerne+++: A large- scale multimodal car dataset with computational fluid dynamics simulations and deep learning benchmarks, 2025.
[15] Mohamed Elrefaie, Dule Shu, Matt Klenk, and Faez Ahmed. Carbench: A comprehen- sive benchmark for neural surrogates on high-fidelity 3d car aerodynamics. arXiv preprini arXiv:2112.07847, 2025.
[16] Léonard Equer, T Konstantin Rusch, and Siddhartha Mishra. Multi-scale message passing neural PDE solvers. arXiv preprint arXiv:2302.0338/ 2023.
[17] Lawrence C Evans. Partial differential equations, volume 19. American Mathematical Society, 2022.
[18] Meire Fortunato, Tobias Pfaff, Peter Wimssbegge, Alexander Pritzel, and Peter Battaglia. Multiscale meshgraphnets. arXiv preprint arXiv:2210.00612, 2022.
[19] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. International conference on machine learning. pages 1263-1272. PMLR, 2017.
[20] Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, pages 12556-12569. PMLR, 2023,
[21] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016.
[22] Maximilian Herde, Bogdan Raonic, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Em- manuel de Bezenac, and Siddhartha Mishra. Poseidon: Efficient foundation models for PDEs. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[23] Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311. 1.04000, 2023.
[24] Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1-97, 2023.
[25] Brenda Kulfan. A Universal Parametric Geometry Representation Method "CST".
[26] Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. Journal of Machine Learning Research, 24(388):1-26, 2023.
[27] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differen- tial equations. arXiv preprint arXiv:2010.08895 2020.
[28) Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar, Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03448 2020.
[29] Zongyi Li, Nikola Borislavov Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Prakash Otta, Mohammad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadenesheli, and Anima Anandkumar. Geometry-informer neural operator for large-scale 3d PDEs. In seirthy-eventh Conference on Neural Information Processing Systems, 2023.
[30] Levi F. Lingsch, Mike Yan Michelis, Emmanuel de Bezenac, Sirani M. Perera, Robert K. Katzschmann, and Siddhartha Mishra. Beyond regular grids: hourier-based neural operators on arbitrary domains. In Forty-first International Conference Machine Learning, 2024.
12

---

[31] Adrien Loseille, Alain Dervieux, Pascal Frey, and Frédéric Alauzet. Achievement of global second order mesh convergence for discontinuous flows with adapted unstructured meshes, In 18th AIAA computational fluid dynamics conference, page 4186, 2007.
[32] Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218-22 2021.
[33] Huakun Luo, Haixu Wu, Hang Zhou, Lanxiang Xing, Yichen Di, Jianmin Wang, and Mingsheng Long. Transolver++: An accurate neural solver for PDEs million-scale geometries. In Forty-second International Conference on Machine Learning, 2025.
[34] Siddhartha Mishra and Alex (Eds.) Townsend. Numerical Analysis meets Machine Learning. Handbook of Numerical Analysis. Springer, 2024.
[35] Roberto Molinaro, Samuel Lanthaler, Bogdan Raonić, Tobias Rohner, Victor Armegioiu, Stephan Simonis, Dana Grund, Yannick Ramic, Zhong Yi Wan, Fei Sha, Siddhartha Mishra, and Leonardo Zepeda-Núñez. Generative ai for fast and accurate statistical computation of fluids, 2025.
[36] Sepehr Mousavi, Shizheng Wen, Levi Lingsch, Maximilian Herde, Bogdan Raonić, and Sid- dhartha Mishra. Rigno: A graph-based framework for robust and accurate operator learning for pdes on arbitrary domains, 2025.
[37] Cens-Dominik Muller. On triangles and flow. University of Michigan, 1996.
[38] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter Battaglia. Learning mesh- based simulation with graph networks. In International Conference on Learning Representations, 2021.
[39] Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation, 2017.
[40] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652-660, 2017.
[41] A. Quarteroni and A. Valli. Numerical approximation of Partial differential equations, vol- ume 23. Springer, 1994.
[42] Bogdan Raonic, Roberto Molinaro. Tim De Ryck. Tobias Rohner, Francesca Bartolucci, Rima Alaifari, Siddhartha Mishra, and Emmanuel de Bezenac. Convolutional neural operators for robust and accurate learning of PDEs. Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[43] Alvaro Sanzhez-Gonvalez,, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph networks. In International conference on machine learning, pages 8459-8468. PMLR, 2020.
[44] Alvaro
Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Ried- miller, Raia Hadsell, and Peter Battaglia. Graph networks learnable physics engines for inference and control. In International Conference on Machine Learning, pages 4470-4479, PMLR, 2018
[45] Louis Serrano, Thomas X Wang, Etienne Le Naour, Jean-Noël Vittaut, and Patrick Gallinari. AROMA: Preserving spatial structure for latent PDE modeling with local neural fields. In The Tiiht Annual Conference on Neural Information Processing Systems, 2024.
[46] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Nearocomputingg 127063, 2024.
[47] Vaswani, Attention all you need. Advances Neural Information Processing Systems,
2017.
13

---

[48] Sifan Wang, Jacob H Seidman, Shyam Sankaran, Hanwen Wang, George J. Pappas, and Paris Perdikaris. CVit: Continuous vision transformer for operator learning. In The Thirteenth International Conference on Learning Representations, 2025.
[49] Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: A fast transformer solver for PDEs on general geometries. In Forty- first International Conference on Machine Learning. 2024.
[50] Anthony Zhou, Zijie Li, Michael Schneier, John R Buchanan Jr, and Amir Barati Farimani. Tex12PDE: Latent diffusion models for accessible physics simulation. In The Thirleenth International Conference Learning Reprexentations, 2025.
14

---

Supplementary Material for:
Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on Arbitrary Domains
Details of Problem Formulation.
17
A.1 General Forms of PDEs
17
A.2 Solution Operators for PDEs
17
A.3 Discretizations.
18
Details of GAOT Architecture
18
B.1 Choice of Latent Grid
19
B.2 Multiscale Attentional Graph Neural Operator
20
B.3
Geometry Embeddings
B.4 Processor
23
B.5
Graph Building Tricks
24
B.6
Training Details.
25
B.7 Inference
27
Baselines
29
C.1 UPT
29
C.2 Transolver
30
C,3 GNOT
D Datasets
32
D.1 Poisson-C-Sines.
................................
34
D.2 Compressible Flow Past Airfoils & Bluff Bodies
................................
E Additional Results
36
E,
Asymptotic Complexity
................................
36
E.2
Runtime Profiling
36
E,3 Accuracy, Robustness and Computational Efficiency Metrics
37
E.4
Results for Radar Chart in Main Text.
39
E.5
Regular Grid Dataset
40
E,6
Model and Dataset Scaling
E,7 Resolution Invariance
43
E.8
Comparison of Neural Field with UPT
44
E,9
Training full vs. low Resolution for the +rivAcrnet++ Dataset.
45
E.
Transfer Learning..
46
E.11
Training Randomness
46
E,12 Runtime Comparison between GAOT and Classical Solvers
F
Ablation Studies
47
15

---

F.1
47
F2
Tokenization Strategies
48
F3
Time-Stepping Method
49
F4
Geometric Embedding
50
E5
Multiscale Features
51
Visualizations Datasets
16

---

A Details of Problem Formulation.
Here, we introduce the core concepts of operator learning for both time-dependent and time- independent partial differential equations (PDEs). We begin by defining the general forms of PDEs under consideration and then discuss the associated operator-leamnin tasks.
A.1 General Forms of PDES
We focus two broad classes of PDEs: time-dependent and time-independent.
Depen-Depnndett PDE. Let D C Rd be a d-dimensional spatial domain, and let (0, T) denote the time interval. A general time-dependent PDE (see Main Text Eqn. (2)) can be written as
01 x(l, + D(c, 1, u, uzu, V.u...) = 0,
v(,x)) E (0, x D,
((u, Va Bu....) ub.
v(t,2) (0,T) OD,
(A.1)
u(0,x)) = (x)),,
Ve D,
where
u(t, r) is the PDE solution in (0, x D;
<(l,x) is a known, possibly spatio-temporal parameter (e.g., material coefficient or source term);
D is a (spatial) differential operator;
B is boundary operator acting on aD;
(x) are boundary values;
* uo(z) is the initial condition at f 0.
We assume u(l,)) .) E X C LP(D; Rm)) for some S < oo and integer m 2 1. Likewise, uo(x) E X20 C X is an element of the initial-condition space, and Q L"(D; ("m)))) is taken from a parameter space.
Timee-nneepeneen PDE. A time- limeiindpeendent (steady-state) PDE of the general form (Main Text Eqn. (1)) can be written as
D(c, ū. V, "......) B(ū, V U, Vuu,...) ub
Va E D,
(A.2)
Vx E OD,
where ū(x) E and c(x) E Q are now independent and f is a source term. certain scenarios, one may view (A.2) as the long-time limit of (A.1), i.e.
(
lim utt,
(A.3)
20000
Hence, much of the theory for time- demendeennddnt PDEs can be adapted to Lime-independen problems by recognizing steady-state solutions as limiting cases.
A.2 Solution Operators for PDEs
Let us denote the solution to the time-dependent PDE (A.1) by
u(t.-) = S(a, t).
(A.4)
where S: 2 x (0, 7) - 2' the solution operator, mapping any initial datum un E 2 (and
parameter functions Q) to the solution u(t) time t.
17

---

We start by describing these discretizations for the lime-independent PDE (A.2). To this end, fix the i-th sample and let D (x D(O), for و < ( denote set of sampling points on the underlying domain D Observe that the underlying domain itself can be an input to the solution operator of (A.2). We assume access to the functions (c(*), (x,)) (x,)) and the corresponding discretized boundary values. Denoting these discretized inputs and outputs as Time-Shifted Operator. In many operator-learning strategies, it is useful to consider a time-shifted operator that predicts solutions at future time from a current snapshot. Specifically, define
st
x (0,T) RR
X,
such that
st (t',, c t, 7) St (u' c' 7)
a 4T
(A.5)
Here, u² = u(t, -) is the solution snapshot at time t, which now serves as an initial condition on the restricted time interval (t, T). Likewise, ct is the corresponding parameter snapshot at time
Steady-State Operator. For the lime-independen PDE (A.2), we define
S 2 - X
(A.6)
to be the analogous solution operator, such that ū = S(c) solves the boundary-value problem for any parameter/boundary data c. Although many operator-learning methods primarily focus on the time-dependent form S, the same ideas apply steady-state problems by treating ū limiting case.
Operator Learning Task (OLT), A central goal is to approximate these solution operators without repeatedly resorting expensive, high-fidelity numerical solvers. Formally, the OLT can be stated as:
Given a data distribution μ Prob(x*)) x for initial/boumdary conditions and parameters E Q), learn approximation S as S to the true solution operator S. That is, for any a ~ is, we want (1, a) to closely approximate u(t) for all i E [0, T). For time- indeeenddeendent problems, this goal changes accordingly learning S ~ S.
A.3 Discretizations.
In practice, we only have access to a discretized form of the data as the labelled data is generated either through experiments/obseevationss or numerical simulations. In both cases, we can only evaluate the inputs and outputs to the underlying solution operator at discrete points.
(4) ( (where a fc u))) and (6) Am respectively, the underlying learning task boils down to (1 approximating 8 from the discretized data-pairs (aAi): "Ai) Note that although the data is given in a discretized form, still require that our operator learning algorithm can provide values of the output function u at any query point E D.
For the dimeddependent PDE (A.1), in addition to the spatial discretization DAm = (2 DDO), (4) time-dependent operator learning task consists inputs (cf (z₂), u (æz)) and outputs u uax)) (1) ((0)) from which the space- and continnnsuuuus solution operator 8, has be learned at every query point x E D and time point t E [O.
for 1 ر v ((), we only have access to data at time snapshots ( [0, [0, Thus, the data to the
Summarizing, for both time-independent and time-dependent PDEs, the operator learning task amounts to approximating the underlying (space-time) continuous solution operators, given dis- cretized data-pairs.
B Details of GAOT Architecture
This appendix provides a detailed explanation the core components of the GAOT model architecture, including the choice of latent token grid, the Multiscale Attentional Graph Neural Operator (MAGNO) used the encoder and decoder, the geometry embedding mechanisms, and the transformer-basee processor.
18

---

B.1 Choice of Latent Grid
Given the input point cloud, denoted by Da above, the first step in our design is to select latent point cloud, consisting of points at which our spatial tokens are going to be specified. Here, we explore three distinct ways of choosing spatial tokens, each offering different trade-offs in terms of computational cost, geometric coverage, and ease of patching for efficient attention. Figure B. schematically illustrates these strategies.
(a) Strategy I
(b) Strategy II
(c) Strategy III
Figure B.1: Schematic illustration of the three tokenization strategies 2D domain. Blue points and red circles corresponding physical grid and latent token grid, respectively. (a) A structured stencil grid overlaying the domain. (b) Downsampled unstructured points used directly as tokens. (c) Projecting the 2D domain onto -iw-dimesiionll planes.
Structured Stencil Grid (Strategy I). In this approach, we overlay structured grid of tokens across Da For 2D domains, this may be uniform mesh of cells; for 3D domains, an analogous dense grid can be used. GINO [29] a good example for the use of such latent grids.
Advantages: This grid can be quite fine if needed, ensuring adequate coverage. Moreover, can group tokens into patches before feeding them into the transformer processor (see below), effectively reducing the token count (number of latent points). Our following experiments in SM Sec. F:2 suggest that patch size has a negligible effect on performance; hence, one can choose large patches to speed up training.
Limitations. The main drawback is that token count grows exponentially with the dimension; for 3D, the number of grid cells be prohibitively large. Also, if the input data lie on a low-dimensional manifold embedded Nonetheless, we find in practice that even empty tokens (those with no neighboring input points) can still contribute to better global encoding and improved convergence.
Da some tokens may remain underutilized.
Downsampled Unstructured Points (Strategy IT). This method directly downsamples the input unstructured point cloud and treat each sampled point as a token, RIGNO [36] and UPT [1] are typical methods based on this strategy. If the data are denser in some regions, naturally more tokens appear there.
Advantages.
This method avoids the pitfalls of having many tokens in empty regions, as might happen if the data indeed lie on lower-dimensional manifold. By adaptive sampling, it can allocate tokens more efficiently.
Limitations. Unstructured tokens are harder to patch effectively for attention mechanisms in the processor. In following experiment, observed that this strategy can be less effective than Strategy even when the domain indeed partially low-dimensional.
Projected Low- -ww-Dimnnsional Grid (Strategy III.. Inspired by certain 3D computer vision mod- els [9, 23], one can project the 3D domain (or higher-dimensional space) into diwer-dimmnsional representation and then place a structured stencil grid in the reduced coordinates for example, using triplane embeddings 3D [10].
Advantages. Such projection drastically reduces the token count in 3D, and avoids the purely low-dimensional manifold disadvantage of Strategy I. Moreover, one can still apply patching on the structured plane.
Limitations. Decomposing d-dimensional coordinates into disjoint projections (e.g., splitting x, y, axes) can introduce additional approximation errors. Some local neighborhood
19

---

information is inevitably lost during the projection. This trade-off can degrade the final accuracy compared to direct methods (Strategy I or II).
In this paper, we mainly adopt Strategy i.e. a structured stencil grid, for all experiments due to its robustness and simplicity. While using stencil grid indeed creates some empty tokens in low-density regions, we consistently observe fast convergence and strong generalization across various PDE datasets, see the ablation studies in SM Sec. F.2.
B.2 Multiscale Attentional Graph Neural Operator
Both the encoder and decoder in GAOT (see Figure 2 in the main text) employ the proposed Multiscale Attentional Graph Neural Operator (MAGNO). MAGNO is designed to augment classical Graph Neural Operators (GNOs) by incorporating multiscale information processing and attention- basedd weighting. A traditional GNO constructs a local graph for each query point (or token) by collecting all neighboring nodes within a specified radius, approximating (kerell) integral operator over this local neighborhood. Below, we first recap the standard single-scale GNO scheme, then extend it to a multiscale version, and finally incorporate attention mechanisms for adaptive weighting.
Recap of Single-Scale Local Integration (GNO Basis) For any point y the latent space D (in the encoder) or a query point in the original domain Da (in the decoder), GNO layer aims to aggregate information from its neighborhood via kernel integral. For the encoder, given input data a(x;)) on the original point cloud Da {æ the GNO transforms it into latent features (w).. The fundamental GNO computation is given by Eq. (3) from the main text:
nly
W. (y)
Σ agK(y, Tx, a(zx)))p(((())))
(B.1)
where the sum is over By points Fe in the original point cloud Da such that (y - vxl S r. K and are MLPs, a(xx) is the imput feature point 2g, and ag are given quadrature weights. This form can be seen as a discrete approximation of an integral operator:
K Ky, al('') (((())))))))
(B.2)
B.GDDD
where (y) is ball of radius centered at y.
Multiscale Neighborhood Construction The single-scale approach, while effective for capturing local interactions within a fixed radius r, may not efficiently perceive multiscale information crucial for many PDE problems. To address this, we introduce multiple radii. As described in the main text, we choose Im Smro. where To is a base radius and 8m are scale factors (m = 1...., m.) For each scale m, we gather points r& from the original point cloud DA within the ball B, (v) centered at E DD (for the encoder) with radius Tm. A GNO-like like local integration is then performed for each scale m, as shown in Eq. (4) from the main text:
o m KTm (y, (U, TK, ((((a)((((
(B.3)
Here, n. is the number of neighbors Ty within radius Tm The MLPs Km and yp can be scale-specific or share parameters across scales. This paper chooses the shared parameters across all scales.
1.2.1 Attentional Weighting in Local Integration (AGNO)
In the main text, we further propose an attention-based choice for the quadrature weights of as given by Eq. (5):
exp
W
"" y. W
O
(B.4)
exp(e
where W WT E axd (assuming original and latent coordinate dimension d, and attention dimension d are learnable query and key matrices. This mechanism allows the model to dynamically assign contribution weights each neighbor øy based on the relationship between y and æx. This forms the final form of our Attentional Graph Neural Operator or AGNO at each scale m.
20

---

B.2.2 Attentional Fusion of Multiscale Features
After computing the AGNO features W (y) for each scale (which are then fused with geometry embeddings, detailed in Sec. B.3, form @M (y)), we need to integrate this information from different scales. As described in the main text (Fig. and Eq. (6)), instead of simple summation or concatenation, we introduce a small MLP Vm to learn the relative contribution of each scale to the final encoded feature wy(y):
exp(vm(())
We (y)
Σ Bmly))" Bm
(B.5)
m Σ exp(um (1))
mm'=1
Here, 1m (y) is typically computed based on coordinates of y. Bm the attention weight for the m-th scale at point
The final output of the MAGNO encoder, (e(y). is thus feature representation that adaptively weights and fuses multiscale local information with attention mechanisms. The MAGNO in the decoder follows the exact same structure, with different inputs, outputs, and operating objects, as described in the Main Text.
B.3 Geometry Embeddings
While the Multiscale Attentional GNO already leverages geometric structure via local neighborhoods, one often needs to incorporate more explicit shape or domain information in practical PDE scenarios. For instance, when the geometry of the domain itself (e.g.. the shape of airfoil) plays critical role in the solution operator, coordinates alone may be insufficient to encode all the necessary geometric priors. Therefore, introduce geometry embeddings to enhance the model's geometric awareness. These embeddings work in tandem with the MAGNO encoder (and decoder), providing a rich geometric description for each token (latent point y) and its neighborhood various scales m.
Prior work on including geometric information in neural PDE solvers typically resorts to two major approaches: (i) appending geometry features directly into node/edge attributes [36], or (ii) using signed distance function (SDF) [29]. However, we argue that:
Simply merging geometry and physical features at the node level may entangle them prematurely, potentially hurting performance when the geometry is complex or when additional modalities (e.g. material properties) must be fused.
Computing SDF to represent geometry is often cumbersome, especially for unstructured datasets or when the boundary is only partially known. Each new shape would require re-computation, and the SDF values may be inaccurate the surface is not well-defined.
Instead, we advocate two more direct and flexible mechanisms for extracting geometric descriptors: local Statistical embedding and PointNet-based embedding, shown in Figure B.2.
Surteiial
Peaturs
MIP
Suztesical
Diflernnes
C2
MLP 06A
[C)
PointNet
10
Gronetrie Embeedding
Eanhelding
N
(b)
Figure B.2: Schematic of the Geometric Embedding for Statistical Embedding (a) and PointNet-based Embedding (b). LC denotes the lifting channels for MAGNO output.
Local Statistical Embedding The core idea is to extract statistical descriptors from the neighbor- hood Br.,, (y) (or Byn, (x) for the decoder) of original point cloud points I& (or latent points ye for the decoder) for each latent point (for the encoder) or query point x (for the decoder) at each scale m. Taking the encoder as an example, for latent point y D and scale m, its neighborhood is N.. Nm, (y) = {x E DA y F&l S vm) containing "y points. We compute the following statistics:
Number of Neighbors n. Mcasures local density around point y.
21

---

Average Distance D
avg (y):
"
avg (v)
2
(B.6)
Jmn
Describes the average spatial extent of the neighborhood. Distance Variance De (v):
Thy
Da (V)
Σ
D
(1))
(B.7)
71
Reflects the dispersion of points within the neighborhood. Neighbor Centroid Offset Vector A.:
Th
Tk
y
(B.8)
The vector from to the centroid of its neighbors Zk.-
PCA Features: These features aim capture the local shape anisotropy of the distribution of the ny neighbor points {ax} within the m-th scale ball Br.. (v). This is achieved by
performing PCA on the set of these neighbor coordinates {ra} Using the centroid of the neighbors Fabes,y coordinates is calculated as:
Σ k= D the x covariance matrix of the neighbor
12 (xx
C
7. nbrs.
abbss.y
(B.9)
If n = 0 (or too few points for meaningful covariance, e.g. my d), the covariance matrix C is treated as zero matrix, leading to zero eigenvalues. Otherwise, the d real eigenvalues of this symmetric, positive semi-definite covariance matrix, sorted in descending order (A1" 2 Ag
2 XT 0), are used the PCA features. These eigenvalues represent the variance the neighbor data along the principal component directions, thus describing the extent and orientation of the local point cloud cluster.
These statistical descriptors, computed for each scale m and each point y E DD are concatenated into a vector "yy normalized (e.g., to have zero mean and unit variance for each component), and then fed into MLP to yield the geometry embedding g (y) for that scale:
g"r (y) MLP gon (Normalize( 2.""))
(B.10)
This MLP goo is typically shared across all points and scales.
Point-Based Embedding As an alternative, we can train a PointNet-style network [40] to derive compact geometric descriptor from each token's 's neighborhood. Classical PointNet architectures typically include:
Input Transformer: aligns input points canonical space (optional),
Shared MLP: processes each point individually.
Symmetric Pooling: aggregates per-point features into a global descriptor, ensuring permu- tation invariance.
our PDE setting, we not necessarily need an input transformer; the local coordinates can directly serve as input features. We replace the typical max-pooling with mean-pooling to produce smoother local embeddings (though other pooling strategies are also possible). For a point y and scale m, we a
collect the relative coordinates of its neighbors { fed into a shared MLP (point-wise MLP);
FR
y These relative coordinates k=1
MLP, (o
(B.11)
22

---

Then, a symmetric pooling operation (e.g.. mean pooling or max pooling) aggregates these per point features into a global geometric feature:
h" (y) McanPool((hy))
Jk=1)
(B.12)
Rmm
This aggregated feature hm (y) can optionally be passed through another small MLP to produce the final geometry embedding g (v).
B.3.1 Integration of Geometry Embeddings in MAGNO
As depicted in Fig. of the main text and described in the MAGNO paragraph, in the MAGNO component of the encoder (or decoder), the geometry embedding is fused with the AGNO output at each scale. The specific workflow (for the encoder) is follows:
1. Scale-Specific AGNO Features: For latent point y and scale m, compute the AGNO output (y) (as described in Sec. B.2.1).
2. Scale-Specific Geometry Embedding: In parallel, using methods from Sec. B.3, compute the geometry embedding g (y) from the same neighborhood (v).
3. Feature Fusion: Concatenate the AGNO features T. (y) and the geometry embedding
gddh (v), and pass them through an MLP for fusion, yielding the scale-specific latent feature function Thth (y):
i (v) MLP fuse (lwi" (v) gg"(w)))
(B.13)
where denotes concatenation. And MLP
fuse is shared across scales.
4. Multiscale Aggregation:
Finally, as described in Sec. B.2.2, an attention mechanism is used to perform a weighted sum of the fused features { (w (v) (y) from all scales, yielding the final encoder output (y) (Eq. (B.5)).
Compared to merging geometry and PDE features at the node level before any operator updates, this per-scale integration offers several benefits:
Scale-Adapted Geometry. Each scale has a correspondingly sized neighborhood, allowing the geometric embedding to reflect local shape details at the appropriate radius. Small radii capture fine-graines features (e.g. sharp corners), while large radii convey coarse global context.
Modular Flexibility. Both MAGNO and Geometric Embeddings act as distinct modules. One can upgrade either component (e.g. adopting more customized local aggregator geometry encoder) without changing the overall pipeline.
Unified Per-Token Fusion. The final aggregated feature w. (y( collects information from all relevant scales and from geometric descriptors, leading to richer token representation. This is particularly advantageous in settings with complex boundaries (e.g. airfoils, porous media) where multiple length scales and shape cues matter.
This design preserves the encode- encode-process-dee process-decode philosophy: each token gains geometry-aware. multiscale PDE features during the encoder stage, facilitating global attention and final decoding later in the pipeline.
B.4 Processor
After constructing geometry aware tokens, we employ a Transformer- baased processor to enable global message passing among all tokens. Depending on the chosen tokenization strategy (SM B. we can choose the following strategies, respectively:
Regular Grid (Strategy I or III): If the latent points {ye E DD lie on a regular grid (e.g., via a structured stencil or a projected owwddimensional regular grid), we adopt strategy similar to vision transformers (ViTs) [12]. The latent points are grouped into non-overlapping "patches." All token features (y) within each patch are flattened and linearly projected into single patch token embedding. These patch tokens then serve as the input sequence to the Transformer.
23

---

Randomly Downsampled Points (Strategy II): If the latent points {ze) are randomly down- sampled from the original point cloud Da, they lack regular grid structure. In this case, there is no obvious "patching" method, and each latent token (ye) directly serves as an element in the Transformer's input sequence.
Positional Encoding Transformers themselves are permutation invariint and do not inherently process sequential order or spatial position. Thus, positional information must be injected. In GAOT, the Relative Positional Embeddings (RoPE) [46]. which is method that integrates relative positional information directly into the self-attention mechanism. It achieves this by applying rotations, dependent their relative positions, to the Query and Key vectors. This has shown strong performance in many Transformer models.
Transformer Block Structure For the transformer Blocks, we adopt an RMS norm RMSNorm())
at the beginning of attention and feedforward layers:
RMSNorm(x) RMSNorm(x),
x
a
(B.14)
V (x)
where a is a learned scaling parameter. This approach is akin LayerNorm but uses the root mean square of feature magnitudes rather than computing mean-and-varian separately. This prenorm
design helps stabilize the gradient flow compared to the conventions [47]. Each block has the structure,
Zattn =
X + MultiHeadAAtn (RM)II/eaA (RMSNorm(X)), Zin = Zatta + FFN (FMMSNONom (Zattn)
(B.15)
Furthermore, we use Group Query and Flash Attention in the code for efficient multi-head self- attention.
Long-Range Skip Connections In addition to the intra-block residual connections, we also intro- duce long-range skip connections across multiple Transformer blocks, as suggested in works like [5]. For instance, the Transformer blocks can be divided into an earlier part and later part, and layers can be symmetrically connected (c.g., the first with the last, the second with the accond-to-last, etc.), allowing later blocks to directly receive information from earlier blocks, further improving information flow.
By stacking these blocks, the Transformer processor learns complex global dependencies among tokens, transforming the locally geometry-aware tokens wr.ws) from the encoder into processed tokens (V:) that incorporate richer contextual information. These processed tokens are then converted by the MAGNO decoder to the desired approximation of the output of the underlying solution operator (see Main Text and Fig. 2).
B.5 Graph Building Tricks
For 2D datasets, we construct neighborhoods for each latent token using a fixed-radius rule, following the Ref. [29]. This works well because local density variations moderate and the total edge count remains manageable. However, this strategy is not scalable for large 3D industrial datasets, where meshes could be highly adaptive. A single global radius will lead sharp trade-off: if is large, the number of edges explodes in dense regions and quickly exceeds GPU memory; if r is small, the Overlap of the local neighborhoods fails cover the whole physical domain and induces information loss. To simultancously control edge count and guarantee the whole domain coverage, we introduce a bidirectional graph-building strategy that merges two complementary graphs:
1. Radius graph centered at latent tokens. For each latent token ye, we connect all physical points z, within a radius r, producing edges that encode local aggregation around latent tokens. This yields good locality, but may not be able to cover the physical regions if is too small.
2. KNN graph centered at physical points. For each physical grid point (node) Øjs we connect it to its k nearest latent tokens, adding at least one attachment per physical node. This forms a lightweight backbone that ensures every physical location is represented the latent space, even where radius neighborhoods are empty.
24

---

Graph Butscing DEEnSRRY
Ses Rnoader Scrategy
Ses beaster Boraregy
KNN
Radrac lave creamed plyeeal witten
Bidemctonn? KNNN modic
KNN
Radtac plytial comrred wilten
Bldtmertons KKN + cuBic bart plyp.
cad plosical poin
cadi pla pcsk.
mocmm iance payo
nome Ioora
Figure B.3: Neighbor strategy dispatcher used GAOT. For 2D datasets, a radius- based graph is adopted; for large 3D datasets, the bidirectional strategy (radius KNN) is used to ensure domain coverage without edge explosion.
We then merge the two edge sets (remove the repeated pairs). The resulting bidirectional graph only ensures all the physical points are considered, but control the total edge count through the tunable hyperparameter & (typically k E [1, 4)) and radius r. In practice, we use the encoder direction physical latent and the decoder direction latent + physical; the same bidirectional recipe applies to both by swapping source/target roles, shown in Fig. B.3. Although the bidirectional build is more expensive than a pure-radius build, we precompute graphs offline and cache them, so the extra cost does not affect training throughput.
On all 2D benchmarks, retain the pure-radius construction. On 3D industrial-scale datasets, switch to the bidirectional construction to avoid radius-induced edge explosion while preserving coverage. To further control memory and improve robustness, also apply edge masking during training.
B.6 Training Details.
This section discusses the details of how the GAOT models were trained, including the loss functions, data normalization procedures, general training hyperparameters, and default model configurations. In our experiments, we address both time-independent and time-dependent PDEs. The application of GAOT to time-independent PDEs straightforward. For time-dependent PDEs, we employ three different time-stepping methods and all2all training [22]. discussed in the main text. More details on these methods can be found in [36].
B.6.1 Loss Function
The loss function used for training GAOT is the Mean Squared Error (MSE), computed between the model's s final predictions and the true physical quantities. For a set of N, samples and Np spatial points, the loss is:
N,
F
LMSE
())())((z)) Utrno, (x;))1
(B.
N.N,
where (a())(<r)))
) is the model's prediction for sample at point Ty, and ( is the correspond- ing ground truth. The exact form of 8a(-) depends on whether the problem is time-independent or time-dependent.
Time-Inde Inneeenndeendeett PDEs. For time-independent PDEs, given an input a(z;) (e.g.. boundary conditions, coefficients c), GAOT directly predicts the solution u(x,)). Thus, 8o (x) is the direct output of the GAOT architecture, and the MSE loss is computed between Sp(a) (o(a)(r) (I,) and the true steady-state solution utrue(3) (I₂) of PDE (A.2).
To learn the solution operator for time-dependent PDEs, GAOT used to update the solution forward in time. Given the solution u(t) at time t and coefficients e (forming the augmented input (t) = (c, u()))), the model predicts the solution at t + т. The GAOT architecture produces an output Sz t, a(t). The final prediction for u(t + r), denoted Sz 7, a(t)), is constructed using general stee-st strategy as per Eq. (7) from the main text, see also [36]:
T'iee-Depnndent PDEs.
Sp(t, yu(t) + (2,
(B.17)
25

---

The MSE loss is then computed between this Sy(t, a(t)) and the true solution Meuue + r). The choice of parameters (7, 6) determines the time-stepping strategy and what the network output Sg
effectively learns:
Output Stepping (? 0, 1):
The final prediction is Sp 7, a(t)) = (,,
The network output 8g directly learns approximate u(t ((++T).
Residual Stepping (r 1,8 1): The final prediction St a(()) u(t)
Sa t, T, at)). The network output Sa learns to approximate the residual, u(t + r) -tt).
Time-Derivative Stepping (? = 1, 6 r): The final prediction is So(t, 7, a())) = u(t)
S0(x,, T,a(t)). The network output S learns to approximate the time-derivative, (u(t + r) u(t))/TT
GAOT offers the flexibility to use any of these strategies. A detailed ablation of their comparative performance is described in SM Sec. F.
B.6.2 Data Normalization
Data normalization applied to stabilize training. We typically use Z-score normalization, where for a quantity X, its normalized version X is (X - px)//xx The mean μax and standard deviation ox are computed over the training dataset.
Iimee-nneependent PDEs. For input features a(zz) and output solution fields u(z;), normalization parameters are computed across all samples and spatial points in the training set for each channel independently. The model is trained on normalized inputs to predict normalized outputs.
Time-Dependent PDEs. The input u(t) is normalized using its global mean and standard deviation computed over all time steps and samples in the training set. The normalization of the target for the network output (1, L, r, a()) depends on the chosen time-stepping strategy, as Sy learns a different physical quantity in each case:
Output Stepping: The network 8g aims to predict u(t + r). Thus, the ground truth values u(l + r) are normalized, and S, is trained to predict these normalized values. Statistics and a, computed from all values u(t') in the training set. The normalized target for Sq is ā(t +) = (u(t + + - pu)/ou.
Residual Stepping: The network 8g aims to predict the residual R(I, r) = ult + u(t). Thus, these true residual values are computed from the training data, and their statistics (ARR an) are used for normalization. The normalized target for So is R(t,
(R(t, r) pR)/OR-
Time-Derivative Stepping: The network Sy aims to predict the time-derivative (tt, (u(t r) - n(t))/r. These true derivative values are computed, and their statistics (UD, d)) are used for normalization. The normalized target for Sg is D(t. r) = (D(t,)) - -HD)/OD.
Time 1 and lead-time т inputs are also typically scaled or normalized. Further details on these normalizations can be found in [36].
B.6.3 General Training Setup
This section provides an overview of our training hyperparameters. Unless otherwise noted, all experiments follow these settings. Table B. summarizes the primary hyperparameters and training schedules. In particular, we distinguish between deeeddepndd and time- inne-independent PDE tasks in terms of epoch count and highlight the differences in hardware usage for the industrial 3D dataset. All models except industrial 3D dataset run on single GeForce RTX 4090 GPU. For the DrivAerNett+ dataset, we use GeForce A100 GPUs in data parallel mode for epochs, and each GPU holds a batch size of 1. For DrivA and NASA-CRM dataset, we use GeForce A 100 GPUs for 200 epochs. For the scheduler, we warm up to mitigate instability at early epochs, then adopt cosine schedule for gradual decay, and finalize with step-based drop for fine-tuning the last epoch range.
26

---

Table B.1: Key training hyperparameters and schedulers used for all models, unless otherwise specified.
Hardware
Single-GPU: All models for 2D dataset are trained on a single GeForce RTX 4090 with batch size = 64.
Single-GPU: Models for DrivAerML and NASA-CRM are
trained on single GeForce A100 GPUs with batch size = 1.
Four-GPU: For the DrivaerNet++ dataset, we use four GeForce A100 GPUs, each with batch size = 1.
Optimizer
Algorithm: AdamW
Weight Decay: X 10-5
Epochs
21) Time-Deppndent PDEs: 500 epochs
2D Time-Independent PDEs: 1000 epochs
DrivAerNet++: 50 epochs.
3D NASA-CRM and DrivAerML: 200 epochs.
Learning Rate Scheduler
Warmup (first 10% epochs): LR increases linearly from 8x 3x10-+ 10-3
Cosine Decay (next 85% epochs): LR decays from x 10-3
1 x 10-4
StepLR (final 5% epochs): LR drops from 1 1 to 5 10-5
B.6.4 GAOT Model Configuration
Table B.2 outlines the default configuration of the GAOT framework. This includes the MAGNO used in both the encoder and decoder stages, as well as the Transformer-basee global processor. MAGNO converts node features into geometry-aware tokens (encoder) and reconstructs continuous fields (decoder). By default, the coordinates will be rescaled in the domain [-1, and we use a single aggregation radius 0.033 for adequate coverage. If multiscale enabled, we adopt radii (0.022, 0.033, 0.044). The default geometric embedding method is local Statistical Embedding (e.g. as SM Sec. B.3), and will typically be implemented for unstructured datasets. The Transformer processes geometry-awen aware tokens globally via multi-hear self-attention. We set the hidden dimension to 256 with a 1024-dim feed-forver fordforwrdd layer, residual connections, RMSNorm, and RoPE for positional embeddings. By adjusting the patch size and the number of tokens, can trade off computational cost and model resolution, discussed SM Sec. F.2.
The configuration slightly differs for the industrial 3D datasets accommodate large-scale adaptive meshes. We employ the following number of latent tokens for each dataset:
NASA-CRM: [64, 64, 32),
DrivAerML: [64, 32, 32],
DrivAerNet++: [64, 64, 32).
In these setups, the graph construction follows the bidirectional strategy strater described in Appendix B.5, using a single-scale radius of 0.033 and KNN size of k 1. The Transformer processor adopts a patch size of and consists of 10 layers. All other hyperparameters remain consistent with the default configuration in Table B.2.
B.7 Inference
When predicting solutions for time-independent PDEs, simply feed the input parameters (e.g. boundary conditions, coefficients, or geometric shape) into our learned operator Sa(a) and obtain the steady-state output u(x) directly. However, for time-dependent PDEs, there are two different strategies for forecasting the solution at a future time, using the learned one-step advancement operator Sa(1, r, a(l)) which, as defined in Eq. (B. (B.17), takes the current time I, lead-time т, and the augmented input a(l) = (c, u()) to predict the solution + т. The two main inference strategies direct inference and autoregressive inference.
27

---

Table B.2: Default architectural hyperparameters for GAOT.
Abbreviation
Default Value
Description
MAGNO (Encoder / Decoder)
PJC
256 164, 64, 64]
Dimensionality for MAGNO's internal hidden layers.
ENC-MLP DEC-MLP
Hidden layers of the encoder MLP in MAGNO.
[64, 64] 32
Hidden layers of the decoder MLP in MAGNO. Output/Lifting channels after MAGNO (both encoder and de- coder).
LC
TS
I [64, 64] 0.033
Tokenization Strategy I.
NT
Number of tokens for Strategy (e.g., a 64 x 64 stencil grid).
GR
Aggregation radius (single-scale for every token. If multiscale: {0.022, 0.033, 0.044].
GeoEmb EM
statistical 0.3
Geometric Embedding for the encoder and decoder.
Edge masking ratio for the MAGNO, used for 3D drivaernet++ dataset.
Transformer
PS
2 RMSNorm
Default patch size for token grouping if Strategy I or III is used. Normalization used in attention and MLP layers. Pre-norm configuration.
Norm
PE
RoPE
Positional embedding used in the Transformer. Rotary positional embeddings.
RES-CON
True
Residual connections between transformer blocks.
TL
5
Number of Transformer blocks.
THS
256
Hidden dimension per self-attentio block. Number of attention heads.
HEAD
8
Dropout
0.2
Dropout ratio in the attention module.
FFN
1024
4x hidden size (THS) for the feedforward layer.
Direct Inference (DR) Recall from the main text that our learned operator Sp takes the lead-time T as an explicit input, allowing for predictions over variable time steps. Given a snapshot of the solution u((n) (which is part of a(t,)) )), the network can directly predict the solution at any later time tn + Tsarget. up to a maximum trained horizon fmax. by evaluating Sg Ttarget, a(t,)). ). Hence, for each possible time increment Ttarget = k At (where At is a base time step in the dataset, and 1 S. k S. kmax), we can produce the model's s estimate u(tn + Tearget) from u(n,) in a single step, without iterating through intermediate time steps. Concretely, if our dataset is discretized at times 26 = { fo, f1,...,A, IN ). we can directly evaluate So (fa, Ttarget, atrr)) for various Starget values originating from any t, E D2A This provides a sequence of direct predictions at each possible time offset Ttarget from any initial time tn
Autoregressive Inference (AR) While direct inference estimates the solution single future time, an alternative is to iterate the operator in multiple, typically smaller, sub-steps to reach the final time. This approach is called autoregressive (AR) inference. Formally, given an initial snapshot u(lo), we repeatedly apply the learned operator 8n with a chosen fixed time increment for each step, Al ALAR to advance the solution:
u((x+1) S0k, ALAR, a(lx))
(B.18)
where 4++1 = tx + AR. and a(tx) = (c, u(tx)) uses the solution u(ts) from the previous step (or the initial condition if k 0). This process is repeated until the desired final time fimal is reached.
We examine two types of autoregressive step sizes in our experiments:
AR-2:
Use an autoregressive time increment of ALar = 2 (assuming time units are consistent with the dataset). Starting from u(lo). we compute u(4o + 2) = So 2, a(lo)). Then, using u44o + 2), we compute u(to 4) S 2,2, a(to + 2)), and so on, up to tu In total, we perform 7 consecutive evaluations of Sg.
28

---

AR-4: Use an autoregressive time increment of AtAR = 4. In this scenario, we predict u(to + 4) Sy(tn, 4, a(to)), then from u(tn + 4), u(to + 8) So + 4, 4, al + 4)). and so on, eventually reaching 414 in just 4 iterations (assuming to = 0 and final 16 for this example, or 114 is the target after some steps).
Generally, the choice of the AR step size AR is flexible. One could select any valid ALaR S Tmax (where Tmax the maximum lead-time the model was reliably trained for in a single step) at each sub-step. Note that using fewer, larger time steps (c.g., AR 4) can reduce computational cost but potentially compounds prediction errors more quickly if the operator Sg is less accurate for larger single-step lead-times. Conversely, smaller increments (e.g. AtAn = 2) tend to accumulate errors more gradually but require more iterations (and thus more computation) to reach the final time. Details can be found in [22, 36].
C Baselines
For the time-dependent benchmarks (including those on unstructured grids detailed in Table and regular grids in Table E.5, the corresponding baseline results are primarily obtained from the work by [36]. These baseline models include RIGNO-12, RIGNO-18, CNO, scOT, FNO, GeoFNO, FNO DSE, and GINO. For further details these methods, please refer to the paper [36]. In this work, we have additionally included three more recent models for a comprehensive comparison: Transolver [49], GNOT [20], and UPT [1]. Brief descriptions of these newly added models and the specific hyperparameters adopted our experiments are provided below.
C.1 UPT
Universal Physics Transformers (UPT) [1] form neural-operator framework that fits into the canonical encode-prooe process decode pipeline:
u = AoAEE
(C.1)
where & (Encoder) compresses input pomnts- from arbitrary Bulerian mesh Lagrangian particle cloud-into a fixed set of Matem tokens. first embeds the features and coordinates through radius-graph message-passing layer that aggregates information into n, supernodes, and finally employs transformer and perceiver pooling blocks to obtain the latent representation E "R Xh
A (Approximator) is stack of transformer blocks that advances the latent state in time, A: 24
244 At, enabling fast latent roll-outs without repeatedly decoding to the spatial domain.
D (Decoder) is Perceiver-style cross aross-attention module that evaluates the latent field at any set of query positions (y: 1/' yielding ur+ (x = D( U:) with (MMate)) complexity independent
of k.'
In the setting of time-independent problems, we bypass the latent roll-out stage. and adopt lightweight configuration our experiments. Specifically, we use latent tokens 64 and em- bedding dimensions = 64, which results in a model size of 0.74M. A large variant with 256 latent tokens and 192 embedding dimension as setup [1] was found suffer from optimization difficulties and was not adopted in our baseline results. The same optimizer setup as GAOT is used here.
Considered Hy pepraaarastes
29

---

Architecture
Trainable parameters
0.74M
Number of supernodes n
2048
Radius for message passing Embedding (feature) channels Encoder transformer blocks
0.033
64
4
Encoder attention heads
Latent tokens Vatatent Latent dimension h
64
64
Approximator transformer blocks
Approximator attention heads
4
Decoder attention heads
Training
Optimizer
AdamW
Scheduler
same as in B.2
Initial learning rate
1 10-3
Weight decay
10-5
Number of epochs
500
Batch size
64
C.2 Transolver
Transolver [49] a transformer-basee operator learning model designed for PDEs on unstructured grids. It follows an encode proeede-rocssdeeodee paradigm by stacking multiple Transolver blocks. The core of each block is the Physics-Attentinn mechanism.
Given input features Xphys E NxCC for N mesh points:
1. Encoding to Tokens: First, for each mesh point feature 2; E Xphys M slice weights W/ RI are learned, typically via projection followed by Softmax function: w, Softmax (Pronc((rre))((:)))........)) These weights determine the assignment of mesh points to learnable "slices". The j-th physics-awart token 2j gIxc is then encoded by weighted aggregation of all mesh point features, using the slice weights:
ΣN N
Wi
2j =
(C.2)
ΣN Wif
This results in M tokens 2 = ( E MXC
2. Token Processing: These M tokens Z are processed by a standard attention mechanism (e.g., multi-head self-attention) capture correlations between different physical states represented by the tokens:
7proe Attention (7)
(C.3)
The processed tokens are 7proc [=] RMC.
3. Decoding to Physical Grid (Deslicing): The updated token features Z are then broadcast back and recomposed onto the N physical mesh points using the original slice weights we
M
Wi.j2
(C.4)
This yields the output features for the Physics-Atteninn Attention block. X/hys = { E NXC.
full Transolver layer typically incorporates this Physics-Attention mechanism within standard Transformer layer structure, including Layer Normalization and Feed-Forward Networks.
30

---

While the Physics-Attention mechanism itself is designed to have computational complexity linear with respect to the number of mesh points, it is important note that the slicing (Eq. C.2) and deslicing (Fq. C.4) operations, which involve all N points, are performed within each of the T. Transolver layers. This repeated mapping can lead significant computational costs and memory overhead, especially for large N. This contrasts with architectures like GAOT and UPT, which perform the encoding to a latent space and decoding from it only once, with intermediate processing happening entirely in the latent token domain. In our experiments with üme-independen partial differential equations, we followed the settings from the original Transolver paper [49];
Considered Hyperparameters
Architecture
Trainable parameters
3,85 M
Hidden channels
256
Attention heads
Number of Layers
MLP ratio
number of slice
32
Training
Optimizer
AdamW
Scheduler
same as in B.2
Initial learning rate
1 1--3
Weight decay
10-5
Number of epochs
500
Batch size
20
C.3 GNOT
General Neural Operator Transformer (GNOT) [20] is a Transformer-basee based framework designed for operator learning, particularly addressing challenges such irregular meshes, multiple heterogeneous input functions, and multiscale problems. Its overall architecture can be represented as:
g
F (B) cE
(C.5)
processor
where E is the encoder, B represents GNOT Transformer block (repeated L times), and F is the output decoder.
1. Encoder (E): The encoder maps diverse input sources (geometry, fields, parameters, edges) to embeddings using dedicated MLPs. This yields query embeddings Q E N for target points and a set of m conditional embeddings (Y(()) E Nexd Jm from other input functions.
2. GNOT Transformer Block (B): Each block refines query embeddings Q using conditional embeddings (!) via:
Heterogeneous Normalized linear using separate MLPs for keys/values from different ((), followed by normalization and averaging.
(HNA): Fuses Q with each (()
1 T.
Levos Q
+ NormLiinearCrossAtn00. y()
(C.6)
Te t=l
Normalized Self-Attention: Applies normalized linear self-attention to & for further refinement.
Quet NommLinearSSlfftttt
(C.7)
31

---

Geometric Gating FFN: A Mixture-of -xpeertss (MoE) FFN where expert FFNs (E₂) are weighted by Ps (. These weights are predicted by gating network Gi-) using query point coordinates Zeoond enabling soft domain decomposition for multiscale problems.
K
FFN
Σ km
Pk (Feoor) (x),,, Pk (Fooad)) = Softmax(Gx (C.8)
These components, with Layer Normalization and residual connections, form the block.
3. Decoder (F): After L blocks, a final decoder (typically an MLP) maps processed query features to the output solution.
Considered Hyperparameters
Architecture
Trainable parameters
4.87 M
Hidden channels
128
Attention heads
Number of Layers
8
MLP ratio
Training
Optimizer
AdamW
Scheduler
same as in B.2
Initial learning rate
1 10-3
Weight decay
10-5
Number of epochs
500
Batch size
20
D Datasets
In this work, we test GAOT on 28 benchmarks for both time-independent and time-dependent PDEs of various types, ranging from regular grids to random point clouds to highly unstructured adapted grids. The time-dependent and Poisson-
datasets are sourced from [22] and [36], respectively. The static elasticity dataset is from [26]. The Driv AeiNe+++ and DrivAerML datasets are taken from [14] and [3], respectively, where the objective is to predict the surface pressure and wall shear stress. The NASA-CRM dataset is obtained from [7], where we similarly predict the pressure field and surface friction coefficient on the NASA Common Research Model (CRM) surface. For 3D car and wing aerodynamics, the full-resolution surface field is directly predicted without any downsampling. Furthermore, we have also generated five additional challenging datasets: a Poisson-C-Sine dataset exhibiting multiscale properties, and four datasets for compressible fluid dynamics with highly unstructured adapted grids. Detailed information regarding these datasets can be found in the Tab D.I.
focus on the following PDE Types simulated for various initial/boundarry conditions and domain geometries:
Hyper-Elastic Equation (HEE)
puu u
v 0,
(D.1)
012
where 0² is the mass density, is the displacement vector, and is the stress tensor. A constitutive model links the strain tensor to the stress tensor. The material is the incompressible Rivlin-Saunders
type, characterized by Bw((c) with w(e) Ci(li - 3) C2(12 - 3).
de
32

---

Table D.I: Overview of the datasets used in this work. Datasets listed above the line are time- independent, while those below are time deneddeendent. Geometry variation (GeoVan describes whether every data sample in dataset has a different geometry. Characteristics briefly describe each dataset's s geometry or PDE setup. The PDE Type column indicates the corresponding class. Visual- ization (Vis.) provides references visual examples; for time-dependent datasets, this may include visualizations for both unstructured partially ones and original regular grid ones. Datascts marked with are newly proposed in this work.
Abbreviation
Geo Var
Characteristic
PDE Type
Vis.
Cussoon-C-Sine"
F
Circular domain with sines f
PE
G.1
Poisson-Gauss
F
Gaussian source
PE
G.2
Elasticity
T
Hole boundary distance
HEE
G.3
NACA0012'
T
Flow past NACA0012 airfoil
CE
G.4
NACA2412*
T
Flow past NACA2412 airfoil
CE
G.5
RAE2822*
T
Flow past RAE2822 airfoil
CE
G.6
Mluff-Body"
T
Flow past Surface pressure
CE
G.7
DrivAenNet++tpp)
T
INS
G.8
DrivAerNet++wws)
T
Surface wall shear stress
INS
G.9
DrivAerML(p)
T
Surface pressure coefficient
INS
G.10
DrivAerML
T
Surface wall shear stress
INS
G.11
NASA-CRM (p)
T
Surface pressure
INS
G.12
NASA-CRM (sfc)
T
Surface friction coefficient
INS
G.13
NS-Gauss
F
Gaussian vorticity IC
INS
G.14, G.22
NS-PwC
F
Piecewise const. IC
INS
G.15, G.23
NS-SL
F
Shear layer IC
INS
G.16, G.24
NS-SVS
F
Sinusoidal vortex sheet IC
INS
G.17, G.25
CE-Gauss
F
Gaussian vorticity IC
CE
G.18, G.26
CE-RP
F
4-quadrant RP
CE
G.19, G.27
Wave-Layer
F
Layered wave medium
WE
G.20, G.28
Wave-C-Sines
F
Circular domain with sines IC
WF
G.21
Poisson Equation (PE)
Au f. in (0,
(D.2)
with homogeneous Dirichlet boundary conditions. The dataset related to the Poisson equation either sinusoidal or Gaussian-like source terms on square or circular domains (see Table D.
Incompressible Navier-Stokes (INS)
V = 0,
where is the velocity field, is the pressure, and viscosity is v. It assumes periodic boundary conditions and samples various initial conditions (e.g., Gaussian, piecewise-constant, sinusoidal vortex sheets).
Compressible Euler (CE)
0,u + V.F
0,
(p, | -
(D.3)
with 7 = 1.4. Showing in [22], it imposes periodic boundary conditions and ignore gravity effects. Data are generated using random initial/boundary conditions such as Gaussian or Riemann problem (RP) setups.
33

---

Wave Equation (WE)
0r g),,)) v2u 0.
(D.4)
with spatially varying propagation speed c(a,, y) in an inhomogeneous medium. The dataset employs absorbing or homogeneous Dirichlet boundaries. Initial conditions ge.g,, sinusoidal or layered) are drawn from parameterized distributions.
All time-dependent problems are numerically integrated up to T (except for the Wave- C-Sines where T=0.005). collected (up to) N; = 21 uniform snapshots per sample at E {0. 2, 4, 6, 8, 10, 12, 14). For time-dependent PDE, as mentioned before, we use the same all2all training strategy proposed in Poseidon [22]. This means that each trajectory can generate 28 pairs for training.
D.1
Poisson-C-Sines
This dataset contains solutions to the two-dimensional Poisson equation with a circular domain. The Poisson equation is fundamental linear elliptic partial differential equation (PDE) given by Eq. D.2. Unlike standard benchmarks that map point-wise evaluations of the source term to the solution, this dataset defines the operator learning task within the context of the finite element discretization. Specifically, we aim to learn the mapping from the projection coefficients of the source term f onto the finite element basis functions to the nodal solution values u.
The continuous source term is constructed via a spectral expansion:
f(z,8)
π K c-=1
Σ
Oij (12 + +2) sin(mix) sin(Tjy), v(r.y) D, (D.5)
where r = -055 and K = 16. The coefficients ai are sampled i.i.d. uniformly from [-1, 1]. To
generate the input data, this source term is projected onto the finite element test space spanned by the nodal basis functions {ox}. The input to the model the resulting load vector b, where each entry corresponds to the integral b% Jo f (z. y)ox(2) y)d.. Consequently, the operator learning task is detined as G b + u, representing the inverse of the discretized stiffness operator. The mesh is generated using the Delaunay algorithm with 16,431 points and 22,441 elements, and zero Dirichlet boundary conditions are enforced during the assembly.
D.2 Compressible Flow Past Airfoils & Bluff Bodies
classical benchmark for compressible flow physics used for testing the accuracy of neural operators and PDE foundation models is the case of flow past airfoils [22, 26). The datasets used in these papers are limited to transonic flow past perturbations of single airfoil. To capture a broader range of rich flow phenomena, it is essential explore the parameter space spanned by the Mach number Ma, angle of attack a and the shape function. To address this issue, this new dataset introduces samples comprising range of flow phenomena from subsonic supersonic flow for varying angles of attack across classical airfoils and various bluff body geometries. The steady-state compressible Euler equations govern the flow phenomena in this dataset. The equations have been solved using the finite-volume EULER solver of the open-source software SU2 [13] on an unstructured grid generated (JST)
by Delaundo [37]. Convective flux discretization is done using the Jameson- Sameson-Schmid- scheme that is designed especially for achieving quick convergence to steady-state solutions of the compressible Fuler equations. Figure D.1 represents an O-type unstructured mesh generated using Delaundo for the RAF2822 airfoil. Similar O-type unstructured meshes have been generated for all airfoils and bluff- bodies considered. The free-stream pressure and temperature conditions for all simulations in this dataset are Poo 1 atm and T 288.15 K.
D.2.1 Airfoils
Flow past airfoils considered for 0.5 < Ma < 1.4,0.5 Sas 5.0 and for 500 unique perturbations applied to shape functions of the NACA2412, NACA0012, and RAE2822 airfoils. Anisotropic adaptive mesh refinement for highly accurate shock resolution (oblique and bow shocks) is performed using INRIA's py AMG library coupled with SU2 [31]. The anisotropic mesh refinement done using a Mach sensor that generates refined meshes based on the simulation on coarse grid such in Figure D.I. The final simulations are then performed by the SU2 EULER solver on the
34

---

Figure D.1: O-type unstructured mesh RAE2822 airfoil
new refined mesh. Figure D.2 represents the highly unstructured adapted grids for transonic and supersonic flow past the RAE2822 airfoil.
We consider the reference airfoil shapes with the upper and lower surface coordinates located at (z, (E)) and (x. (E)) where = et
x c the chord length. We use the Class Function/Shape Function Transformation (CST) Method [25] for parameterizing the airfoil surfaces in terms of
class function C(E) and shape functions S (E), using an in-house MATLAB code. The airfoil
upper surface function v/ v)((() (E) and lower surface function (E) parametrized follows:
(E) C(E)S
v) C(EsS (E)
(D.6)
where the class function for airfoils is given as:
C( 55(1 -
(D.7)
and the upper and lower surface shape functions are respectively
n!
7
n!
(S) = Σ A il(n i)!
5t - g)- st Σ
ilnn - i)! 5²(1 - g)=
(D.8)
n! The polynomials Si, 2(n oy (1 =)- associated with the coefficients (A,, B;) are Bernstein polynomials and is chosen. The parameters (A,, B,) directly influence key airfoil design variables such as the leading edge radius, trailing edge boattail angle, maximum airfoil thickness and
Figure D.2: Adaptively refined meshes for flow past the RAE2822 airfoil at 2.0 and different
Ma 0.8 (left), 1.0 (center), 1.4 (right).
35

---

maximum thickness location. The parameters (Ao. Bb) are linked to the leading edge radius RLE as follows.
RLE
Ag = - - V 2r, 7
(D.9)
and the parameters (An, B,) are linked the upper boattail angle By and lower boattail angle 3
A, = (St). B, = tan
(D.10)
To generate perturbed variations of the airfoils, minor random perturbations made the CST parameters (A,, B,) keeping in mind the constraints Ao = - -B,, A; B, and By > PL. We have randomly sampled 5384 solutions from our dataset for each classical airfoil shape with train/validation/tess split of 5000/128/256. For each data, we sub-samples 8000 points for training, validation and testing.
D.2.2 Bluff-Body
Flow past bluff-bodies is considered at 0.3 < Ma S 1.3, aa 15.0 for a wide variety of simple bludf-body geometries. Steady-state solutions for the compressible Euler equation for flow past bluff- bodies may not exist or are often unstable making difficult to attain convergence. This bluff-body aerodynamics dataset comprises of samples that at pseudo-steady state in a large finite-time limit. Figure E.6a describes all the bluff body geometries taken into consideration in this dataset.
We sample 4384 solutions from our dataset with a train//alidattionneest split of 4000/128/256 for all bluff-body geometries used for "Training and Testing". For each data, sub-sample 14000 points for training, validation and testing.
E Additional Results
E.1 Asymptotic Complexity
Table E.1 summarizes the asymptotic complexities of the compared models in Fig. In practice, the number of edges E is typically 3- 5x the number of nodes N. For consistency, we denote the number of latent tokens by T across all models; however, ViT-style patchifying in GAOT effectively reduces T and thereby improves scalability. Furthermore, different models employ distinct hidden dimensions C' for their edge update functions, which introduces further variation the total computational cost.
Table E.1: Asymptotic complexity. N: number of nodes, E: number of edges, T: latent tokens, C: hidden channels, p. patch size, L: number of layers/blocks.
Model
Total Complexity
GAOT
O + I(T/D)3C)
GINO
O + L.T
RIGNO
O(LEC) +
Transolver
The main computational burden in graph-based models arises from message passing, and memory usage dominated by storing edge activations, scaling linearly with both E and C. GAOT, RIGNO, and GINO compress information into latent tokens and then perform L layers of latent processing. In contrast, Transolver performs compression and decoding within each block, operating directly on nodes of N rather than edge E, but accumulating higher total computation across layers.
It worth noting that these theoretical costs only. On modern hardware, actual runtime can deviate significantly because graph-based operations rely on sparse and irregular memory access patterns, which are ineflicient on GPUs. By contrast, transformer-style dense computations are highly optimized and throughput-fffcientt on current accelerators. The following Sec E.2 provides a more practical profiling comparison.
E.2 Runtime Profiling
To further analyze the computational efficiency of our implementation, we compare the detailed runtime breakdowns of GINO [29] and GAOT on the NACA0012 dataset using single NVIDIA
36

---

GeForce RTX 4090 GPU. Table E.2 presents the per-component execution times for both models at batch size of 1.
Table E.2: Comparison of time breakdown between GINO (6.07M parameters) and GAOT (5.62M
parameters) with batch size = I on the NACA0012 dataset.
GINO
GAOT
Component
Sub-Operation
Time (ms)
Sub-Operation
Time (ms)
Graph building
1.984
AGNO
2.296
Encoder
GNO
2.793
GeoEmb
3.412
Total Encoder
4.777
Total Encoder
5.708
Per-layer FNO 1.178 x 5 layers
1.178
Per-layer Transformer (ps=2)
2.02
Processor
5.89
2.02 x 5 layers
10.1
Graph building
1.984
AGNO GeoEmb Total Decoder
1.356
Decoder
GNO
1.140
1.865
Total Decoder
3.124
3.221
Total Time
13.791
19.029
For GINO, the total runtime per sample is 13.79 ms, where the encoder and decoder each spend almost ms on graph building, comparable to or even exceeding the actual GNO operations. Also, the official implementation of GINO does not support batch processing when the underlying graphs change within batch, as is typical for PDEs defined on arbitrary domains. This limitation arises because autch-processing the encoder and decoder often leads to out-of-memory (OOM) errors, making GINO scale poorly with batch size.
In contrast, GAOT eliminates the runtime overhead of graph building by precomputing graphs and loading them efficiently from cache. More importantly, to overcome the memory bottleneck batch processing, we adopt hybrid execution strategy: the encoder and decoder are processed sequentially through lightweight for-loop, which avoids memory overflow without introducing noticeable computational overhead, while the processor (Transformer) is fully batch-processed to maximize throughput. This design choice is based on the observation that, for larger models, the processor typically dominates the compute cost, making it the optimal module to parallelize. As shown in Table E.3, GAOT achieves a total runtime of 263.1 1 ms for batch size 32, while GINO, lacking batching support, must loop 32 times, resulting in 13.791 x 32 441.3 ms. The Transformer processor scales efficiently (from 10.1 1 ms for batch size 1 to only 30.1 ms for batch size 32), while maintaining manageable total memory usage of 7.74 GB. Consequently, GAOT achieves significantly better scalability than GINO both input size and model size, without sacrificing computational efficiency. For very large graphs, e.g. DrivAerML dataset [3], edge masking is used to control memory usage ensuring stable training even on limited-ment amited-memory GPUs.
Table E.3: Breakdown of time and memory for GAOT (5.62M parameters) with batch size 32 on the NACA0012 dataset.
Component
Sub-Operation
Time (ms)
Memory (GB)
Encoder
AGNO + GeoEmb
126
1.836
Processor
5-layer Transformer, ps=2
30.1
3.592
Decoder
AGNO + GeoEmb
107
2,310
Total
263.1
7.738
E.3 Accuracy, Robustness and Computational Efficiency Metrics
Accuracy For benchmarks Table and Table E.5, we adopt the relative L error metric, following the manner of CNO [42], to measure the discrepancy between the ground-truth operator output S(a) and the model's 'ss prediction Sa(a) over discrete set of points. Suppose given sample is discretized into N points (cither on regular grid unstructured mesh). For single-component solution
37

---

field, the discretized relative L' eπror E is defined as
1 N
(S(a);; (So(a)); (S(()).
(E.1)
Because the test set contains multiple upputootputt pairs ((a, we obtain a distribution of errors. We report the median of these errors rather than the mean to mitigate the influence of strong outliers. For multi-component PDE solutions (e.g., velocity and pressure fields), compute the median error per component, then average these medians to obtain a single scalar metric. In time-dependent tasks, we specifically report the relative T.* error at the final time snapshot, as errors usually accumulate over time and thus the last snapshot often poses the greatest challenge.
For the pressure and wall shear stress (WSS) in the AerNet++ dataset, evaluated the model on 1154 samples according to the official leaderboard. The errors are calculated based on normalized pressure and WSS. For pressure, the mean and standard deviation (std) for normalization were obtained from the open-source code of [14]. However, for WSS, as the normalization statistics not open-sourced, we caleulated the mean and variance for the y. and components over 8000 samples to be used for normalization. The Mean Squared Error (MSE) and Mean Absolute Error (MAE) are first computed for each individual sample. Then, the average of these errors across the 1154 test samples is reported as the final result. This entire procedure strictly follows their open-source code methodology of [14].
In Table E.4, we further provide an aggregate performance comparison of GAOT and three represen- tative baseline models (Transolver, GINO, RIGNO-18) on both time-dependent and time-independent dataset categories, described in Table Specifically, for each individual dataset, we calculate the normalized scores for every model. The best-performing model is assigned score of 1. The scores for other models are calculated as the ratio of the best model's 's error to their respective errors:
errorhest
Sworm
(E.2)
eITOT
These dataset scores then summed for each model to derive total scores for the time- denendennnddntt and iiee-independent dataset categories, respectively, offering a complete view of model performance.
Robustness To evaluate the consistency of model performance across different datasets, intro- duce a Robustness Score. This score is calculated for both the time-dependent and time-independent categories of datasets. Leveraging the normalized scores obtained by each model on the individual datasets within these categories, the Robustness Score for a model is defined as:
Robustness Score = Smorm x (1 - CV):
(E.3)
where Snorm is the mean of the model's normalized scores across all datasets in a specific category (either time-dependent time-independent), The term CV represents the Coefficient of Variation of these normalized scores, calculated as:
CV
Spen
(E.4)
Sporm
where Seen is the standard deviation of the model's normalized scores within that same category. A higher Robustness Score suggests that a model not only achieves high average performance (high mean normalized score) but also exhibits less variability in its performance across the different datasets within the category (low CV), indicating greater reliability. The robustness scores for GAOT, Transolver, GINO and RIGNO-18 18 are shown in Table E.4.
Computational Efficiency To provide a comprehensive characterization model performance and analyze the inherent accuracy-efficiency trade-off, we further evaluate the computational efficiency of the models during both training and inference phases.
Training efficiency is quantified by the training throughput, defined as the number of samples the model can process per second during training. encompassing the forward pass, backward pass, and gradient update. A high training throughput is indicative of a model's ability to learn quickly from data. This is essential for handling large-scale datasets or developing large foundation models where training time can be a significant bottleneck. For measuring throughput, the batch size for each
38

---

model was determined first by identifying the maximum value that could be run without encountering Out-of-Memory (OOM) errors on the target hardware. The actual batch size used for the throughput measurement was then set to approximately half of this maximum. This heuristic is based on the observation that peak throughput is often not achieved at the absolute maximum batch size, but rather at point (frequently around half the maximum) where GPU resources, such shared memory bandwidth, are optimally utilized, leading to the highest processing rates.
Inference efficiency is measured by the inference latency, which is the time taken for the model to perform single forward pass on an individual sample (i.e., batch size of 1). Low inference latency is critical attribute for the practical deployment of models, particularly in applications requiring real-time or near real-time predictions, such as in engineering simulations, interactive design tools, or control systems.
All computational efficiency metrics were henchmarked the Bluff-Body dataset with NVIDIA- 4090 hardware. To ensure reliable and stable measurements, the GPU was warmed up prior to data collection, and each reported metric the average of 100 repeated measurements.
E,4 Results for Radar Chart in Main Text.
Table E.4 presents the raw data used to generate the radar chart in Figure of the main text. The metrics depicted in the radar chart include:
Accuracy (Acc. and Acc.(t)): Overall accuracy time-independent (Acc.) and time- dependent (t) datasets.
Robustness (Robust. and (t):st.(t):: Robustness time-independent (Robust.) and dmeedeeendeett (t)) datasets.
Training Throughput (Trui,(train)):: The number of samples processed per second during training.
Inference Lateney (Infer. Latency): The time (ms) taken for a single forward pass on one sample during inference.
The precise definitions and calculation methodologies for these metrics are detailed in Sec. E.3
Table E.4: Data for Radar Chart
Model
acc.(t)
Tput(train)
Infer Latency
Poak memory
InputScal.
ModelScal
robust(()
robust
GAOT
7.45
6,30
97.5
6.966
101.7
68.12
48.7
0,80
0.77
Transolver
4.12
39.5
15.295
144.0
8.96
6.69
0
0.22
GINO
2.77
2.94
60.4
8.455
556.8
30.53
40.00
0.15
RIGNO-18
7.37
4.38
50.3
12.749
188.8
12.52
7.51
0.85
0.29
Table E.4 also includes Peak Memory (MB), which records the peak GPU memory consumption of each model during inference with a batch size of 1. Although not visualized in the radar chart (Figure 1), the data indicates that GAOT exhibits the lowest peak memory usage among the compared models. Furthermore, Figure E. (a, c) illustrates the scaling of peak memory with increasing input grid size and model size, respectively. These plots demonstrate GAOT's superior memory utilization capabilities.
The Input Scalability and Model Scalability scores presented in the radar chart are derived from the training throughput measured under specific conditions:
Input Scalability is based on throughput at an input grid size of 50,000 points.
Model Scalability is based on throughput for model size of approximately 70 million
parameters.
These particular evaluation points were chosen due to the performance limitations encountered with models like RIGNO-18 18 and Transolver on a single NVIDIA 4090 GPU, which prevented us from benchmarking them larger scales. It is important to note that GAOT's architecture allows it to scale significantly beyond these tested limits.
SM Figure E.1 provides detailed scaling curves for both peak memory and training throughput as functions of input grid size and model size. To vary the model size for these comparisons, we systematically adjusted key architectural width parameters for each model:
39

---

QADES
CADES ROCIROS
Thinnng throogppust (samphos))))
Thekk m1ees 61SS (KR5)
BBON0ARR Tiaander
09
100
WP
dize (nodes)
Exput gril. (nodes)
(a) Grid vs. Peak Memory
(b) Grid vs. Throughput
LAMDES GIND BICINO-
Thaning www.ggghhgett.oooooooo (osmplescc
Pook men WISS (KBB)
Thaasdeee
0/-
10 Mode presmctoss)
Model presantess)
(c) Model vs. Peak Memory
(d) Model vs. Throughput
Figure E.1: Performance scaling comparisons across different metrics.
For Transolver, scaled its hidden channel dimension through [64, 128, 256, 512, 1024).
For RIGNO, the hidden channel dimensions of its node and edge functions were varied across [64, 128, 256, 512, 1024].
For GINO, the hidden channel dimension of its FNO processor layers was selected from [16, 32, 64, 128. 256, 512].
For our GAOT model, we scaled the hidden channel dimension of its attention layers using values from [64, 128, 256, 512, 1024, 2048), while the hidden dimension of its FFN layers was maintained at four times the attention layer's hidden dimension.
This figure also introduces results for GAOT-S, a variant GAOT where the patch size in the transformer processor is set to 8 (the default GAOT employs a patch size of 2). As shown, GAOT-8 can achieve enhanced computational performance. Furthermore, as detailed in our ablation studies (Section F.2), this improvement in efficiency with GAOT-8 does not give rise to the substantial accuracy degradation.
E.5 Regular Grid Dataset
In addition to datasets with arbitrary point cloud geometries, also evaluated the performance of our GAOT model on time-dependent PDE datasets where the inputs are provided on regular (structured) grids. The results for GAOT are compared against several baselines, including RIGNO (RIGNO-18 and RIGNO-122 -2), CNO, scOT, and FNO. The performance data for these baseline models are sourced from the original RIGNO paper [36].
As demonstrated in Table E.5, GAOT also performs well on these structured grid datasets. Our model consistently ranks within the top two across six of the seven benchmark datasets, achieving the leading (first place) performance on five of them. This highlights GAOT's robustness and strong generalization capabilities across different input discretizations.
40

---

Furthermore, we ablated GAOT further by just running the VIT processor (without the en- coder/decoder) on these Cartesian datasets. From these results, is clear that some cases, ViT is comparable to or slightly better than GAOT, indicating that the power of GAOT in these cases stemmed from a good processor. However in many more cases, GAOT significantly superior to ViT also showing that the encoder/decoder contribute significantly to expressivity and together, this combination achieves SOTA performance. This interesting experiment clearly delineates the relative contributions of encoder/decoder vs. processor.
Table E.5: Benchmarks with time-dependent datasets with regular grid inputs. Best and 2nd best
models shown in blue and orange fonts for each dataset.
Dataset
Median relative L error [%]
Structured
GAOT
RIGNO-18
RIGNO-12
CNO
VIT
scOT
FNO
NS-Gauss
2.29
2.74
3.78
10.9
3.16
2.92
14.41
NS-PwC
1.23
1.12
1.82
5.03
3.89
7.12
12.55
NS-SL
0.98
1.13
1.82
2.12
0.73
2.49
2.08
NS-SVS
0.46
0.56
0.75
0.70
0.39
1.01
7.52
CE-Gauss
5.28
5,47
7.56
22.0
6.81
9.44
28.69
CE-RP 4.98
3.49
4.43
18.4
4.30
9.74
38.48
Wave-Layer 5.40
6.75
8.97
8.28
5.48
13.44
28.13
E.6 Model and Dataset Scaling
To further investigate the scalability of our approach, we conduct an ablation study on how different model sizes affect performance. We focus on the two compressible Euler datasets, CE-Gauss and CE-RP, each with 1,024 training trajectories. We measure the final-time relative L error (L = (14 and record the total number parameters and per-epoch training time under various hyperparameter configurations
Model Size
We vary the following components our GAOT architecture explained in the Tab B.2:
LC (Lifting Channels):
The number of channels used during the encoder stage to project from the unstructured node features to latent tokens. Intuitively, a larger L.C can preserve more local features when mapping from the input domain to the latent space.
TL (Transformer Layers): The depth of the transformer-
processor. Increasing TL
typically increases modeling capacity for global interactions.
THS (Transformer Hidden Size): The hidden dimension of each self-attention block. A larger THS can capture richer representations.
FFN (Feed-Forwart Network Size):
The hidden dimension inside the FFN sub-layer, which set to 4 THS following standard vision transformer practice.
Table E.6 summarizes the performance across a range of these hyperparameters. We also record the total number of trainable parameters millions) and the approximate epoch time (in seconds) on one NVIDIA 4090 GPU with a batch size of 64. Here, all experiments are done with patch size equal 2.
From the top block of Table E,6 (rows 1-4), we observe that as we increase THS from 32 to 256 (keeping T1.=5 and LC=32), the final-time emoor both CE-Gauss and CE-RP decrease significantly. For example, on CE-Gauss, the error drops from 48.4% down to 6.88%. This trend reflects the transformer's ability to scale with hidden dimension. The training time per epoch grows from roughly 84 seconds to 143 seconds. While performance improves, larger THS demands more computational resources.
Next, we fix (TL,THS))(55256) and vary LC in the middle block (rows 5-7). Setting LC=32 consistently achieves strong results. Lowering LC to 16 slightly degrades performance, while pushing LC to 64 or 128 yields only marginal gains. Hence, LC=32 appears sufficient to capture the encoder-level geometry information.
Finally, the bottom block (rows 8-10) examines the effect of transformer layers TL. from up to 10. With TL=I, emrors remain quite high (25.0% on CE-Gauss); adding layers substantially reduces error
41

---

Table E.6: Relative L test errors / = (14 with different architectural hyperparameters, Time refers to training time with batch size equals 64 NVIDIA-4090 GPU, and the patch size is set to 2.
The size training trajectories 1024.
Model size
Hyperparameters
Median relative I error [%]
Parameters [M]
Time [s]
I.C
TI.
THS
FFN
CE-Gauss
CE-RP
0.14
84
32
5
32
128
48.4
26.5
0.41
32
64
256
13.2
12.0
1.42
100
32
5
128
512
9.17
7.90
5.6
143
32
256
1024
6.88
5.28
5,5
142
16
256
1024
7.97
5.94
5,6
154
64
256
1024
6,94
5.18
6.1
181
128
5
256
1024
7.33
5.20
1.16
50
32
256
1024
25.0
14.5
3.39
98
32
256
1024
9.00
6.80
11.2
260
32
10
256
1024
5.28
5.35
Median Relstive L Error [%6]
Median Relative I Eerur [90
25
32
16
Dataset
Dataset
CE-Gans
CE-GGus
Transformer Hiddln Size
Transformer Layers
(a)
(b)
Figure E.2: Relative median L test errors at 114 with different strategies for scaling model sizes.
The x-axis in the left plot corresponds to all the following hyperparameters: THS and TL.
to 9.0% TT. = 3, and ultimately down to 5. 5.28% at TI.10 on CE-Gauss. Increasing TL to 10 also expands the parameter count 11.20 1..MM, nearly doubling the training time per epoch (260s).
Figure E.2 illustrates how errors decrease as we scale THS or TL, while Figure E.3 shows the effect of changing LC the total parameter count. The largest model tested reaches 11.2M parameters and attains around 5% error on CE-Gauss, demonstrating the potential to improve accuracy by investing in more computational resources.
Data Size So far, have discussed how increasing model size affects accuracy. In this subsection. we turn our attention to data scaling: we examine how the learned operator's performance changes as the number of training samples (trajectories or static solutions) grows. Figure E.4 illustrates sets of experiments:
(a) Time- Semee-Deenndnt (Fluid) Datasets. We plot the final-time (t (14) error multiple fluid PDE benchmarks as function of the training set size (128, 256, 512, 1024). All of these datasets use partial-gric -riid subsampling. The results confirm that errors consistently drop across all fluid datasets, often in a near-linear fashion with respect to the number of training trajectories. This trend highlights the model's capacity to benefit from additional time-series diversity.
increase the number of training trajectories,
(b) Time-Indepenne Indepededdnn (Static) Datasets. We similarly measure how the final solution enror decreases when expanding the dataset size to (128, 256, 512, 1024, 2048) for three static PDE tasks: Poisson- Gauss, Poisson-C-Sines, and Elasticity. Note that elasticity is limited to most 1024 samples due to
42

---

Median Relative L' Error [96]
50
Median Relative I Envor
26
16
6
Datasct
Dataset
CR Gauss
Gimer
CE-RP
16
32
61
128
0012
1.00
3.00
10.00
Lifting Channels
Number of Parameters, 10"
(a)
(b)
Figure E.3: Relative L' test errors at t = 114 with different strategies for scaling model sizes. The x-axis in the left plot corresponds all the following hyperparameters: LC, and the total number of trainable parameters.
20
100
Median Relative L Error [96]
10
Median Relative L Error [%]]
Dalasel
NS PwC
NS Garzer
10
Dalaset
Poisson Gouss
NS
Posesom S-Sinss
Elusticity
CE-Gonse
CERR
128
512
1024
1024 2018 Number of Training Samples
Number of Training Trajectories
(a)
(b)
Figure E.4: Relative LI errors against the size of training dataset. The left plot shows these results for the time-dependent fluid datasets, and the right plot for time-independent datasets. The lines show linear regression slopes for each dataset.
data availability. Across all these static problems, observe consistent downward slope in error as the number of samples increases, again underscoring the advantage of larger training sets.
Overall, both dynamic (time-dependent) and static (time-independent) scenarios, GAOT exhibits a scalable relationship between training set size and error reduction. As the training data grows, the learned operator converges more reliably to the underlying PDE solution. This robust data scaling property supports premise that GAOT serve strong foundation model backbone for PDE tasks, becoming increasingly accurate with more extensive datasets.
E.7 Resolution Invariance
One of the core properties for operator learning is resolution invariance the ability train specific discretization yet accurately predict solutions at higher/lower resolutions. To validate this property in GAOT framework, conduct experiments
time-independent PDE, Poisson- Gauss.
Figure E.5 illustrates the resolution invariance capabilities of GAOT. this experiment, GAOT was trained using data discretized by 2048 points. Its performance was then evaluated across spectrum
43

---

10
GAOT
Training Resolution
Median Relative I Error (%6]
RIGNO-18
'rranslvver
GINO
256
512
1024
2048
4096
8192
16384
Resolution (Number of Test Points)
Figure E.5: The GAOT model is trained resolution of 2048 and evaluate various resolutions. The results for RIGNO-18, Transolver and GINO correspond to models trained and tested at resolution 8192.
of seven distinet resolutions: three sub-resolution settings (256, 512, and 1024 points), the training resolution itself (2048 points), and three super-resolution settings (4096, 8 8192, and 16384 points). For comparative purposes, Figure E.5 also displays the performance of the top three baseline models from Table (excluding GAOT itself), namely RIGNO-18, Transolver, and GINO. These baseline results were obtained from models that were both trained and tested resolution of 8192 points.
The results demonstrate that GAOT possesses excellent resolution invariance, Notably, even when trained a resolution 2048 points, GAOT only generalizes well higher resolutions but also achieves the best performance when tested at 8192 points. It outperforms the baseline models which were specifically trained for and tested this higher resolution (8192 points), underscoring GAOT's efficiency and robustness in learning essoutiionindepeende solution operators.
E.8
Comparison of Neural Field with UPT
The neural field property enables continuous evaluation of the learned mapping at arbitrary spatial points, allowing models train and test at different resolutions-a highly desirable capability for PDE operator learning [1, 45). This feature allows a model trained on coarser resolution for efficiency to be tested at finer resolutions for higher accuracy.
In Table 3 of the main paper, we demonstrated this property for GAOT on the industrial-scale 3D Drivaernet++ dataset. However, noted in this result, possessing the neural field property does not necessarily imply higher accuracy. Indeed, GAOT trained using only 10% of input points but tested on the full resolution (500K surface points) is less accurate than the fully trained version, though still outperforming several baselines.
To further examine this, compared GAOT with the hertlll-ied--b-seeed UPT model [1] the 2D NACA0012 dataset under two settings: (i) both trained and tested at full resolution, and (ii) trained at one-fourth resolution but tested at full resolution. The results shown in Table E.7.
Table E.7: Neural field training with 1/4 sub-sampling on the NACA0012 dataset. Model Error (Full Resolution) Error (Neural Field)
GAOT
6.81
9.54
UPT
16.1
16.3
44

---

These results reveal that: (a) training at full resolution yields better accuracy than coarse training; (b) GAOT consistently outperforms UPT under both configurations; and (c) UPT's limited improvement with resolution suggests poor convergence, further highlighting the robustness of GAOT as neural surrogate with flexible spatial generalization.
Table E.8: Quantitative comparison the Driv dataset. The performance both subsam- pled point clouds (10k points) and full-resolution meshes (487k points) is taken from the CarBench paper [15]. While baseline models struggle with the high-resolution input, GAOT is trained directly on the full-resolution mesh, enabling it to capture fine-grained geometric details. We report results for representative unseen sample (E_S, WW__W_6445 and the dataset-wide average all
1154 test samples.
MAE (m² 2/5)
RMSE (m²/s²
Rel L2
R
Model
10k
Full
10k
Full
10k
Full
10k
Full
PointNet
32.2
32.5
62.4
63.3
0.395
0.403
0.757
0.747
NeuralOperator
25.9
26.3
51.2
54.4
0.324
0.347
0.837
0.813
PointMAE
23.0
24.7
46.2
52.0
0.292
0.331
0.867
0.829
PointNetLargo
22.0
25.1
42.1
55.3
0.266
0.352
0.890
0.807
RegDGCNN
16.5
19.9
32.4
46.1
0.205
0.293
0.935
0.866
PointTransformer
16.2
20.6
29.5
47.0
0.187
0.299
0.946
0.861
TransolverL.are
12.2
18.5
25.5
47.2
0.161
0.300
0.959
0.860
TripNet
13.6
19.8
25.1
44.0
0.158
0.297
0.961
0.863
Transolver++
12.3
18.3
24.6
46.0
0.156
0.293
0.962
0.867
Transolver
11.7
17.7
23.7
46.1
0.150
0.294
0.965
0.866
AB-UPT
10.2
17.2
21.5
45.7
0.136
0.291
0.971
0.869
GAOT (Sample 648)
11.4
22.8
0.145
0.967
GAOT (Avg. 1154 samples)
12.9
23.9
0.157
0.958
E.9 Training at full vs. low Resolution for the DrivAernet++ Dataset.
Given these results, we turn our attention to the recent investigation of the question of whether training full resolution confers any benefits over training at low resolution and simply inferring at full resolution. This question was recently explored in the context of the industrial scale DrivAemet++ dataset within the Carbench framework [15]. their study, Al surrogates at subsampled resolution of 10K points from the full surface point cloud of 487 K points. Then, they tested the models on an unseen sample at the low-resolution of 10K points as well the full resolution of 487K points. To ensure uniformity, they interpolated the results from testing at wew-resouuton to the full resolution. We reproduce their results verbatim in Table E,8 where all the except the last two are identical to Table of [15]. As seen from the Table, the performance of models trained at low-resolution degrades severely when interpolated to the full resolution. In fact, all models suffer from this issue and models such as Transolver and AB-UPT [2] which perform very well at low-resold www-resolution suffer from so much degradation in accuracy that their performance is at par with the models such RegDGCNN, which were also inaccurate at low-resolution.
authors of [15] trained various
On the other hand, we trained GAOT on the full resolution and present the results the single test sample as well as the average for all test samples in the last two rows of Table E.8 observe that GOAT, trained at full resolution is twice more accurate than all the other models. Morcover, the performance of GAOT trained on full resolution is par with the models that been trained and even tested on the low-resolution. Thus, this experiment further indicates that training at full resolution, possible, is the best strategy for yielding accurate neural surrogates.
Finally, we would like to point out that models such as AB-UPT [2] also possess the neural field property and can be tested at full resolution, when trained at low resolution, using the neural field property, rather than just interpolating to high resolution. However, as shown in Table 3, we recall that the neural field property does not necessarily translate to performance retention and one can expect a possible degradation of accuracy, even with this paradigm, while training at low resolutions. Nevertheless, this issue needs to investigated further.
45

---

E.10
Transfer Learning
The set of geometries illustrated Figure E.6a, represent the varying bluff-body geometries in Bluff- dataset, which is one of the benchmarks presented in Tab. 1 of the main text. This dataset constructed by simulating compressible flow across diverse bluff-body geometries at varying Ma and as described in Sec. D.2. The distinct shapes depicted in Figure E.6b were specifically employed for the fine-tuning stage of our transfer learning experiments. The corresponding transfer learning performance, demonstrating the model's ability adapt from the shapes used for pretraining to the novel ones indicated in Figure E.6b, is presented in Figure 3(c) in the main text.
Square
Circle
Cone
Rectangle-I
Semiriscke-F
Kilipse
Bllipee
Elliyse
Kllipse
Rillywe
(a) Shapes utilized for pretraining in the transfer learning experiments.
ReclanglesS
Bilipse)]
Hillpwe
(b) Bluff body shapes employed for fine-tuning (FT) phase of transfer learning task.
Figure E.6: The geometries in (a) are included in the Bluff Bluff-Body dataset in Tab. 1. Shape- (8: C, F, S, L; Shape: Semicirele, Cone, Rectangle) indicates shapes and their contact surfaces (*) with respect to the flow. Here, F flat surface, C curved surface, L larger side, S smaller side.
E.11 Training Randomness
In order to quantify the dependence of the final model performance on the inherent randomness in the training process, such as weight initialization, we trained the GAOT model six independent times. Each training run utilized different seed for the pseudo-random number generator. These experiments were conducted on Bluff-Body dataset. The statistics of the resulting relative L,7 test enrors across these six runs are summarized in Table F.9. The standard deviation of these errors is 0.12. This relatively small standard deviation suggests that the GAOT model exhibits good stability with respect to the random aspects of the training procedure.
Table E.9: Statistics of relative I test errors with different random seeds. We train GAOT on Bluff-Booe Bodf-BBddy dataset, which is repeated times.
Dataset
Error [%]
Mean + Standard deviation
Bluff Bluff-Body
2.39 + 0.12
46

---

E.12 Runtime Comparison between GAOT and Classical Solvers
The main rationale for the design of efficient neural surrogates such as GAOT lies in the fact that classical numerical PDE solvers are slow, particularly in industrial 3D problems. On the other hand, neural operators are ultra-fast to infer. We discuss this issue from the perspective of GAOT in this section. To begin with, Table E.10 summarizes the inference times of GAOT and compares them with the runtimes of classical numerical solvers across several CFD datasets. For fairness, all GAOT inference times include graph construction overhead.
Table E.10: Comparison of GAOT inference time with traditional CFD solvers.
Dataset
Mesh Points Inference Time (ms) Traditional Solver
Speedup
Bluff-body 14,000
9.77
283 ~ 419
~ 10a
NS-SL
16,384
10.14
0.1
~ 10
DrivaerNet++ 500,000
365.36
375 core hours
3.7 x 107
DrivaerML 9,000.000
14,091.08
61,440 core-hours
1.5 x 10
The NS-SL dataset is taken from Ref. [22], following the setup of Ref. [36] with random point- cloud inputs. Their optimized GPU-based spectral viscosity solver achieves approximately 0. per sample. For our new Mattasets-NAAOON compressible flow around airfoils and bluff bodies, the traditional CPU-based solver required between 283s and 419s per sample due to adaptive grids and iterative refinement (see SM D for details). For the large-scale 3D benchmark DrivaerNet+++ [14], the original authors reported an average cost of approximately 375 CPU hours per sample. For DrivAcrML, [3] indicates that each scale-resolving CFD simulation using the hybrid RAÑS-LES approach required around 40 hours 1536 cores (61,440 core-hours) on AWS HPC clusters, which represents an industrial -grade high-fidelity CFD workflow that includes near-wal wall resolution and statistical convergence monitoring.
NACA2412, RAE2822, and Bluff-body-which simulate
Overall, traditional numerical solvers span a runtime range from 0. to few minutes for 2D problems discussed in this work, and up to tens of thousands of core-hours for industrial 3D simulations. In contrast, GAOT performs inference within 8.95-10.14 14 ms for 2D datasets and around 365ms for 3D DrivaerNet++, achieving a remarkable speedup of 1-5 orders of magnitude for 2D problems and up to orders of magnitude for 3D industrial-scab benchmarks. The reason why Driv AerML shows a significantly higher runtime compared to other datasets is mainly due to the excessive time spent graph building. In fact, if we exclude the graph building process, its inference speed is 446.06 ms. The relatively long graph building time can be attributed to two factors: first, the large scale of the input mesh points; and second, to control memory usage while covering the entire solution domain, we employed more complex graph building techniques (see Section B.5). Nevertheless, even under these conditions, the efficiency improvement compared to traditional solvers remains at the billion order of magnitude level as its runtime of approximately 15 secs is completely dwarfed by the more than the K core hours run time of the LES based ground truth simulations.
One can argue that the runtime of classical numerical PDE solvers can be reduced by coarsening the resolution. This does not hold for the industrial scale datasets as there is a minimum mesh resolution that is essential for the underlying physics to be resolved. However, the academic 2D datasets, can perform such a coarsening and examine the cost-accuracy pay-off for GAOT still holds. To this end, we consider the NACA0012 dataset and coarsen the resolution of the underlying finite-volume solver by two levels of refinement. The underlying error is now 4.5%, when compared to the ground truth but the run-time reduces from 7 mins to approximately minute. This error is comparable the error of GAOT (6.8),, while the runtime of GAOT for this dataset is approximately 10 milli-seconae leading to a speedup of 6000. This further demonstrates the enormous advantage that an efficient and accurate neural operator such as GAOT can provide, when compared to classical numerical PDE solvers.
F
Ablation Studies
F1
Pacoud--rrcess-Devode
In this subsection, we investigate the performance of four different encode-process-deee decode architec- tures on both time-dependent and time inme-independent PDE benchmarks. The four models considered
47

---

are GAOT (ours), Regional Attentional Neural Operator (RANO), Regional Fourier Neural Operator (RFNO) and GINO [29], with components are Messsage-Passing (MP) graph neural network [19]. Transformer [47], Fourier Neural Operator (FNO) [27], Graph Neural Operator (GNO) [28] and proposed Multi-scal Attentional GNO (MAGNO). Table F.1 summarizes the components of each model. All variants follow an encode-procee decode pipeline but differ in how graph, Fourier, or transformer- mechanisms are deployed.
Table F.1: Components for different encode proceee---esssssee decode designs.
Model Encode
Process
Decode
GAOT MAGNO
Transformer
MAGNO
RANO
MP
Transformer
MP
RFNO
MP
FNO
MP
GINO
GNO
FNO
GNO
Figure F.1 shows the median relative L' error for each model on six PDE datasets (4 time-dependent PDEs, 2 time-independent PDEs). All models are trained for 500 epochs under the same data splits and hyperparameter conditions. We can see that GAOT consistently achieves strong performance and robustness across all six datasets. Its errors remain low, highlighting the effectiveness of combining MAGNO for local geometric encoding with transformer-based global attention. GINO ranks second in overall accuracy, yet exhibits noticeable difficulties on NS-Gauss and Poisson-C-Sine and RFNO perform moderately well on simpler datasets (e.g. Elasticity), but show instability more challenging tasks (e.g. NS-SVS or NS- (a-ss. This indicates that reliance on message paassagg-pssingg or FNO-based processors alone may not be sufficient handle diverse PDE and geometry conditions with the same level of robustness.
RANO
85 PW
K8.60
N8.8V9
Easide
Moddl
OXOT
RANO
GBNO
Median velutiive azzor
Figure F.1: Median relative I,1 errors (%) of GAOT, RANO, RFNO, and GINO on six PDE bench- marks.
Overall, these results reinforce GAOT's stability across multiple PDE settings. Even with a fixed training protocol (500 epochs for each dataset). GAOT consistently converges faster and more reliably, underscoring the advantage of geometry-aware tokens, multiscale attention, and the flexible transformer backbone.
F.2 Tokenization Strategies
In Section B.1, we have introduced three tokenization methods. Here, we compare these strategies on two datasets, Elasticity and Poisson-C-Sines. Figure F.2 shows the final median relative L' errors for each approach. The Strategy consistently achieves the best performance on both unstructured datasets. Strategy II & III perform similarly to Strategy on simpler datasets (elasticity), but can fail to converge on the more challenging one, Poisson-C-Sines. Similar situations also happen on models like UPT and GNOT in Tab. 1 of main text. Overall, Strategy I emerges as the most robust approach our current experiments. While Strategies and III show promise, they require more careful optimization match Strategy I's reliability.
48

---

Medean melatro 12 seme 1
Figure F:2: Median relative I errors (%) comparing three tokenization strategies Elasticity, and Poisson-C-Sinee The Strategy I, II, III corresponds to the methods discussed in Section B.1.
Next, we focus on Strategy I and study how varying the number of latent tokens (LT), patch size (PS), and rudius (GR) affect performance. Note that here we do not use multiscale radii; each token has single radius. Table F.2 summarizes experiments on the elasticity and Poisson-Gauss datasets. Results show that fewer tokens (e.g., [32,32]) can degrade performance in some cases
Table F22 Median relative L' errors (%), parameter counts, and training time with different numbers of latent tokens (LT), patch sizes (PS), and radii (GR).
Model Size
Hyperparameters
Median Relative T,a Error [%] Elasticity
Params [M]
Time [s/it]
LT
PS
GR
Poisson- Gauss
5.60
10.1
[64, 64]
2
0.033
1.80
1.05
6,00
3.06
[64, 64]
4
0,033
1.71
1.57
10.7
2.20
[64, 64]
0.033
1.60
1.65
5,56
10.1
[32, 32]
0.066
3.41
1.22
5,60
3.00
[32, 32]
2
0,066
2.25
1.22
6.00
11.8
[128, 128]
0.033
1.67
1.72
10.7
5.02
[128, 128]
0.033
1.62
1.22
(clasticity), presumably because the domain coverage becomes coarser, making it harder to capture local variations. More tokens ([64,64) or [128, 128]) typically improve accuracy and stabilize convergence. Nevertheless, computational costs rise when the number of tokens grows, transformer attention scales quadratically with token count. Increasing the patch size (PS) reduces the number of tokens entering the transformer, lowering the training time. Encouragingly, performance does not degrade sharply with larger patches. For instance, going from PS = 2 to 8 is fairly stable across datasets. Note that the overall parameter count increase if each token aggregates larger local features, but in practice, training runs faster due to fewer tokens in self-attention. Radius (GR) grows if we reduce the number of latent tokens because we need to ensure coverage of the entire physical domain by enlarging the receptive field. This is critical for unstructured or irregular samples, especially if tokens must capture a bigger subregion.
F.3 Time-Stepping Method
We now investigate how different time-stepping formulations (see Section B.6) affect performance on time-dependent PDEs. Specifically, we compare the output, residual, and derivative stepping strategies. Table F.3 reports the median relative 1 errors for six representative fluid dynamics benchmarks on regular grids.
time derivative (derivative column) often yields the lowest final-time errors on all but one dataset (CE-RP, where the Outpul strategy slightly outperforms the others). We hypothesize that treating the operator as d,u naturally enforces continuous dependence on time, analogous to neural ODEs residual networks [21, 11], which can improve stability and accuracy over multiple steps. In experiments involving time-dependent PDEs, therefore use
As shown, modeling the operator
49

---

Table F.3: Median relative L errors (%) at final time (14 for GAOT with three different time-stepping methods.
Dataset
Median relative L error [%]
Output
Residual
Derivative
NS-Gauss
3.57
3.60
2.52
NS-PwC
1.95
1.70
1.23
NS-SL
1.78
1.49
1.29
NS-SVS
0.60
0.60
0.56
Ca-Gauss
8.80
8.93
7.97
CE-RP 5.17
6.12
5,94
derivative time stepping as the default unless stated otherwise. This approach not only achieves strong final-time accuracy, but also aligns with our design goal of differentiable, cone-coninuuous operator.
F.4 Geometric Embedding
As discussed in Section B.3, our framework incorporates geometric embedding network to encode shape and domain information separately from the physical (PDE) state. Table F.4 compares these ge- ometric embedding approaches against baseline "original" (i.e., no additional geometry embedding) on two original unstructured datasets (Wave-C-Sines. -oisson-C-Sines).
Table F:4: Median relative L errors (%) for various geometry embedding approaches. Original omits geometric embedding, while Statistical and PointNet follow Section B.3.
Dataset
Median relative I, error [%]
original statistical pointnet
Wave-C-Sines
6.50
5.69
6.07
Poisson-C-Sines 6.60
4.66
23.7
In the unstructured datasets, including Wave-C-Sines and Poisson-C-Sines, explicitly encoding domain geometry yields a more pronounced benefit. In particular, the statistical strategy consistently outperforms PointNet these irregular meshes, and in Poisson-C-Sines, training with the PointNet approach appears unstable (23.7% error). Based on these observations, we use statistical embedding by default for unstructured dataset given its stable and superior performance in most cases.
We further conducted ablation studies on DrivaerNet++ and DrivaerML to examine the effect of statistical geometric embeddings. Three configurations were considered: (1) without geometric embedding (nGEmb-nGEmb), (2) applying geometric embedding only in the encoder (GEmb- nGEmb), and (3) applying geometric embedding in both the encoder and decoder (GEmb-GEmb).
Table F.5: Ablation study on the effect of geometric embedding configurations across DrivAerNet++ and DrivAerMI. datasets.
GEmb-nGEmb
GEmb-GEmt
nGEmb-nGEmb
Dataset
MSE
Mean AE
MSE
Mean AE
MSE
Mean AE
DrivAerNet++(pp)
4.2694
1.0699
4.3119
1.0818
4.5278
1.1036
DriVAerNet++(wss))
8.6878
1.5429
8.7783
1.5690
9.3192
1.6125
DrivAerML(p) 5.1729
1.2352
10.8591
1.7344
8.6625
1.5693
DrivAerML(wss) 16.9818
2.1640
41.0027
3.0822
24.8614
2.4965
The results show that introducing geometric embeddings consistently improves model performance compared to not using them. Interestingly, we observed that applying the geometric embedding only in the encoder yields even better performance, but less computational effort. Moreover, for the DrivAerMr dataset, adding a geometric embedding in the decoder actually degrades performance, producing results even worse than the model without any geometric embedding. Our analysis indicates that while the model's training loss continues to decrease, its validation loss quickly saturates. We think that incorporating geometric embedding in the decoder, where operations very close to the
50

---

final predictions, makes the model more prone to overfitting, Especially the 3D industrial dataset, techniques. These complex graph-building tricks lack
where we used more complex the unified principles for the model to learn and likely amplify the overfitting tendency when the decoder also encodes geometric information. Therefore, we only use geometric embeddings in the encoder for our 3D industrial benchmarks.
F,5 Multiscale Features
As introduced in Section B.2, our encoder can capture multiscale local information by aggregating neighborhood features across multiple radii. Specifically, we compare:
Single-scale: using a single fixed radius 0.033 for each point.
multiscale: using three radii [0.022, 0.033, 0.044 for each point.
Table F.6: Median relative L errors (%)
comparing single-scale vs, multiscale features.
Dataset
Median relative L error [%] Single-scale multiscale
Wave-C-Sines
5.69
4.6
Roisson-C-Sines
4.66
3.04
Table F.6 reports the mean relative L' errors on unstructured datasets including Caisson-C-Since Results show that multiscale neighbors yield clear reduction in error. For instance, in Poisson-C-Sinn the error decreases from 4.66% to 3.04%. This contrast reflects the fact that single, fixed receptive field on a regularly spaced grid is often sufficient. However, unstructured domains where the mesh density can vary, using multiple radii helps the network capture both fine and coarse local structures.
and
G Visualizations of Datasets
Estimates produced by trained models are visualized in this section for different datasets.
Input
Ground-truth
Model estimate
-2.5
0.0
2.5
-2
0
1
1c-4
10-3
Figure G.I: Model input, ground-truth solution, and model estimate of a test sample of the Poisson- C-Sines dataset.
51

---

Input
Ground-truth
Model estimate
5
0
5
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
1e-1
1e-3
Figure G.2: Model input, ground-truth solution, and model estimate of a test sample of the Poisson- Gauss dataset.
Input
Ground-truth
Model estimate
2
4
6
8
1e-1
1e2
Figure G.3: Model input, ground-truth solution, and model estimate of test sample of the Elasticity dataset.
52

---

Input
Ground truth
Model estimate
distribution
SDF
0.5
0.72
Ma 0.7, 2.0
Input
Ground-truth
Model estimate
Point distribution
SDF
(b) Ma 0.9, 3.0
Input
Ground-truth
Model estimate
Point distribution
20
0se8 0.28 oan8
1068 1200 1200
1.28
1.000
1.848 2,00
(c) Ma 1.2, a 2.0
Figure G.4: Model input, ground-truth solution, model estimate and point distribution of test samples of the NACA0012 dataset.
53

---

Input
Ground truth
Model estimate
distribution
SDF
0.5
Ma 0.9, 4.0
Input
Ground-truth
Model estimate
Point distribution
SDF
(b) Ma 1.0, 1.0
Input
Ground-truth
Model estimate
Point distribution
20
0400
0.36
0.22
088
120
1.30
1.52
(c) Ma 1.3, a 3.0
Figure G.5: Model input, ground-truth solution, model estimate and point distribution of test samples of the NACA2412 dataset.
54

---

Input
Ground-truth
Model distribution
SDF
1.0
2.0
0508
0,500
0.24
Ma 0.6, 3..2
Input
Ground-truth
Model estimate
Point distribution
SDF
(b) Ma 1.1, 4.0%
Input
Ground-trute
Model estimate
Point distribution
2.0
0368 0696 0.22 0982 1.000 1200 1336 1.664 1.92 1.720
(c) Ma 1.2, a 3.5
Figure G.6: Model input, ground-truth solution, model estimate and point distribution of test samples of the RAE2822 dataset.
55

---

Input
Ground-truth
Model estimate
Point distribution
0.75
1.00
0473 0375
0073 a77a 6875 6975 1075 1375 1275 1373
(a) Cone at Ma = 0.9, or = 7.0%
Input
Ground-truth
Model estimate
Point distribution
SDF
0.00
0.25
0.50
0.75
1.00
0.100
0312
0404 asse area 0920 1072 1,24
1.76 1.52
(b) Ellipse
1.05, 12.5
Input
Ground-truth
Model estimate
Point distribution
SDF
(c) Semicircle-Fas Ma. 1.2, 6.0
Figure G.7: Model input, ground-truth solution, model estimate and point distribution of test samples of the Bluff-Body dataset.
Input Mesh
Ground-truch
Model estimare
Figure G.8: Model input, ground-truth solution, model estimate of a test sample N_S_WWS WM__WM_172 of the surface pressure on the DrivAerNet++.
56

---

Input Mesh
Ground-trurh
Model estimare
(a) WSS X
Input
Ground truch
Model estimare
sherr (Lrvepreee) 120 029
(b) WSS Y
Input Mesh
Ground-truch
Model estimare
(scompannr)
(c) WSS Z
Figure G.9: Model input, ground-truth solution, model estimate of a test sample N_S_WWS WWS_WM_172 of the surface wall shear stress on the DrivAerNet++.
57

---

Input Mesh
Ground-trure -oundd-truch
Model estimate
Frosur: Ocllkies
Figure 1.10: 10: Model input, ground-truth solution, model estimate of test sample Boundary_78 78 of the surface pressure coefficient on the DervAL
58

---

Input Mesh
Ground-trurh
Model estimare
Sharstay (oompee 323
(a) WSS- X
Input
Ground truch
Model estimare
3306
2012 3.
sher (Lrvepree)
12.30
(b) WSS Y
Input Mesh
Ground-truch
Model estimare
(scoepasee)
(c) WSS Z
Figure 1.11: Model input, ground-truth solution, model estimate of test sample Boundary_ 78 of the surface wall shear stress on the DrivAerML.
59

---

Input Mesh
Ground-trune truth
Model estimate
Figure G.12: Model input, ground-truth solution, model estimate test sample of the surface pressure the NASA-CRM.
60

---

Input Mesh
Ground-trurh
Model estimare
(a) SFC X
Input
Ground
Model estimare
4
247-20
(b) SFC-Y Y
Input Mesh
Ground-truch
Model estimare
SFC Z
Figure G.13: Model input, ground-truth solution, model estimate of test sample of the surface friction coefficient on the NASA-CRM.
61

---

Input
Ground-truth
Model estimate
0
-1.0
-0.5
0.0
0.5
1.0
-1.0
-0.5
0.0
0.5
1.0
Figure G.14: Model input at t = fo. ground-truth solution and model estimate at t = f14 of a test sample unstructured NS-Gauss dataset.
62

---

Input
Ground-truth
Model estimate
Vx
0
-4
-2
2
1e-1
1
5
0
-2
4
1e-1
1e-1
Figure G.15: Model input at = to. ground-truth solution and model estimate at t = f14 of a test sample unstructured NS-PwC dataset.
63

---

Input
Ground-truth
Model estimate
-15
-1.0
-0.5
0.0
0.5
1.0
1.5
-1.0
-0.5
0.0
1.0
1e-1
Figure G.16: Model input at t = fo, ground-truth solution and model estimate at t = f14 of a test sample unstructured NS-SL dataset.
64

---

Input
Ground-truth
Model estimate
-4
1e-1
1e-1 1
-2
2
-4
-2
0
2
3
1e-1
1e-1
Figure G.17: Model input to, ground-truth solution and model estimate f f14 of test sample unstructured NS-SVS dataset.
65

---

Input
Ground-truth
Model estimate
1.0
1.1
0.7
0.8
0.9
1.0
1.1
1.2
1.3
1.4
-15
-1.0
-0.5
0.0
1.0
wy
0
1100
00.75
-0.50
-0.25
0.00
0.25
0.50
0.75
1.00
1.5
2.0
2.5
3.0
3.5
4.0
1e-7+2.5
Figure G.18: Model input at = fo. ground-truth solution and model estimate at t = f14 of a test sample unstructured CE-Gauss dataset.
66

---

Input
Ground-truth
Model estimate
1.2
1.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
-2
1e-1
le-1 1
-2.5
0.0
2.5
4
-2
2
4
le-1 1
1e-1
2.0
2.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
1e-1
1e-1
Figure G.19: Model input at fo, ground-truth solution and model estimate t f14 of test sample unstructured CE-RP dataset.
67

---

Input
bround-truth
Model estimate
-3
-2
-1
2
3
1c-1
2.5
3.0
3.5
4.0
2.6
2.8
3.0
3.4
3.8
4.0
1e3
1e3
Model input at = lo. ground-truth solution and model estimate at / = 414 of a test sample unstructured Wave-Layer dataset.
Figure G.20:
Input
Ground-truth
Model estimate
-2.5
0.0
2.5
-3
-2
-1
0
1
2
3
1e-2
1c-2
Figure G.21: Model input to. ground-truth solution and model estimate f f44 of test sample Wave-C-Sines dataset.
68

---

Input
Ground-truth
Model estimate
-1.
-0.5
0.0
0.5
1.0
Vy
-1
-1.0
-0.5
0.0
0.5
1.0
Figure G.22: Model input at t = for ground-truth solution and model estimate at t = f14 of a test sample NS-Gauss dataset.
69

---

Input
Ground-truth
Model estimate
0
-6
-2
2
6
1e-1 1
1e-1
1
-5
0
-2
2
4
16-1
1e-1
Figure G.23: Model input at to. ground-truth solution and model estimate at t = f14 of a test sample NS-PwC dataset.
70

---

Input
Ground-truth
Model estimate
-1.5
-1.0
-0.5
0.0
1.0
1.5
-1.0
-0.5
0.0
0.5
1.0
1e-1
Figure G.24: Model input at = 40 ground-truth solution and model estimate at / = 414 of test sample NS-SI. dataset.
71

---

Input
Ground-truth
Model estimate
Vx
-2
0
2
-4
-2
2
16
1
1e
Wy
-2
0
-4
-3
-2
-1
2
3
16-1
1e-1
Figure G.25; Model input at to. ground-truth solution and model estimate at t = f14 of a test sample NS-SVS dataset.
72

---

Input
Ground-truth
Model estimate
0.9
1.1
0.8
0.9
1.0
1.4
Vx
-2.0
-1.5
-1.0
-0.5
0.0
1.0
1.5
2.0
-7.5
-5.0
-2.5
0.0
2.5
5.0
7.5
1e-1
2
1.5
2.0
2.5
3.0
3.5
4.0
1e-7+2.5
Model input at = lo. ground-truth solution and model estimate at / = (14 of test sample CE-Gaaus dataset.
Figure G.26:
73

---

Input
Ground-truth
Model estimate
1.2
1.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
-2
1e-1
le 1
-2.5
0.0
2.5
-4
-3
-2
-1
0
2
3
1e-1
1e
1
2.0
2.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
1e-1
1e-1
Figure G.27: Model input at to. ground-truth solution and model estimate t f14 of test sample CE-RP dataset.
74

---

Input
Ground-truth
Model estimate
2
3
1e-1
2.5
3.0
3.5
4.0
2.6
2.8
3.0
3.2
3.4
3.6
3.8
4.0
1e3
1e3
Figure G.28: Model input at = 4o. ground-truth solution and model estimate at / = 414 of test sample Wave-Layer datasct.
75

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction (lines 55-81) clearly state the paper's S contribu- tions regarding the GAOT model, and demonstrate its performance in terms of accuracy and efficiency across various PDE learning tasks.
Justification:
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: The Limitations and Extensions section (line 355-364) outlines several poten- tial limitations: this paper doesn't It address the rigorous theoretical analysis of GAOT. The method has not yet been evaluated on real 3D datasets, and it has not been tested on down- stream tasks such as uncertainty quantification (UQ), inverse problems, or PDE-conse con-ttnsre optimization.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: We do not present theoretical results in our paper.
4. Experimental result reproducibility
Question: Does paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper states the detailed setup for generating experimental results and is provided in Supplemental Material sections B and C.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We have submitted our anonymized version of our code with the full paper submission. This includes the necessary details to set up the environment and run the experiments which will recreate the results from our proposed algorithm.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have illustrated the detailed training and testing setup in the SM sections
B.6 and C.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We provide sensitivity to training randomness in SM Sec. E.1I.
76

---

8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides the detailed computer resources needed to reproduce these experiments in the SM B.6.
9. Code of ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //te:://neuuee newrips.....pe.. cates.ccppuut cc/public Etthi iessGuuddeinness
Answer: [Yes]
Justification: The paper follows the NeurIPS Code of Ethics in every respect. Our research does not include the use of human subjects and focuses on models with minimal risk for safety and security concerns misuse.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The paper mentions wide- side-spread applications in science and engineering as positive impact. Moreover, our work is focused solely on scientific applications, with little to no ability used for potentially harmful or nefarious tasks such as the generation of false or misleading information or social engineering.
Justification:
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This work does not present any resources or artifacts with risks of misuse.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All the original owners of assets (e.g., code, data, models) are explicitly mentioned in the paper. Any resources presented for the first time, i.e. the aforementioned time-independent datasets, are owned by the authors and made publicly available under the appropriate license.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provide documentation detailing the parameters within the dataset, as well as how the data may loaded be used for use in other works.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
77

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve research with human subjects.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does involve LLMs any important, original, or non-standard components.
78

---

79


Reviews:
[{"review_id": "rzGFfSVypo", "reviewer": "Reviewer_SKSS", "summary": "The work proposes the geometry-aware operator transformer (GAOT) for predicting PDE solutions as a neural operator. The authors incorporated the multiscale attentional graph neural operator (MAGNO) to capture multiscale information, utilizing statistical information regarding geometry. Also, they used the patching strategies as in ViTs for faster prediction. The experimental evaluation suggests that the proposed method has the best or second-best accuracy among the selected baselines.", "questions": "See the weaknesses above.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n* The method showed good accuracy and computational efficiency compared to other neural operators.\n\nWeaknesses:\n* The reason behind the choice of formulations is not clear, making the method look ad hoc. The author needs to specify the concrete problems persisting in existing methods and explain the strategy to overcome them. For instance, more justification would be necessary for the choice of statistical descriptors, as well as the need for multiscale computation.\n* The writing is not clear. For instance, the Efficient implementation section is written in a qualitative manner, but almost no quantitative information is provided. The reviewer recommends adding some quantitative evaluation of the algorithm, such as computational complexity.\n* The choice of baseline is not complete. Judging from the motivation, there needs to be a comparison against pure-GNN-based PDE solvers such as meshgraphnet [Pfaff+ 2020] and MP-PDE [Brandstetter+ 2022] because these methods are also geometry aware. In addition, the comparison of the speed-accuracy tradeoff against classical solvers (with various spatiotemporal resolutions to control the tradeoff) is necessary.\n* The evaluation of the neural field property of GOAT (Table 3) is not clear because there is no other method with the same setting. The reviewer recommends adding baselines with the neural field setting.\n\nMinor points:\n* Equation 2 is confusing, since there is no x in PDEs. The author recommends using x in PDEs or rewriting the condition, e.g., writing $\\mathrm{in} \\ D$ instead of $\\forall x \\in D$.\n* $t$ should be an open set in Equation 2 because we cannot define a proper derivative on the boundary (that's why we need initial and boundary conditions).\n* SM is not clear at first glance. Using Appendix would be more understandable.", "was_revised": true, "final_justification": "The authors addressed all the concerns raised by the reviewer. Therefore, I would recommend acceptance.", "rebuttal": "We start by thanking the reviewer for their comments and suggestions. Below, we provide a detailed response to all of the reviewer's points of criticism.\n\n[1.]  Regarding the point about *reasons behind formulations*: we start by reiterating the main motivation for our paper: as clearly stated in the introduction (l41-50 and Fig.1), we observed an **accuracy-efficiency tradeoff** for all existing ML models for PDEs on arbitrary domains. In particular, accurate models such as GNNs (RIGNO) were not efficient and efficient models such as GINO were not as accurate. This motivates the design of a model that can be both efficient and accurate at the same time -- this is main premise of our paper. To do so, we observe that end-to-end GNNs such as RIGNO involve repeated sparse computations and memory accesses (l204-205) and will not necessarily be efficient and scalable on modern GPU hardware. Hence, we would like to minimize sparse computations. To this end, we focus on a paradigm, where the inputs are encoded to a latent grid, where they can be batch processed efficiently with transformers and then decoded to the desired output. Although our processor is a standard ViT, the encoder and decoder are designed to enhance accuracy by adding 2 novel features: *multiscale attentional GNO* and *geometric embeddings* based on statistical descriptors. \n\nThe reason behind a *multiscale GNO encoder-decoder* lies in the fact that solutions of PDEs contain a whole range of spatial scales (SM l764). Almost all the benchmarks that we consider in Tab. 1 contain multiscale solutions (see SM Eq D.5 for an explicit example in the Poisson-C-Sines dataset). Hence, adding multiscale features to the encoder-decoder equips the model with the *ability to sense different perceptive fields*, making it a reasonable choice in this regard. This is also justified by the ablation study presented in SM F.5 (Tab F.5), where we show that using a multiscale encoder-decoder is significantly more accurate than a single-scale version. \n\nWe have already discussed the reasons behind our novel *geometry embeddings* briefly in l134-140 and more extensively in SM l675-692. Repeating the main points, existing models either provide coordinates of input points or SDFs to encode geometric information. We argue that this does not suffice for problems where domain geometry itself plays an important role in the solution operator (e.g flow past airfoils) and hence additional geometric information needs to be provided to the model as done with the statistical embeddings based on neighborhood information (l140-149 and SM l693-719). The addition of these embeddings is also justified with the ablation study presented in SM F.4 (Tab F.4) where adding statistical embedding significantly increases accuracy over just inputting coordinates and SDFs.  We would be happy to elaborate on the above points further in a camera ready version (CRV) if accepted.\n\n[2.] Regarding the reviewer's concern about *lack of clarity regarding efficient implementation*, we begin by pointing out that our implementation of GAOT is the **key to achieving efficiency**, which is the second axis in our rationale for this paper. Following the reviewer's suggestion, we can readily include theoretical complexity estimates for GAOT and baselines in a CRV. Given E edges, C channels, T Latent tokens, p Patch size and L Layers, compute for GAOT scales as $O((E+L(T/p)^2)C)$, GINO as $O((E+LT\\log T)C)$ and GNN-based RIGNO as $O(LEC)$. Clearly, GAOT and GINO will outperform RIGNO when $E>>T$ (which can be ensured by design) as we see from Fig 3 (a). This also suggests GAOT will be competitive wrt GINO for large patch sizes. Yet, we show in Fig. 3(a) that GAOT is **even more efficient** than GINO. To achieve this was the key objective of our efficient implementation and we outline the main steps in l209-216. Following a detailed analysis of the compute cost of each individual component of GINO (pl. see the exact numbers in pt. [4] in the reply to Reviewer diAB below), we observed that for GINO: i) graph building is almost as expensive as encoding/decoding and ii) there is no batch processing due to the heterogeneity in the underlying graphs for PDEs on arbitrary domains, implying that the realized computational cost of GINO is not as low as the theoretical scaling would suggest. To alleviate these issues, as described in l209-216, we were able to i) move graph construction for GAOT outside the model evaluation and ii) implement batch processing in the GAOT transformer processor while sequentially processing inputs for the encoder/decoder. It is precisely these implementation tricks (which can only be described qualitatively and realized in the submitted code) which allow GAOT to be more efficient than GINO (pl. see exact numbers in pt [4] in the reply to Reviewer diAB below) and provide the context for the scaling results in Fig 3a and 3b. We are happy to add theoretical complexity estimates as well as a detailed compute breakdown in a CRV, if accepted and thank the reviewer for suggesting it. \n\n[3.] RIGNO (Ref. [39]) is a state of the art end-to-end GNN that *greatly improves upon MeshGraphNets* (MGN). In fact, MGN can be realized as a special case of RIGNO. As we considered RIGNO as a baseline, we did not include results with MGN. However, following the suggestions of the reviewer, we ran MGN on several of our benchmarks to obtain the following % relative errors: NS-Gauss (76.4 (MGN) vs 2.91 (GAOT)), NS-PwC (33.1 (MGN) vs 1.5 (GAOT)), NS-SL (18.8 (MGN) vs 1.21 (GAOT)), NS-SVS (76.4 (MGN) vs 6.4 (GAOT)), CE-RP (13.9 (MGN) vs 5.97 (GAOT)), Wave-Layer (84.8 (MGN) vs 5.78 (GAOT)), Wave-C-Sines (17.3 (MGN) vs 4.65 (GAOT)), Poisson-Gauss (30.9 (MGN) vs. 0.83 (GAOT)), Elasticity (11.9 (MGN) vs. 1.34 (GAOT)). All the experiments were run with 2M parameter MGN. Given the limited time of the rebuttal, we could only benchmark MGN on 10 of the 15 tasks of Tab. 1. But, the trend is crystal clear. GOAT is an order (sometime two orders) of magnitude more accurate than MGN. Moreover, one can see from Tab. 1 that RIGNO is also much more accurate than MGN. This justifies our choice of benchmarking against a state of the art GNN such as RIGNO rather than a more traditional MGN baseline. Nevertheless, we will include MGN results in a CRV as an additional baseline. MP-PDE has mostly been run on simple 1-D PDEs and was already shown to be significantly inferior to UPT, which have already benchmarked against (Tab. 1). \n\n[4.] The *neural field property* is a desirable feature for any ML surrogate for PDEs as it enables evaluation of the input and output fields at any point in the underlying spatial domain. In particular, it allows the model to be trained and tested at different resolutions, a highly desirable feature in PDE operator learning (Ref [22]). For example, with a neural field model, one can train at a coarser resolution (for training efficiency) but test at a higher resolution (for potentially more accuracy). It is this feature of GAOT that we wanted to demonstrate in Tab. 3 for the industrial scale 3-D Drivaernet++ dataset. However, as described in l296-301, having the neural field property does not imply that the model is more accurate. We can observe from Tab. 3 that the version of GAOT trained with only 10% of the total input points but tested at the full dataset resolution of 500K surface points using the neural field property is nowhere as accurate as the version of GAOT trained and tested at full resolution, although it is still better than some of the baselines. Thus, comparing with a neural field model on this dataset will not lead to more accurate results. Nevertheless, to further explore the consequences of model endowed with the neural field property, we observe that the UPT baseline provides a neural field. As testing UPT on an industrial scale dataset in the limited time of rebuttal is not possible, we considered the 2-D NACA0012 airfoil dataset with 2 different settings: i) training and testing GAOT and UPT at full resolution leading to relative % errors of 6.81 (GOAT) vs. 16.1 (UPT) and ii) training on 1/4th of the resolution and testing at full resolution leading to errors of 9.54 (GOAT) vs. 16.3 (UPT). This experiment reinforces that a) training at full resolution is simply more accurate than training at coarser resolutions b) GOAT is significantly more accurate than UPT in both settings and iii) UPT does not improve much with full resolution training, indicating lack of convergence with increasing resolution. We will gladly add these results together with a more detailed discussion of the neural field property in a CRV. \n\n[5.] We will address the pertinent minor points raised by the reviewer in a CRV, if accepted. \n\nFinally, in addition to the very low overall score, the reviewer has given our article the lowest possible score of 1 for originality and clarity. We take this opportunity to bring to your attention the fact that i) multiscale attentional encoder/decoder ii) statistical geometric embeddings and iii) multiple tricks for efficient implementation as described in pt [2.] of our reply above are completely novel to the best of our knowledge and are acknowledged as such by the other reviewers. These original contributions have enabled us to propose a model that is highly accurate, very efficient and scalable. We have demonstrated its state of the art performance on 24 different datasets, corresponding to a wide spectrum of challenging PDEs, including industrial scale 3-D benchmarks. To the best of our knowledge, no existing ML PDE surrogate model has the scalability, range and performance of GAOT. We hope that this rebuttal has allowed us to clarify our contributions accordingly and to reiterate the novelty, scope and performance of our model. Therefore, we kindly request the reviewer to upgrade their assessment accordingly and we are at your disposal to answer any further questions about our article."}, {"review_id": "R0vBDOz14e", "reviewer": "Reviewer_uofs", "summary": "The paper introduces Geometry-Aware Operator Transformer (GAOT) as a surrogate model that learns solution operators for PDEs defined on arbitrary, non-Cartesian domains. It combines a multiscale attentional graph neural operator (MAGNO) encoder, geometry embeddings, and a ViT-style token processor/decoder to map irregular point-cloud inputs into geometry-aware latent tokens and accurately reconstruct the physical field at any query point. Across 24 benchmarks it shows SOTA results while substantially improving training throughput and inference speed.", "questions": "None.", "limitations": "None.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths\n\n1. It is a solid work. Addresses the problem learning PDE operators on arbitrary (non-Cartesian) domains while remaining computationally tractable. \n2. The experiments are convincing as it introduces 24 different benchmarks and multiple popular SOTA methods as baselines. \n3. The pictures of the paper are clear and the paper is easy to read or follow.\n\nWeakness\n1. Is it well defined to be neural operators which are resolution-invariant?\n2. Caption of figures are too small.", "was_revised": false, "final_justification": "", "rebuttal": "We start by thanking the reviewer for their careful reading, appreciation of the merits of our paper and their comments and suggestions for improving it. Below we address their points of concern. \n\n[1.] Regarding the reviewer's question about **neural operators**, our claim is based on the fact that GOAT is endowed with a *neural field* property (l173-175) that allows it to process inputs on arbitrary point clouds and provide outputs on *any point* in the underlying domain. This enables GAOT to possess a key attribute for a neural operator, i.e., the ability to process inputs and outputs at arbitrary underlying grid resolutions. Moreover, works such as Refs. [22,4] argue that this attribute does not suffice for a neural operator and it should possess some (approximate) *resolution invariance* for genuine operator learning. We test this attribute for GAOT in SM E.5. In this context, Fig E.5 clearly demonstrates how GAOT can generalize seamlessly to test resolutions, different from the training resolution. Thus, our labelling of GAOT as a neural operator is justified in this sense. \n\n[2.] We will gladly increase the size of the figures in a camera-ready version (CRV), if accepted. This will be aided by the additional page that the CRV is allowed to have. \n\nWe take this opportunity to thank the reviewer again for their constructive comments and suggestions."}, {"review_id": "q1rTuVnMiq", "reviewer": "Reviewer_diAB", "summary": "This paper presents GAOT, a novel neural neural operator approach for solving time-dependent and independent PDEs on arbitrary domains. The authors proposed a new transformer approach for solving PDEs in an efficient way, when dealing with PDEs discretized over dense grids. Notably, they propose a new multi-scale approach inspired by GINO for encoding arbitrary point clouds into a regular (or not) latent grid. Their encoding and geometry embedding approach allows them to patch and thus reduce the number of physical tokens, reducing the computational cost for their transformer that does the time-stepping operation, trained in a all2all fashion. They notably evaluated their approach on a large number of challenging datasets, with time-dependent and independent problems with regular and irregular grids. Through their experiments, they exhibit competitive or sota performance while remaining computationally effective.", "questions": "In the contributions, you mention that you use a *coarser* latent grid. In the appendix, it is mentioned that the size is $64 \\times 64$. It is therefore a relevant compression rate when dealing with PDEs discretized over a $128 \\times 128$ grid, but for denser grids, how restrictive is this choice? In other words, how does the latent grid size influence the performance of the model?\n\ni am a bit concerned by the training throughput results you have presented. From my knowledge, GINO encoding is particularly costly and needs to proceed each sample observation one by one. This seems to be costly as you cannot vectorize the operation. In addition, you also different scales, which means you have to also add a new loop depending on the number of scales considered. I don't really see how you still manage to be more efficient to other NO approaches.\n\nFor the Transolver and UPT, you mentioned that it was too expensive to train it an all2all fashion. I don't really see why it is that expensive? For instance, for UPT, you can control the number of super-nodes to consider and adjust it such that the number of tokens match the number of tokens used for GAOT no?\n\nFor GAOT, if I am correct, you embed physical fields into a 2D latent grid of shape $64 \\times 64$ so this means after the patchify operation (2 according to appendix) you get 1024 tokens. In UPT, you said you use 2048 super-nodes. Decreasing the number of super-nodes wouldn't make it work for time-dependent data? \n\nThe same could apply for Transolver where we could decrease the number of slices, I guess. Also an extension has been proposed to tackle problems with denser grids [3]. I think it could be mentioned somewhere as it relates to neural solver efficiency.\n\n[3] Luo et al, Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries, arXiv preprint arXiv:2405.13998", "limitations": "Limitations have been addressed. I might have miss the information as they mentioned they addressed negative social impacts but I did not find any broader impact section.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The paper's quality is good and the experiments clearly highlight the benefits of GAOT over existing baselines. The number of experiments, datasets and baselines considered is huge. They proposed extensive ablations in the appendix and provide a lot of technical details to explain their approach in the SM. \n\nWhile learning neural operators is a well-known research direction, GAOT seems to be a new effective solution for solving PDEs in a cost-effective way while still being more accurate. They notably compared the computational costs of their method with respect to existing baselines when increasing the number of model parameters, and provided experimental results showing that GAOT seems faster to be trained and to infer new PDE solutions.\n\nThis might not be considered as an inconvenient to some readers/reviewers, but I feel like the main paper is just a succession of small descriptions of the method, but it is very difficult to get a correct understanding from it. Most of the technical details are actually in the SM which is very large. \n\nRegarding the novelty of the paper, it comes from the multi-scale approach that extends GINO and the statistical embeddings used for better encoding geometries. The processor is a classical vision transformer with a patchify operation as in [1] with an all2all approach as done in Poseidon [2].\n\n[1] Wang et al, CViT: Continuous Vision Transformer for Operator Learning, ICLR 2025.\n[2] Herde et al, Poseidon: Efficient Foundation Models for PDEs, NeurIPs 2024.", "was_revised": true, "final_justification": "I think the authors provided an interesting approach that is both computationally efficient and very accurate. The authors answered some of concerns and recognized that the computational efficiency of their method is not specifically due to the model architecture but to relatively straightforward implementation tricks that could be used on GINO.\n\nIn its current form, the paper's contributions are a bit over-claimed to me, preventing me to put a higher score and only recommend a borderline accept. \n\nStill, I think the paper could be accepted, because the experiments are very consequent and the method very effective. If they provide the revision needed, it can be a good paper valuable for the neural operator literature.", "rebuttal": "We start by thanking the reviewer for their careful reading of our paper, their appreciation of its merits and their suggestions for improving it. We provide a detailed response to their points of concern below.\n\n[1.] Regarding the reviewer's point about *clarity of exposition*, we would be happy to significantly elaborate on the overview section (l102-109) in the main text in a camera-ready version (CRV), if accepted. This will allow us to better inform the reader on how our model is designed. \n\n[2.] We will add a reference to CVIT paper in a CRV as it is indeed relevant. We agree with the reviewer that the *multiscale encoder-decoder* and *statistical geometric embeddings* are our main novel contributions at the level of model design. \n\n[3.] On the reviewer's question about *different choices of the latent grid*, we would like to point to SM F.2 (l1344-1358), in particular Tab. F.2, where we have precisely explored this question. From this table, we observe that reducing the latent grid size does affect the overall accuracy to a minor extent, but not in any significant manner, particularly for the correct patch sizes. This is consistent with our heuristic that the main point of the latent grid should be to provide adequate coverage of the physical domain. As long as it holds, even coarser latent grids are admissible. Summarizing, GAOT seems to be very robust to varying latent grid size, radii of aggregation and patch size. We will highlight this point further in a CRV, if accepted. \n\n[4.]  The reviewer is absolutely spot on questioning the *computational costs of GINO*. To further quantify your concerns, we ran GINO (6.07 M params) for the NACA0012 dataset on a NVIDIA-4090 GPU to obtain the following breakdown of compute time in ms for GINO for a single sample:  Encoder (Graph  building: 1.984, GNO: 2.793, Total: 4.777), Processor (5-Layer FNO): 5.89, Decoder (Graph Build: 1.984, GNO: 1.140, Total: 3.124). Total GINO run time per sample = 13.791 ms. However, as the reviewer very astutely pointed out, **GINO does not support batch processing** when the underlying graphs change in a batch (as in general for PDEs on arbitrary domains). Thus to process a batch of 32, GINO has to loop over the entire batch, leading to a runtime of 441.32 ms ! Thus, we observe 2 computational bottlenecks here: i) Graph building in the encoder/decoder is as (or more) expensive than the GNO module and ii) lack of batch processing severely impedes computational throughput. Our challenge was to overcome these bottlenecks. \n\nTo this end, we came up with the following tricks: First, we take graph building out of the GAOT model evaluation loop to reduce computational wastage by precomputing the graphs and storing them as cache files and loading with efficient data loaders. Next to enable batch processing, initially we explored vectorizing the encoder and decoder operations. However, this approach proved impractical for two key reasons. First, the grid sizes within a batch may exhibit significant variations, complicating efficient vectorization. Second, for relatively large graphs, vectorization introduces an operation that triggers a transient spike in memory consumption, which can cause the overflow of GPU memory. Consequently, straightforward vectorization is still neither memory-efficient nor scalable. To address this, we adopted a sequential processing strategy for the encoder and decoder while batch processing the transformer. Importantly, this only introduces a lightweight loop whose overhead is negligible compared to the transformer processor, ensuring that it does not become a computational bottleneck when the processor is scaled.\n\nWith these implementation tricks, we have the following compute breakdown for a 5.62M GAOT  (same setting as in GINO) for a batch size of 1: Encoder (AGNO: 2.296, GeoEmb: 3.412, Total: 5.708), 5-Layer ViT processor with patch size 2: 10.1, Decoder (AGNO:1.356, GeoEmb:1.865, Total: 3.221). Total GAOT run time per sample=19.029 ms. Thus, as the reviewer anticipated GAOT is slower than GINO for a batch size of 1 as the multi-scale GNO and GeoEmb embeddings add cost although Graph building has been taken out of the loop. However, the results for a batch size of 32 are: Encoder (AGNO+GeoEmb) = 126 ms, Transformer processor = 30.1 ms, Decoder (AGNO+GeoEmb) = 107 ms, leading to an overall batch compute time of 263.1 ms, which is *178 ms* faster than GINO !!. The payoff happens in the **batch processing of the transformer** as this module only increased from 10.1 ms (batch size 1) to 30.1 ms (batch size = 32). It was precisely this novel trick that allows GAOT to be more efficient than GINO, while being much more accurate. \n\nMoreover, this set of implementation tricks become even more impactful when the input size and model size is scaled up. As this happens, the memory bottleneck in the encoder/decoder and the compute usage in the processor grows proportionately. Consequently, moving graph building outside the encoder/decoder and batch processing the transformer processor makes GAOT scale significantly better than GINO in both input size and model size as we observe from Figs. 3a and 3b. \n\nAs a final trick for very large (particularly 3-D) graphs, we also introduce an edge masking procedure to further reduce memory bottlenecks in the encoder/decoder (l214). \n\nFollowing the reviewer's excellent suggestion, we will elaborate on this point in a CRV and provide the above detailed computational breakdown such that readers can better appreciate the effort we put into optimizing our implementation for efficiency and scalability. \n\n[5.] As the reviewer correctly claims, UPT is a scalable model. Our problem with UPT is that it lacked stability while training, making convergence very difficult, if not impossible. Despite considerable efforts, we could not make UPT (particularly for larger model sizes) converge for many for our time-dependent datasets. Hence, we did not present these results in the main text. However, we obtained the following relative % errors for the time-dependent datasets of Tab. 1 : NS-Gauss (did not converge (UPT) vs. 2.91 (GAOT)), NS-PwC (did not converge (UPT) vs. 1.5 (GAOT)), NS-SL (51.5 (UPT) vs. 1.21 (GAOT)), NS-SVS (4.20 (UPT) vs. 0.46 (GAOT)), CE-Gauss (64.2 (UPT) vs. 6.4 (GAOT)), CE-RP (26.8 (UPT) vs. 5.97 (GAOT)), Wave-Layer (19.6 (UPT) vs. 5.78 (GAOT)), Wave-C-Sines (12.7 (UPT) vs. 4.65 (GAOT)). Together with the time-independent results from Tab. 1, we see that GOAT is often one to two orders of magnitude more accurate than UPT.  We are happy to add these UPT results in a CRV, if accepted. \n\n[6.] Regarding Transolver, we found this model to be very slow computationally. In our understanding, this is because the Transolver model, although it also adopts an encoder–processor–decoder architecture, this architecture is applied at the level of a single Transolver block and there is no sequential processing of information in the latent space. As a result, when stacking many Transolver blocks, the model repeatedly executes both the encoder and the decoder. Since the encoder and decoder each time need to establish mappings between the input grid size and the latent tokens, this leads to a substantial amount of computation (as input grids are typically large) and significant memory overhead. Moreover, the computations in the encoder and decoder cannot fully exploit the hardware’s throughput capabilities, and the large memory footprint forces us to use very small batch sizes as soon as the input grid size becomes slightly larger. Consequently, the model struggles to achieve high throughput as we observe from Figs 3a and 3b. Using the original setup described in the Transolver paper, we found it nearly infeasible to train time-dependent datasets (each with around 20,000 samples) in an all-to-all manner on our hardware.\n\nTransolver++ was published concurrently with the submission of our article, preventing us from using it as a baseline. Although Transolver++ also benchmarks on the Drivaernet++ dataset, their model is trained with only 200 of the total of 8000 samples, making it impossible for us to compare with their results as more data will surely lead to more accuracy. Instead, we have compared with models that sit on top of the Drivaernet++ leaderboard to emerge as the best performing model for this entire dataset.  Nevertheless, we do agree with the reviewer that Transolver++ is a contribution to neural PDE surrogate efficiency and we will refer it in a  CRV, if accepted. \n\nWe take this opportunity to thank the reviewer again and hope that we have satisfactorily addressed their concerns, particularly regarding the computational efficiency of GAOT and its comparison with UPT and Transolver. If so, we request the reviewer to kindly upgrade their assessment accordingly, while remaining at your disposal to answer any further questions about our paper."}, {"review_id": "PLkt5Mdyo3", "reviewer": "Reviewer_Kj85", "summary": "The proposed paper presents a novel architecture for PDE solving on arbitrary geometries. The model is built-upon on a combination of several architectures (GINO, ViT…). The novelty lies in an improved multiscale GINO encoder with geometry embeddings. The model is evaluated on a lot of datasets and compared to several baselines.", "questions": "-\tAs the claim of the model is its scalability, I couldn’t find a comparison of the training time required for the proposed method wrt the baselines. This is also an important indicator. I found the training throughput (baseline) and the training time for one batch (but only for GAOT). \n-\tDoes your time stepping strategy allows for irregular queries in the time dimension ? \n-\tLine 203  is stated that « the computational burden should fall on the processor ». Could you elaborate on this aspect? Do you have experiments/explaination about this ? If so, why using encoder/decoder ? Isn’t it usefull to compress information to make model more compact and thus more scalable for real-world applications?  \n-\tHow would behave the ViT processor (let’s say on a regular grid dataset) with and without the encoder/decoder proposed ? My point here is to isolate the improvement from the encoder/decoder.\n-\tHave you considered to evaluate the performances of the AE alone ? since the contribution relies strongly on this part of the model, I think you should evaluate reconstruction performances compared to other baselines.", "limitations": "Yes a limitation section is added. However, I only see discussions about extensions and future works in this part. What limitations has GAOT?", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strength\n -\tThe numerous experiments: the model is evaluated on a lot of datasets, in several settings (regular/irregular grids, time dependant and independant PDEs…)\n -\tThe paper is well-writen and despite the rich content, the reader still understands the method\n -\tThe paper proposes several interesting studies and ablations on the proposed model (time stepping strategies, discretisation…). \n\n### Weaknesses\n -\tNovelty : the architecture is built upon several well-known components : GINO+ViT. \n -\tLimited related work : the related work is only a paragraph at the end of the paper. Due to the numerous existing references, I think this part is important in a paper. The related work can be added in the appendix, when limited space is available in the main part. This could help the reader to clearly see the similarities and differences with other existing methods. Some references are provided in the introduction, but no clear positionnig of the paper is provided. \n -\tMissing references : a few references are missing in my opinion eg [1, 2, 3]. [1] : While you discuss about UPT and transolver, you are missing AROMA which is also using transformers+neural field model. [2] : As the purpose of the model is to scale on large geometries, I think that you should consider baselines with similar objectives, which is the case of Transolver ++, an extension of transolver that is design to handle large geometries (and compares on DrivAernet ++). [3] : Matching irregular to regular grids for the encoder has been studied is several models such as Text2PDE, that makes use of a GINO + CNN encoder/decoder architecture. \n- Finally, I couldn't identify which part of the model greatly helps in reducing the computation cost. Compared to GINO, what explains that GAOT is faster? \n\n\n[1] AROMA: Preserving Spatial Structure for Latent PDE Modeling with Local Neural Fields, Louis Serrano, Thomas X Wang, Etienne Le Naour, Jean-Noël Vittaut, Patrick Gallinari, NeurIPS 2024\n\n[2] Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries, Huakun Luo, Haixu Wu, Hang Zhou, Lanxiang Xing, Yichen Di, Jianmin Wang, Mingsheng Long, ICML 2025\n\n[3] Text2PDE: Latent Diffusion Models for Accessible Physics Simulation, Anthony Zhou, Zijie Li, Michael Schneier, John R Buchanan Jr, Amir Barati Farimani, ICLR 2024", "was_revised": true, "final_justification": "The main contribution of the paper lies in the cost–accuracy trade-off achieved by GAOT, which results from the careful combination and optimization/engineering of several existing architectures. The method is evaluated on a wide range of standard datasets, and ablation studies are included to empirically demonstrate the properties and effectiveness of GAOT.\nSeveral concerns were addressed during the rebuttal, more specifically adjusting some comparisons with existing methods [1, 2, 3, 9], as well as additional ablations to isolate the contribution of GAOT's components [5, 7, 8]. \nTo conclude, with the suggestions and clarifications provided in the rebuttal, I decided to raise my score to 4.", "rebuttal": "We start by thanking the reviewer for their careful reading of our paper, their appreciation of its merits and their suggestions for improving it. We provide a detailed response to their points of concern below.\n\n[1.] Regarding *novelty of our model*, while the reviewer is correct that the ViT processor in GAOT is not novel nor is the encode-process-decode paradigm, we reiterate from l41-55 (Main Text)  and Fig. 1 that there is an **accuracy-efficiency** trade-off for Neural PDE surrogates and accurate GNN type models (RIGNO is state of the art in this family) are not efficient while efficient models (GINO) are not as accurate. So, our main rationale for this paper was to provide a model that could be efficient and accurate at the same time. While combining available elements, we did need to come up with 3 key *novel features* to do this: i) multiscale attentional GNO for encoding/decoding ii) geometric embeddings based on statistical descriptors to enhance geometric inputs to the model and iii) several tricks allowing for an optimized implementation (see pt. [3] below). To the best of our knowledge, these are original contributions and they clearly enhance model performance. See Tab. F.5 and reply pt [8] below, for results on how multiscale information in the encoder/decoder significantly improves upon a single-scale version, Similarly, see Tab F.4. to observe how statistical geometric  embeddings improve accuracy over just inputting coordinates and SDFs. Finally, the role of efficient implementation is described below in pt [3]. \n\n[2.] We acknowledge the reviewer's point that *related work* is rather succinct, primarily due to page limits for the main text. Following the reviewer's suggestion, we are happy to elaborate on the related work section in a camera-ready version (CRV) esp. SM, if accepted. In particular, we thank the reviewer for bringing Refs. [1,3] to our attention and we will gladly add them to related work. Regarding Transolver ++, we will certainly add it with the context described in our reply pt [6.] to reviewer diAB above. \n\n[3.] To answer the reviewer's pertinent question on **why is GAOT faster than GINO** ?, we ran GINO (6.07 M params) for a single sample of the NACA0012 dataset on a NVIDIA-4090 GPU to obtain the following breakdown of compute time in ms for GINO:  Encoder (Graph  building: 1.984, GNO: 2.793, Total: 4.777), Processor (5-Layer FNO): 5.89, Decoder (Graph Build: 1.984, GNO: 1.140, Total: 3.124). Total GINO run time per sample = 13.791 ms. As it happens, **GINO does not support batch processing** when the underlying graphs change in a batch (as in general for PDEs on arbitrary domains). Thus to process a batch of 32, GINO has to loop over the entire batch, leading to a runtime of 441.32 ms ! Thus, we observe 2 computational bottlenecks here: i) Graph building in the encoder/decoder is as (or more) expensive than the GNO module and ii) lack of batch processing severely impedes computational throughput. Our challenge was to overcome these bottlenecks. \n\nTo this end, we came up with the following tricks: a) we take graph building out of the GAOT model evaluation loop to reduce computational wastage by precomputing the graphs and storing them as cache files and loading with efficient data loaders and b) batch processing the transformer processor while looping over the encoder/decoder sequentially. Please see our reply pt [4] to reviewer diAB above for further details on these tricks. \n\nWith these implementation tricks, we have the following compute breakdown for a 5.62M GAOT  (same setting as in GINO) for a batch size of 1: Encoder (AGNO: 2.296, GeoEmb: 3.412, Total: 5.708), 5-Layer ViT processor with patch size 2: 10.1, Decoder (AGNO:1.356, GeoEmb:1.865, Total: 3.221). Total GAOT run time per sample=19.029 ms. Hence, GAOT is slower than GINO for a batch size of 1 as the multi-scale GNO and GeoEmb embeddings add cost although Graph building has been taken out of the loop. However, the results for a batch size of 32 are: Encoder (AGNO+GeoEmb) = 126 ms, Transformer processor = 30.1 ms, Decoder (AGNO+GeoEmb) = 107 ms, leading to an overall batch compute time of 263.1 ms, which is *178 ms* faster than GINO !!. The payoff happens in the **batch processing of the transformer** as this module only increased from 10.1 ms (batch size 1) to 30.1 ms (batch size = 32). It was precisely this novel trick that allows GAOT to be more efficient than GINO, while being much more accurate. As a final trick for very large (particularly 3-D) graphs, we also introduce an edge masking procedure to further reduce memory bottlenecks in the encoder/decoder (l214). We will add this quantitative computational breakdown in the CRV to better inform the readers on how our model achieves computational efficiency. \n\n[4.] As total number of training samples in each dataset is exactly the same for all models, training time per epoch = (Total Number of Samples)/Training throughput, scales exactly as the training throughput for each dataset. Hence, GOAT is 1.6 times faster to train than GINO from Tab E.1. We can also provide the exact training times for each model in a CRV, if accepted. \n\n[5.] Regarding the reviewer's question about *irregular queries* in time, our model provides the current time t and lead time $\\tau$ as inputs (l191), allowing it to be evaluated at any query point in time and at any query point in space (neural field property). To demonstrate this feature, we consider the NS-PwC dataset, where we had used training data at time stamps $t=0.0,0.1,0.2,0.3,0.4.0.5,0.6,0.7$. At test time, we output the velocity fields at times $t=0.35$ and $t=0.45$, which are not included in the training set to obtain relative % errors of 1.4 and 1.91, respectively. This interpolates very nicely with errors of 1.2 ($t=0.3$), 1.65 ($t=0.4$) and 2.2 ($t=0.5$). We thank this reviewer for this suggestion and will add the results in a CRV. \n\n[6.] Perhaps l203 was not clearly articulating our point, which was to say that we would like the compute of the processor to scale very favorably with increasing number of layers as we clearly show in the argument presented in pt [3] above. Clearly the encoder-decoder are necessary to compress information but we want them to be as lightweight computationally as possible as they don't scale as well as the processors. We will adjust this sentence accordingly in a CRV, if accepted. \n\n[7.] The reviewer's aim to understand the precise role of encoder/decoder vs. processor in our model is very much appreciated. We had infact presented results on a Cartesian grid in SM Tab E.2, where we compared GOAT to standard Cartesian grids models such FNO, CNO and the SWIN transformer based scOT, to observe that GAOT was clearly more (or as) accurate as the dedicated cartesian grid architectures on problems on Cartesian grids. Following the reviewer's excellent suggestion, we ablated GAOT further by just running the ViT processor (without the encoder/decoder) on these Cartesian datasets to obtain the following relative % errors: NS-Gauss (ViT (3.16) vs GAOT (2.29)), NS-PwC (ViT (3.89) vs. GAOT (1.23)), NS-SL (ViT (0.73) vs. GAOT (0.98)), NS-SVS (ViT (0.4) vs. GAOT (0.46)), CE-Gauss (ViT (6.81) vs. GAOT (5.28)), Wave-Layer (ViT (5.48) vs. GAOT (5.40)), Poisson Gauss (ViT (2.47) vs. GAOT (0.79)). From these results, it is clear that in some cases, ViT is comparable to or slightly better than GAOT, indicating that the power of GAOT in these cases stemmed from a good processor. However in many more cases, **GAOT is significantly superior to ViT** also showing that the encoder/decoder contribute significantly to expressivity and together, this combination achieves SOTA performance. This interesting experiment clearly delineates the relative contributions of encoder/decoder vs. processor and we thank the reviewer again for asking us to perform it. Needless to say, we will include it in a CRV, if accepted. \n\n[8.] To answer the reviewer's interesting question about if the GAOT encoder-decoder can work as *autoencoders* (AE), we consider the output distribution of the NACA0012 compressible flow dataset. As shown in SM Fig. G.4, second column from left, this is a challenging distribution to learn as it contains sharp shocks separated by smooth regions in the flow and there is considerable variation when the increasing Mach number transitions the flow from sub-sonic to supersonic. Thus, it is worthwhile to see how the MAGNO + geometric embeddings based encoder-decoder of GAOT fairs as an AE. As a baseline, we chose GINO with its GNO encoder/decoder as the only point of difference now is the multi-scale attentional features in MAGNO compared to the single-scale GNO and the geometric embeddings. The resulting relative L1 reconstruction errors are 0.08 (GAOT) vs 0,11 (GINO) showing that the MAGNO encoder-decoder+ geometric embeddings are clearly more powerful than a vanilla GNO. We had already shown the advantage of MAGNO in SM Tab F.5 and geometric embeddings in SM Tab F.4 and this experiments further reinforces these advantages. We thank the reviewer for this interesting suggestion and will be happy to add this point to a CRV. \n\n[9.] We don't see any empirical limitations with GAOT given its very broad scope, scalability and excellent performance. However,, as we wrote on l364, we have not shown any rigorous theoretical results such as an universal approximation property etc. This is a limitation that we hope to address in future work. \n\nWe take this opportunity to thank the reviewer again and hope that we have satisfactorily addressed their concerns, particularly regarding the computational efficiency of GAOT and the contribution of individual elements (encoder/decoder vs. processor). If so, we request the reviewer to kindly upgrade their assessment accordingly, while remaining at your disposal to answer any further questions about our paper."}]

Author rebuttal:

Meta-review: This paper introduces GAOT (geometry-aware operator transformer) to combine an encoder that can handle arbitrary geometry in the gridded data and ViT-based layers for the latent tensors (with a user-chosen topology for the latent grid). Experiemental-wise, this is a very good paper. Please acknowledge the references using similar approaches that the reviewers mentioned such as CViT, Poseidon, and Transolver++. I also suggest adding some earlier works on Transformer neural operators that exploit lower latent dimensions, such as Galerkin or OFormer. In the meantime, multilevel feature aggregation (formula (4)) has been proposed in the Multigrid NO paper (ICLR 2024) using multigrid prolongation and restriction.

BTW: personally I am not a big fan of using the radar chart that is commonly seen in LLM papers to illustrate the performance of different neural operators.
Decision: Accept (poster)