From Cheap Geometry to Expensive Physics: A Physics-agnostic Pretraining Framework for Neural Operators
Authors: Zhizhou Zhang
Youjia Wu
Kaixuan Zhang
Yanjia Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Neural Operator
Pretraining
Physics-agnostic
Partial Differential Equation
Autoencoder
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Industrial design evaluation often relies on high-fidelity simulations of governing partial differential equations (PDEs). While accurate, these simulations are computationally expensive, making dense exploration of design spaces impractical. Operator learning has emerged as a promising surrogate for high-fidelity physical simulations, enabling fast prediction of partial differential equation (PDE) solutions. However, accuracy of neural operators is largely affected by the amount of training data, which must be generated by expensive numerical solvers. In practical industrial scenarios, there exists large collections of candidate geometries remain unsolved due to the high computation cost. These geometry-only samples contain no physical field labels and are therefore ignored in standard operator learning pipelines. In this work, we propose a general physics-agnostic pretraining framework to exploit this abundant geometric resource to improve the performance of neural operators. Specifically, we pretrain an autoencoder on a self-supervised proxy task to reconstruct geometry (e.g., via occupancy), learning an expressive latent representation without PDE supervision. Neural operators then leverage the pretrained latent embedding to learn more effectively from limited physics labels. An error decomposition analysis is provided to help understand the effectiveness of the physics-agnostic pretraining framework. Across four PDE datasets and three state-of-the-art transformer-based neural operators, our pretraining strategy consistently improves prediction accuracy. These results demonstrate that representations from physics-agnostic pretraining provide a powerful foundation for data-efficient operator learning. The code is publicly available at: https://github.com/zzzwoniu/Physics-agnostic-Operator-Pretraining.git

Published as a conference paper at ICLR 2026
FROM CHEAP GEOMETRY TO EXPENSIVE PHYsICS: A PHYSICS-AGNOSTIC PRETRAINING FRAMEWORK FOR NEURAL OPERATORS
Zhizhou Z Youjia Wu', Kaixuan Zhang', Yanjia Wangh.2
Bosch (China) Investment Co., Ltd.
University of Hong Kong {zhizhou. zhang, youjia. wu, kaixuan, (hannnnnnnanneec.. bosch. ansleyweconnect. hku.hk
ABSTRACT
Industrial design evaluation often relies high-fidelity simulations of governing partial differential equations (PDEs). While accurate, these simulations are com- putationally expensive, making dense exploration of design spaces impractical. Operator learning has emerged as a promising surrogate for high-fidelity physical simulations, enabling fast prediction of partial differential equation (PDE) solu- tions. However, accuracy of neural operators is largely affected by the amount training data, which must be generated by expensive numerical solvers. practical industrial scenarios, there exists large collections of candidate geome- tries remain unsolved due to the high computation cost. These geometry-only samples contain no physical field labels and are therefore ignored in standard operator learning pipelines. In this work, we propose a general aggsosi- pretraining framework to exploit this abundant geometric resource improve the performance of neural operators. Specifically, we pretrain an autoencoder self-supervised proxy task reconstruct geometry (e.g.. via occupancy), learning an expressive latent representation without PDE supervision. Neural operators then leverage the pretrained latent embedding learn more effectively from limited physics labels. An еттог decomposition analysis is provided to help understand the effectiveness of the physics-agnostic pretraining framework. Across four PDE datasets and three state-of-the-an transformer-basee neural operators, our pretraining strategy consistently improves prediction accuracy. These results demonstrate that representations from physics-agnostic pretraining provide pow- erful foundation for data-efficient operator learning. The code publicly available at: https::/gihub..com/zzuuua http://hww....mm////////////s/s/ opprtttttrt Operator- Pretr--rrrrrtggppe.
INTRODUCTION
Partial differential equations (PDEs) are fundamental descriptors for various physical and engineering systems (Guenther & Lee 1996). Traditionally, PDEs are solved numerically by various discretization methods such as finite element, finite difference, finite volume, etc. (Reddy) 1993) Perrone & Kao 1975) which offer high accuracy at high computational cost (Greenspan) 1955). This costly simulation process can be significantly accelerated by training surrogate neural operators to learn mappings from input functions to PDE solutions. As the pioneering work on neural operators, DeepONet adopted MLP architectures and established theoretical foundations for operator learning (Lu et al. 2021).
Kernel-based operators (Li et al.) 2020a bb) were then proposed to better handle resolution invariance. Recent studies focus transformer-based neural operators (Cao 2021: Hao al. 2023 Wu et al.)
2024) to handle complex irregular geometry inputs and further improve computational efficiency.
Despite architectural progress, the accuracy of neural operator still depends heavily on labeled PDE solutions produced by costly simulations. Considerable effort has therefore been devoted to improving data efficiency. For example, Cheng et al. (2024) predicts solution deviation relative to
"Corresponding zhou. wwwnggaannnnaoo.. www.b.bchhh............o...c.... com com

---

Published as a conference paper at ICLR 2026
reference field to simplify the learning problem, and Deng et al. (2024) couples neural operators with a Point-BERI pretrained on ULIP-2 to inject geometric priors from large scale surface point clouds. However, these approaches assume particular problem structures, data modalities, or supervision sources, which can limit their general applicability.
On the other hand, operator pretraining has become active research direction for enhancing PDE data efficiency. Most existing pretraining strategies are physics- and can be broadly grouped into two categories: The first pretrains models to reconstruct PDE solution fields inject inductive bias (Serrano et al. 2024 Rahman et al., 2024). The second performs large-scale pretraining and fine-tuning over families of related PDE dynamics, often in autoregressive settings (Hao al.) 2024, Herde et al., 2024 McCabe et al., 2024). While effective, these pretraining techniques still rely on PDE solutions, which remain the primary computational bottleneck in industrial practice. By contrast, geometry-only data (meshes without solver labels) are plentiful yet underused. This physics-agnostic resource offers an untapped path improve operator learning under label scarcity.
Physics-agnostic Pretraining
Operator Learning
Scarce
Expensive
Data with geom physics information
goom information only
Cheap
Figure Data availability and usage in the proposed physics-agmne agnnosic pretraining framework. Pretraining leverages the abundant geometry-only data, while operator learning stage only observes limited PDE labeled samples.
We propose a agysics-agnostic pretraining framework to exploit abundant geometric information while requiring PDE labels only during supervised operator training (Fig. [ Specifically, we pretrain a point-cloud variational autoencoder (VAE) to extract latent geometry representations using large point-cloud/mesh) dataset. A physics-agnostic proxy task is introduced the reconstruction objective for VAE training without requiring any PDE labels. After pretraining, neural operators are conditioned on these latent geometry embeddings rather than raw point clouds when predicting PDE solutions. As the pretraining stage is exposed to far more geometries than labeled PDE solutions, the encoder learns robust geometric structure and mitigates information loss from downsampled point clouds, yielding more informative representations and improving data efficiency on scarce PDE datasets, Our main contributions are:
We introduce novel physics-agnostis pretraining framework for operator learning tasks. Pretraining is performed on unlabeled point clouds via proxy task to learn latent geometry representation. Neural operators then trained predict PDE solutions from these latents. This scheme allows full exploitation of geometry-only data without PDE labels.
We introduce occupancy reconstruction as the objective pretraining, allowing physics- agnostic self-supervised representation learning on irregular geometries realized as point clouds. The selection of proxy task is flexible, which can be swapped for signed distance fields, shortest vector fields, and other choices.
We examine the proposed pretraining framework across four PDE problems and three transformer-basee neural operators. Experiments show consistent performance gain when compared with direct single stage supervised training.
PRELIMINARIES
In this section, we define the problem and data settings. Then we briefly summarize related works on operator learning and pretraining.
2.1 PROBLEM SETUP
We consider PDE problems defined in the domain C. Rd to be approximated with learned operator F A + U where A represents the input function space and U represents the solution function space
2

---

Published as a conference paper at ICLR 2026
Query
Fixed learaale latent
Geom-only Pretraining
Encoder (pereeiver]
Decoder
Data-rich
Point cloud
Latent geometry representation
Occuannc
Operator
Geometry input
Branch
Learning
Transformer based neural operator
Mesh/Random
query
Trunk
Data-scarce
query physics field
query physics field
Figure 2: An illustration our pretraining framework. We pretrain an encoder via a physics-agnostic proxy task (for instance the occupancy field) extract information from rich geometry-only data. Unlike masked autoencoder, we use Conditioning on the encoded embeddings, neural operators are trained with limited physics data to predict PDE solution fields.
to allow arbitrary query coordinates.
of the physics over general scenarios, A may consist of various types information such as object geometries (material distribution), boundary conditions, source functions, etc. For simplicity, the input function space is assumed to contain only object geometry information, and the solution function is assumed to be scalar valued in this work. A realization of the geometry information ag E represented as a point cloud (nodes mesh) a = {xi) = Xx R Nxd with N points in domain Ω. And the physics solution function E U is realized as {vl. ) with 2i (v()), where {v} ? is set of query points.
In this work, we consider two qualitatively different portions D and from each dataset:
D {(aR: ) D' = ((0)))
|D'] > |D]
(1)
Here, D is the physics portion with ground truth fields produced by numerical solver, while D the geometry-only portion that is abundant. This is highly related many industrial scenarios where geometry designs easy generate, but physical field labels are limited. Motivated by such scenarios, we aim to leverage D' to improve prediction accuracy on D, thereby alleviating the persistent data-scarcity bottleneck in operator learning.
2.2 RELATED WORK
Neural Operators. Operator learning aims to establish mapping from input functions to solution functions. DeepONet established theoretical foundations for operator learning (Lu et al. 2021) 1).
followed by two important milestones: Graph Neural Operator (GNO) (Li et al. 2020b) and Fourier Neural Operator (FNO) (Li et al., 2020a). A set of follow-up works further extended GNO and FNO to irregular geometries (Li al. 2023b a and improved computation efficiency (Guibas et al. 2021 Tran et al. 2021). With the rise of attention mechanism, architectural design shifted toward transformers (Cao 2021 et al. 2022). where spatial locations and field values are tokenized as query, key, and value sequences. General Neural Operator Transformer (GNOT) Hao et al. (2023) introduced heterogeneous cross-attention to handle versatile input functions and gating mechanism to adjust network focus. Universal Physics Transformers (UPT) Alkin et al. (2024) adopts encoder- decoder architecture to roll out physics evolution in the latent space, with an end-to-end training process. Transolver Wu al. (2024) proposed the usage of physics attention to efficiently convert point based input tokens into learnable slices, which are projected back to query coordinates at the output layer. Latent Neural Operator (LNO) Wang & Wang (2024) proposed the invertible

---

Published as a conference paper at ICLR 2026
wwssiic-cross-atteene to reduce the number of tokens in latent space. Overall, transformer- based neural operators have achieved state of the art in accuracy, offer greater input flexibility, and exhibit strong parallelization potential.
Operator Pretraining. Pretraining has been shown to be highly effective in natural language
processing tasks Devlin et al. (2019); Raffel et al. (2020), computer vision tasks He et al. (022);
Dosovitskiy al. (2020), and some scientific tasks (hou et al. (2023); Nguyen et al. (2023) Existing operator pretraining work largely follows two paradigms. The first, exemplified by Rahman et al.
(2024) and [Serrano al. (024), pretrain latent representations via physical field reconstruction to enhance training stability and generalization. The second, represented by McCabe et al. (2024); Hao al. (2024); Herde al. (2024). performs large-scale autoregressive pretraining on spatiotemporal fields across datasets that share similar underlying fluid dynamics. While effective, both paradigms are inherently physics- aware and require expensive PDE solution labels, which is precisely the dominant computational bottleneck in many industrial workflows. In contrast, geometry-only data (meshes or point clouds without solver outputs) are often plentiful but underexploited. However, physics-agnostic pretraining remains relatively underexplored, with few existing attempts largely adapt masked autoencoder (MAE) strategies designed for regular grids and pixel space (Chen et al.) 2024). This gap motivates the development of thysics-agnosti pretraining methods that can directly leverage geometry-onl datasets in unstructured settings.
3 METHODS
We propose to enhance the performance of neural operators via a physics-agnostic pretraining framework as seen in Fig. 2 The pretraining is performed on physics-agnostic (a, o) data pairs from D J D' to learn a better representation from imput point clouds a. A proxy task G : A + can be obtained from at negligible computational cost, where o is field corresponding to geometric
properties of point cloud a. The two datasets are therefore augmented to D {(ak, og, ) and D' {(aj oj) ) A neural operator is then trained on D with common supervised recipe to predict u from the pretrained latent representation rather than from a.
3.1 PHYSICS-AGNOSTTC OPERATOR PRETRAINING VIA OCCUPANCY PREDICTION
To obtain a meaningful representation from pretraining without PDE solution labels, a proper proxy task should possess the following three features: (1) Computation efficiency. It should be broadly applicable and computable directly from the mesh/point cloud input a at low computation cost. (2) Consistency with operator learning. It should stay under the umbrella of operator learning tasks to bridge naturally towards physical field prediction. (3) Sampling invariance. It should treat different point cloud realizations of the same underlying geometry equivalently discussed in Appendix B Ideally, the pretrained model would encode regions of high expected physical significance to better inform the neural operator learn PDE solutions. However, a fully general setting without explicit physics supervision, it is difficult to design an inductive bias that consistently improves performance across all problems. Taking all these factors into consideration, we pick occupancy field (Mescheder et al. 2019) E O as the proxy task for the main experiments, which is realized (similar to u)) by its
values on set of query points (= of) ) with of ox(2 E {0, 1}. where o(z) 1 indicates vice versa. Note that our framework can be easily extended to
that 2 inside an object and o(2)
other proxy tasks. In section 4.4 we also explore signed distance fields (SDF) and shortest vectors
(SV) Jessica et al. 2023) as alternative proxy tasks, while reserving occupancy fields for the main experiments.
To perform pretraining on general geometry data via the occupancy reconstruction task, we follow the practice in Zhang et al. (2023) train point cloud variational autoencoder (VAE) shown in Fig, 2 The encoder E adopts a perceiver architecture for aggregating the point cloud information into a fixed length vector, which is then projected into probabilistic latent space via MLP,, RCC RR Co and
4

---

Published as a conference paper at ICLR 2026
MLP, :C - :R::
mh Alln,, PosEmblax xx)))
(hw)): = MLPP, ( )
(2)
(ho) M.LPo(m) (me))e....))...
where X E IR-
is the d dimensional point cloud of N points, PosEmb: IR - RC is positional embedding neural network, L E RMC is a set of learnable tokens with fixed number M. The latent representation E R x is then sampled as:
h (hp) (ha)k-
where cm (0, 1). The decoder then predicts the occupancy value of a set of query points to reconstruct the geometry (occupancy field) on (zx) (zx), that the latent representation resides in function space (Appendix B). The training objective for the point cloud VAE takes the following form:
DUDD Σ
min 0.9
(Exz (CL,, + A KL(N ho) (1, 1)))
(4)
DUDI
where BCE(.;) stands for the binary strategy z ~ for occupancy field consists of two parts: uniform distribution over the computation domain (()), and perturbed point cloud z* x² + where & ~ NO, (7) with Ç being small positive scalar. Physics datasets are oftentimes sparse as they are computationally expensive to collect. However, the geometry dataset can be dense (D DD) as the computation cost for generating new geometries is negligible.
entropy loss, and À is the KL weight. The sampling
3.2 OPERATOR LEARNING
Instead directly taking a as inputs, neural operators J. take advantage of the geometry representation learned from physics-agnostic pretraining (Fig. 2 in the form of ( uk. where 0 is set of trainable parameters. Notice that the latent tokens produced by the pretrained encoder &g can be smoothly integrated transformer based neural operators discussed in section 2.2 The training goal to minimize the relative 1.2 error, where úy denotes the normalized ground truth physical field.
D
EN (F((E)(a))) 0Z)²
min
(5)
D
EM (z))²
When training neural operators on physics data D, we only update parameters while keeping the encoder & frozen. In this work, we consider two types of different query strategies for u that are widely used in various operator learning studies: directly querying the physical field the point cloud (mesh) } { J and querying the physical field according to a set of uniformly sampled random points {vv ((2)
4 EXPERIMENTS
We evaluate the proposed described in section 4.I and compare with direct supervised learning results. The quantitative evaluation of our method is presented in sections 4.3 a.d showing that training latent geometry representation from rich but physics-agnostic data can enhance neural operator's understanding of physical fields.
pretraining framework on four different datasets as
4.1 DATASETS
To examine our method, we modified the Elasticity eLi al. 2023a) and AirfRans (Bonnet et al. 2022) datasets and built two datasets to generate occupancy fields for physics-agnostic pretraining,
5

---

Published as a conference paper at ICLR 2026
and PDE solution fields for operator learning. The geometries in D' are sampled from the same (or similar) distribution as the geometries in D. We list the general statistics (Table I and the generation process of the datasets the following. More details on datasets can be found in Appendix _
Stress. We mimick the practice of the Elasticity dataset (Li et al. 2023a) to generate occupancy fields for physics-agnostic pretraining, and Von Mises stress fields for operator learning.
AirfRans (near). We directly adopt the pressure field data from AirfRans (Bonnet et al., 2022 and truncate the domain ((-0.6, 6, 1.6) x 00.5, 0.5)) to focus only on the near volume of the airfoils. We then follow the instruction (Bonnet et al. 2022) to generate new geometries via OpenFOAM and compute occupancy fields without running the CFD simulations.
Inductor (3D). We solve the Maxwell's equations using Comsol for parameterized 3D iron core subject to the current excitation in a copper coil. The norm of the magnetic flux density field is exported as the physics field to be predicted. Occupancy fields are also generated for the parameterized iron cores.
Electrostatics. We solve the electrostatics Poisson equation in domain that contains two materials with distinct permeability values. The two material domains are separated by a curved interface generated from third order spline interpolation of 10 random points. We then compute the occupancy fields for physics-agnostic pretraining, and electric potential fields for operator learning.
Table 1: Statistics of the four datasets for the main experiments. The second row shows the size of physics data labeled with PDE solutions u. the third row shows the total data size including physics and geometry-only data, while the last two rows show the computation time for solving PDEs and mere mesh generation.
Statistics
Stress 2D
AirfRans (near)
Inductor (3D)
Electrostatics
Dimension
2D
3D
2D
Size of D (train+test)
1900+200
800+200
900+300
1800+200
Size of D U D' (train+test)
7600+500
2400+600
3600+300
7600+400
Cost for D (CPU hour)
270
7680
1200
130
Cost for D' (CPU hour) <0.1
0.14
<0.1
<0.1
All four datasets follow the format of the sample data point visualized in Figure 3 Physics-agnostic pretraining is performed on a combination of the two occupancy field sampling strategies on D U D' then trained D with the physics field prediction accuracy on random uniform queries and mesh-based queries reported separately. We also concatenate the occupancy value of to the query coordinates y as inputs for training neural operators, so that the PDE field data take the form {(vv), ofJ u Note that for mesh-based queries where {v {at W occupancy value is always
as discussed in section 3.1 The neural operators
4.2 MODELS AND TRAINING.
We benchmark three transformer- based neural operators GNOT (Hao et al. 2023). Transolver (Wul ct al.] (2024), LNO (Wang& & Wang 2024) by training each under the standard operator learning setting with and without our proposed ayysics-agnsstic pretraining framework for performance comparison. The models are adjusted to have approximately the same number of learnable parameters. Notice that intentionally avoid exhaustive tuning of model training hyperparameters, is on evaluating the effectiveness of the physics-agnostic pretraining strategy. Therefore, the reported performance should not be interpreted as definitive comparison of model superiority.
focus
Point cloud VAE. We adopt the point cloud VAE architecture in Zhang et al. (023) for physics- agnostic pretraining to extract latent embedding from geometry-only data, trained with loss function in Eq 4 We set the embedding dimension C to 256, latent dimension Co to 32, learnable token number M 256, and depth of self-attention layers latent space 6.

---

Published as a conference paper at ICLR 2026
Normltzed sessure
Occuponcy
label
Mesh query physics field
Random query physics field
Random sampled occupancy field
Perturbed mesh occupancy field
Figure 3: Visualization of sample data point from the near volume AirfRans dataset. All of the datasets are preprocessed so that each data point consists of four types of information: physics field queried on the mesh point cloud, physics field queried on randomly sampled points, occupancy field queried on randomly sampled points, and occupancy field queried on the mesh point cloud perturbed by random displacements. The geometry data D' are labeled only with occupancy fields.
We adopt the GNOT model proposed in (Hao et al. 2023) as the first neural operator baseline, with hidden size of 128, head number of 4, and layer number of (each layer includes a linear self atteentennio and a linear cross- cooss-attention block). The gating strategy is excluded from the model to keep the comparison concise. The latent geometry representation be fed directly into the GNOT branch net without modification.
GNOT.
Transolver. We adopt the Transolver model proposed in Wu al. (0224) the second neural operator baseline, with a hidden size of 128, head number of 4, slice number of 32, and layers of physics-attention. The "get grid" layer excluded from the model keep the comparison concise. As the vanilla Transolver model assumes coupled query positions and point cloud/function observation positions, we replace the first thysics-attention layer of Transolver with a linear cross-attention layer from GNOT to allow for latent geometry representation inputs.
We adopt the LNO model proposed (Wang& & Wang 2024) as the third neural operator baseline, with hidden size of 128, head number of 4, and 8 attention layers. The vanilla LNO assumes shared parameters for encoding position information of the trunk input and branch input. Instead, use two distinct MLPs to allow for latent geometry representation inputs to the branch net.
LNO.
Training strategy. In each pretraining iteration VAE, 2048 points sampled from the mesh to explicitly represent the geometry. 1024 points are sampled from the random occupancy field, and another 1024 points sampled from the perturbed mesh occupancy field (illustrated in Figure 3 to estimate the geometry reconstruction eтror. We run 400 epochs using AdamW, with a half-cycle cosine scheduler of learning rate 1 x 10-3 and minimum learning rate x 10-5 All pretraining processes of VAEs are completely agnostic of physics.
For all datasets, the first 100 physics data are used to normalize the physics field. We then compute relative L2 error (Eq (5 on the normalized physics field for training loss and evaluation metric. Notice that the relative L2 error can appear much smaller on unnormalized datasets with large absolute values in the mean. We run 200 epochs of training for experiments neural operators, using the same optimizer and scheduler strategy as VAEs. In each operator training iteration, 2048 points are sampled from the mesh to explicitly represent the geometry, 4096 points are sampled from either the mesh query or random query physics field to estimate the operator prediction error. More details of GPU usage and compute resources included in Appendix E
4.3 MAIN RESULTS
We compare the performance of transformer- neural operators between taking point clouds as inputs and taking latent embeddings from thysics-annosti pretraining as inputs. Table 2ablessshows shows the operator prediction emror under mesh query and random query, with and without the pretrained
7

---

Published as a conference paper at ICLR 2026
encoder. The VAEs and neural operators in the main experiments trained with datasets listed in Table I with a fixed KL weight 0.001.
Table 2: Comparison of prediction errors on the four datasets. We report the mean error, with the standard deviation from three independent trials shown in parentheses. The physies-agnostic pretraining framework is tested on three transformer- based neural operators and compared with results from standard operator learning. We use bolded font highlight the results when pretraining
enhances the performance of operator learning.
Relative L2 on normalized data (x10 2)
Dataset
Query
GNOT
G+VAE
Trans
T+VAE
LNO
L+VAE
Number of parameters
1.7-1.8 M
1.7-1.8 M
1.8-1.9 M
Mesh
9.8(0.2)
9.0(0.1)
11.5(0.2)
11.2(0.2)
26.3(1.3)
13.6(0.6)
Stress
Random
10.3(0.2)
8.3(0.1)
11.5(0.6)
9.7(0.2)
20.0(0.2)
11.6(0.8)
Mesh
6.8(0.2)
5.6(0.1)
13.4(0.3)
12.7(0.3)
27.4(1.3)
27.1(0.2)
AirfR (near)
Random
7.8(0.2)
5.9(0.1)
15.0(0.2)
10.8(0.1)
25.3(1.8)
10.0(0.2)
Mesh
7.0(0.1)
7.1(0.1)
11.4(0.5)
8.4(0.2)
24.9(1.4)
9.2(0.6)
Inductor (3D)
Random
12.5(0.2)
11.8(0.2)
16.8(0.6)
13.2(0.2)
20.3(1.0)
13.0(0.2)
Mesh
4.2(0.1)
3.3(0.1)
5.0(0.1)
3.8(0.1)
13.5(0.4)
4.6(0.2)
Electrostatics
Random
4.6(0.2)
3.4(0.0)
5.6(0.1)
3.9(0.1)
13.5(0.2)
4.7(0.1)
Across most settings, we observe clear performance improvement on physical field prediction after introducing the physics-agnostic pretraining. The results are consistent across different datasets and neural operators, showing that the pretraining stage potentially constructs a stronger geometry representation than directly feeding raw point clouds. Note that point clouds are discretized ap- proximations of real geometries, while the latent representation (residing in function space since it reconstructs the occupancy field) offers more accurate approximation, especially after being enriched on diverse geometry data. We also observe larger improvements for randomly sampled query points than for mesh-based queries. One potential reason is that mesh-based sampling is denser than uniform sampling, particularly near geometry boundaries, so that baseline models readily capture critical geometric details, leaving less hcadroom for pretraining. This phenomenon is most pronounced on the 3D Inductor dataset, as uniform sampling in 3D space especially inefficient.
4.4 ABLATION STUDY
Number of geometry data. As physics-agnostic pretraining is decoupled from PDE solution labels, it is important study the impact of geometry data amount ID U D'I on operator performance. Experiments are conducted on the Stress dataset where we pretrain VAEs on geometries sampled from four types of different void priors shown in Table We then compare the performance of GNOT and Transolver using the latent space from each VAE as shown in Table VAE1 is only exposed to the physics data D, while VAE2 and VAE3 are trained on different settings of geometry data D'. It can be observed that the latent representation becomes stronger as more geometry data are added, even though some are sampled from different distributions. The improvement is less significant when the latent representation isn't 't augmented by extra geometry data (GNOT/Transolverr VAE1). These observations verify assumptions in Appendix B where the representation error can be reduced with increased number of unlabeled geometries.
Table 3: Metrics of pretrained encoders using different number of geometry data on the stress dataset.
Details of shape priors are discussed in Appendix
Stress data
VAEI
VAE2
VAE3
Ellipse a = 0.3, = 0.15
1900
3800
1900
Ellipse = 0.15, b = 0.3
0
0
1900
Rectangle a = 0.3, b 0.15
0
0
1900
Rectangle a 0.15. b = 0.3
0
0
1900
IOU(%)
99.6
99.8
99.8
KL loss
0.70
0.69
0.955
8

---

Published as a conference paper at ICLR 2026
Table 4: Performance of neural operators using encoders pretrained on different number of geometry- only data.
Dataset
Query
Relative L2 normalized data (x10- 21
GNOT
G+VAEI
G+VAE2
G+VAE3
Mesh
9.8
9.9
9.7
9.0
Stress
Random
10.3
9.1
8.6
8.3
Bllipse 0.3, b 0.15 1900
Trans
T+VAEI
T+VAE2
T+VAE3
Mesh
11.5
12.0
11.4
11.2
Random
11.5
10.4
9.8
9.7
Effect of KL weight. We investigate the effect of KL weight (A in 4) We train three VAEs and one deterministic autoencoder using geometries in the Stress and the AirfRans datasets (Table D.. We observe higher accuracy when relaxing the KI. weight, but all autoencoders achieve more than 99.5% IOU. The prediction accuracy GNOT and Transolver using the encoded latent space of the four autocncoders are listed in Table In most scenarios, the performance of neural operators are improved with the latent representation provided by the pretrained encoder. Although all four autoencoders achieve high reconstruction aceuracy, using probabilistic latent space in pretraining with a smaller regularization weight = 0.001 or À = 0.0001 tends to provide more robust performance. That said, the optimal choice can depend the specific neural operator architecture, as well as training hyperparameters, and the dataset being learned.
Table 5: Training metrics of point cloud autoencoder on the Stress and AiriRans (near) dataset at different KL. weights.
KL weight
0.01
0.001
0.0001
Deterministic
(OU(%)
99.5
99.7
99.8
99.8
Stress
KL loss
0.19
0.98
2.94
N/A
10U(%)
99.9
99.9
99.9
99.9
Airf (near)
KL loss
0.06
0.36
1.62
N/A
Table 6: Prediction error of neural operators using latent geometry representation from VAEs pretrained with different KL weights. Here, AE stands for deterministic autoencoder.
Dataset
Query
Relative L2 normalized data (x10
GNOT
+ +VAB(0.01)
+VAE(0.001)) +VAE(0.0001)
+AE
Mesh
9.8
9.9
9.0
9.3
10.3
Stress
Random
10.3
9.7
8.3
8.9
Mesh
6.8
6.6
5.6
5.3
5.9
Airf (near)
Random
7.8
6.2
5.9
5.2
5.9
Trans
+VA((0.01)
+VAE(0.000))
+VAE(0.0001)
+AE
Mesh
11.5
12.1
11.2
11.3
11.7
Stress
Random
11.5
11.4
9.7
9.4
10.4
Mesh
13.4
13.2
12.7
12.3
11.3
Airf (near)
Random
15.0
11.1
10.8
11.1
10.3
Alternative proxy tasks. While occupancy fields are selected as the default proxy task in the main experiments, our proposed framework is plug-and-play with alternative proxy task choices. discussed in section [ the proxy task can be chosen flexibly as long as it doesn't require PDE solving (computationally cheap), and can expressed as a field. For general adaptability, no prior know mowledge of the governing PDE should be assumed. Under these constraints, suitable candidates derived from material distribution fields or geometric representation fields. In CFD problems such AirfRans, SDF and SV are widely considered informative because they explicitly encode the relationship between arbitrary spatial locations and the nearest boundary, which is the dominant geometric factor in many CFD problems (Jessica et al. 2023). Therefore, adopt SDF SV the
typically

---

Published as a conference paper at ICLR 2026
proxy task in the physics-agnostie pretraining stage and evaluate the pressure prediction accuracy of GNOT on Airfaans as shown in Table When compared with direct supervised operator learning. proposed physics- agnostic pretraining approach remains effective in most scenarios, except for the SDF case on mesh-based physics field query, where the pretraining stage offers no measurable gain. Notably, using SV the proxy task yields the best physies field prediction accuracy. These results demonstrate both the effectiveness and breadth of our framework beyond occupancy fields. However, SDF SV are not necessarily as informative for PDE problems beyond CFD, especially when volumetric properties from the body mesh dominate over surface proximity. Therefore, we still believe that occupancy more general, intuitive, and agnossgi proxy choice.
Table GNOT performance on the AirfRans dataset: our physics-agnostic pretraining framework with occupancy. SDF, and SV proxy tasks vs. the standard operator learning results (also with SDF or SV included in inputs).
Relative L2 on normalized data (x10
Query
GNOT
G+SDF
G+SV
G+OCC.VAE
G+SDF_ VAE
G+SV_VAE
AirfRans (Mesh)
6.8
5.2
5.4
5.6
5.3
4.8
AirlRans (Random)
7.8
6.5
5.1
5,9
5.4
4.9
CONCLUSION AND LIMITATION
The paper explores the feasibility of leveraging the abundant geometry data resource to augment the performance of neural operators on scarce physics labels. Specifically, we introduce physics- agnostic pretraining framework that leverage geometry data via an occupancy field reconstruction proxy task. The pretraining stage can benefit from rich geometry data as it doesn't 't require PDE solu- tions which are the major computation bottleneck in operator learning tasks. The latent representation produced from the pretrained encoder is then integrated to transformer-based neural operators for supervised physical field prediction learning. Incorporation of selysics-agnosti pretraining yields consistent accuracy gains across different datasets and neural operator backbones. The framework is also adaptable to different proxy tasks and operator architectures. The results highlight the importance of properly selecting and preprocessing the representation of geometry inputs to neural operators.
The proposed physics-agnostic pretraining also possesses a few limitations. First, the effectiveness of occupancy reconstruction with multiple interacting geometries remains unexplored. Such settings may require replacing the BCE objective with multi-class cross entropy loss. Second, the default choice of occupancy intuitive. Although small ablation studies SDF and SV proxy tasks included, more systematic study is needed to help identify the most effective physics-agnostic proxy task based on the governing physics, input geometry format, and data accessibility, Lastly, integration between stages could be deepened beyond simply feeding pretrained latent embeddings. Potential strategies include joint or sequential finetuning with adjusted learning rates, hybrid losses that couple reconstruction and PDE supervision, or adapter-based conditioning.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
Detailed descriptions of the experimental setup, task definitions, and evaluation metrics are provided in section Appendix [C] The code is publicly available at: httts://ghhubb.com/////oo////bbs
Agnossii-Oppratoo-Peetriiiin
REFERENCES
Benedikt Alkin, Andreas Fürst, Simon Schmid, Lukas Gruber, Markus Holzleitner, and Johannes Brandstetter. Universal physies transformers: A framework for efficiently scaling neural operators. Advances in Neural Information Processing Systems, 37:25152-25194 2024.
Florent Bonnet, Jocelyn Mazari, Paola Cinnella, and Patrick Gallinari. Airfrans: High fidelity com-
putational fluid dynamics dataset for approximating reynolds-averaged Navier-Stoker solutions. Advances in Neural Information Processing Systems, 35:23463-23478, 2022.
Shuhao Cao. Choose a transformer: Fourier or galerkin. Advances in neural information processing systems, 0112492--24440 2021.
Wuyang Chen, Jialin Song, Pu Ren, Shashank Subramanian, Dmitriy Morozov, and Michael W Mahoney. Data-efficient operator learning via unsupervised pretraining and in-context learning. Advances in Neural Information Processing Systems, 37:6213-624, 2024.
Ze Cheng, Zhongkai Hao, Xiaoqiang Wang, Jianing Huang, Youjia Wu, Xudan Liu, Yiru Zhao, Songming Liu, and Hang Su. Reference neural operators: Learning the smooth dependence of solutions of pdes on geometric deformations. arXiv preprint arXiv:2405.. 10t.: 2024.
Jingyang Deng, Xingjian Li, Haoyi Xiong, Xiaoguang Hu, and Jinwen Ma. crommetry-guied conditional adaptation for surrogate models of large-scale 3d pdes on arbitrary geometries. In Proceedings of the Thirty-Thire International Joint Conference on Artificial Intelligence, pp- 5790-5799. 5798, 2024.
Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pp. 4171-4186, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 2020.
Donald Greenspan. Methods of matrix inversion. The American Mathematical Monthly, 62(5): 303-318, 1955.
Ronald B Guenther and John Lee. Partial differential equations of mathematical physics and integral equations. Courier Corporation, 1996.
John Guibas, Morteza Mardani, Zongyi Li, Andrew Tao, Anima Anandkumar, and Bryan Catan- zaro. Adaptive fourier neural operators: Efficient token mixers for transformers. arXiv preprint
arXiv:2111.13587, 2021.
Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. Gnot: A general neural operator transformer for operator learning. In International Conference Machine Learning, pp. 12556-12563 PMLR, 2023.
Zhongkai Hao, Chang Su. Songming Liu, Julius Berner, Chengyang Ying, Hang Su, Anima Anandku- mar, Jian Song, and Jun Zhu. Dpot: Aeto-segeessive denoising operator transformer for large-scale pde pre-training. arXiv preprint arXiv:2403.03542 2024.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000-16009, 2022.

---

Published as a conference paper at ICLR 2026
Maximilian Herde, Bogdan Raonic, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Emmanuel de Bézenac, and Siddhartha Mishra. Poseidon: Efficient foundation models for pdes. Advances in Neural Information Processing Systems, 37:72525-72622 2024.
Loh Sher En Jessica, Naheed Anjum Arafat, Wei Xian Lim, Wai Lee Chan, and Adams Wai Kin Kong. Finite volume features, global geometry representations, and residual training for deep learning-based CFD simulation. arXiv preprint arXiv:2311. 2023.
Zijie Li, Kazem Meidani, and Amir Barati Farimani. Transformer for partial differential equations' operator learning, arXiv preprint arXiv:2205. 13671, 2022.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.0885) 2020a.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arX1v:2003.03485, 2020b.
Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. Journal of Machine Learning Research, 24(388):1-22 2023a.
Zongyi Li, Nikola Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Otta, Mohammad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadenesheli, et al. Geometry-informed neural operator for large-scale 3d pdes. Advances in Neural Information Processing Systems, 36:
35836-35854, 2023b.
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via Deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218 229, 2021.
Michael McCabe, Bruno Régaldo-Saine Saint Blancard, Liam Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Siavash Golkar, Geraud Krawezik, Francois Lanusse, et al. Multiple physics pretraining for spatiotemporal surrogate models. Advances in Neural Information Processing Systems, 37:119301-119333 2024.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4460-447 2019.
Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K Gupta, and Aditya Grover. Climax: A foundation model for weather and climate. arXiv preprint arX1v:2301.10343, 2023.
Nicholas Perrone and Robert Kao. A general finite difference method for arbitrary meshes. Computers & Structures, 5(1):45-57, 1975.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1-67 2020.
Md Ashiqur Rahman, Robert Joseph George, Mogab Elleithy, Daniel Leibovici, Zongyi Li, Boris Bonev, Colin White, Julius Berner, Raymond A Yeh, Jean Kossaifi, et Pretraining codomain at- tention neural operators for solving multiphysics pdes. Advances in Neural Information Processing Systems, 37:104035-10404, 2024.
Junuthula Narasimha Reddy. An introduction to the finite element method. New York, 27(14), 1993.
Louis Serrano, Thomas X Wang, Etienne Le Naour, Jean-Noël Vittaut, and Patrick Gallinari. Aroma: Preserving spatial structure for latent PDE modeling with local neural fields. Advances in Neural Information Processing Systems, 37:13489-13521 2024.
Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. arXiv preprint 1xXiv:2111.13802. 2021.
12

---

Published as a conference paper at ICLR 2026
Tian Wang and Chuang Wang. Latent neural operator for solving forward and inverse pde problems. arXiv preprint arXiv:2406.03923, 2024.
Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long, Transolver: A fast transformer solver for pdes on general geometries. arXiv preprint wxXvv:2402.02366, 2024.
Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models. ACM Transactions On Graphics (TOG), 1264) 2023.
Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. In The eleventh international conference on learning representations, 2023.
13

---

Published as a conference paper at ICLR 2026
A LLM USAGE
LLM is only used on grammar polishing for this paper.
B PHYSICS-AGNOSTI PRETRAINING: ERROR DECOMPOSITION AND LATENT CHART VIEW OF SHAPE SPACE
Error decomposition. To help understand how physics-agnostic pretraining can improve down- stream operator learning, we analyze the prediction eπror from neural operators. Let Ngeo denote the number of total geometries (including both labeled and unlabeled) used in pretraining stage and php the number of physics-labeld predictor takes the form F6 &g, where & is the pretrained geometry encoder and Fg the neural operator trained physics data. Under standard regularity assumptions for representation learning and empirical risk minimization, the prediction error of the composed predictor can be decomposed into three major sources as the following:
samples used in supervised operator learning stage. The composed
((
Ce VN
Error Fo o Eg) S
+ RNNno (Fo) +
+
Capproz
VNpig
Approximation error
Representation error
Estimation error
Where the representation error term quantifies how well physics- aggnosicc pretraining produces geometry representation. This term can critical as point clouds are only digital approximations (discrete sampling) to the actual geometry. Ideally, the representation would be relevant to the downstream operator learning task, which difficult under the physics-agnostic setting where no prior knowledge is available. When the pretraining stage is well designed. the constant Cc, becomes smaller, indicating smoother or more structured input domain for the PDE operator. The estimation error term is the standard error from learning Fo with finite Nphy physics data. The third term Eapproz the inherent approximation limitation of the selected operator architecture.
Reduce representation error by physics-agnostic pretraining. In canonical settings, neural operators can only learn from data that are labeled with PDE solutions: N Nyeeo N Nphy- However, the effort of expanding Ngeo is significantly lower than obtaining Nphy industrial scenarios due to expensive simulations. Eqlo]points out potential performance improvement of operator learning given fixed number physics data: we can tighten the overall bound by reducing the representation error term via the following two routes: (i) designing good proxy task so that Cε, is small (latent features from strong geometry representation and align well with field prediction tasks). and (ii) leveraging abundant geometry-only data (N geo >> Nphy) to improve the bound rate O /I1/AA / geo). Empirically, these predictions match our observations: incorporating thysics-agnostie pretraining and scaling the amount of geometry data yields consistent improvements on downstream field prediction accuracy.
Design principles for physics-agnostic pretraining. As physics-agnostic data mostly consist of pure geometry information, we choose follow the central objective in shape analysis and geometry processing, which is to represent geometric objects while discarding nuisance variations such as orientation, sampling density, or point ordering. This naturally motivates organizing point clouds into equivalence classes: two point clouds are considered equivalent if they represent the same underlying shape up to transformations such as rigid motion, reparameterization, or sampling artifacts. The set of equivalence classes forms moduli space: space whose points correspond to distinct geometric structures modulo the chosen equivalence relation. this sense, learning a geometry representation corresponds to coordinatizing the moduli space of shapes.
An autoencoder provides a data-driven approach to this coordinatization where the encoder acts as a learned coordinate chart, mapping each point cloud to latent vector that represents its equivalence class, while the decoder ensures that this latent code retains enough information to reconstruct the original geometry. Thus, the latent space be interpreted the moduli space of point cloud geometries. This can be illustrated by the t-SNE plot in Figure 4 Point clouds corresponding to the same geometry form tight, well-separated clusters, indicating
learned, low-dimensional chart
14

---

Published as a conference paper at ICLR 2026
60
Geametry
40
Geometry
t-SNE Component
20
Geometry
Geometry
-20
-40
-60
-60
-40
-20
40
t-SNE Component
Figure Visualization of pretrained VAE latent space via t-EE. The plot shows latent representations of 4 different geometries; for each geometry, different point clouds are sampled and encoded. Point clouds from the same geometry stay closer in latent space, consistent with a learned chart on shape space.
that the encoder learns a meaningful representation in which distances in latent space correlate with differences between underlying geometries.
Formal description. Let C Nxd denote the space of point clouds with points in R Define an equivalence relation such that Pi ~ P3 if they represent the same underlying shape up to transformations from a group G (e.g., permutation, sampling/density variation, and optionally rigid motion via augmentation). The associated moduli space is the quotient space
M = X/
where each point [P] E M represents an equivalence class of point clouds corresponding to single geometric structure.
An autoencoder consists of an encoder & X' + I= and decoder D IR* + X' trained to minimize a reconstruction loss
C(P) = Dist( D(E((P)
where Dist is a distance metric invariant under the group G. Ideally, the encoder should be invariant under ~., i.e., P1 P2 = E(P)) = {(2),, implying that it factors through the projection X M. This yields the commutative diagram:
X
R
M
where & = م æ. Although æ is not explicitly computed in practice, this formalization clarifies the role of physics-agnostic pretraining: the encoder learns data-driven coordinate chart on the moduli space of shapes, producing a structured latent domain that supports a smaller representation constant Cs., for downstream operator learning.
C DATASETS
In this section, explain the details of the datasets including the governing equations, boundary conditions, material property, and geometry configurations. The original AirfRans dataset (Bonnet al.) 2022) includes detailed airfoil boundary shape information for computing occupancy fields and generating new geometries. However, we choose to reproduce the Elasticity dataset (Li et al.) 2023a) the detailed void geometry information is missing.
15

---

Published as a conference paper at ICLR 2026
Norma lited
Type void
Type 2 void
2
-2
The Mises stress field type void
Type 3 void
Type 4 void
Figure 5: Examples of physics data and geometry data from the Stress dataset. Neural Operators only learn from von Mises stress fields on type void, while observing the geometry data from geometries generated from types of different void priors.
Stress. We mimic the Elasticity dataset in (Li et al. fields Fenics. We find the distribution of Mises stress dem in domain with an irregular void inside with constant displacement vector at the top. We solve the following PDE:
2023a and regenerate the geometries and stress
V o (u)
where u(x) is the displacement vector field, c Tu is the strain tensor, and o Swe(e) is the stress de
Ar
tensor. To obtain the energy w(c) we need start with Cauchy Green stretch tensor C 2c and then get w(c)
Ci (tr(CC) - 3) + C2 (tr(C)² tr(C))). Two energy density parameters are: Ci = 1.863 x 10 and C2 9.79 x 10 One can then plug this result back into Equation to get an equation in terms of u(x). Note that although Equation 7 is linear in terms or, is highly non-linear in terms of actual underlying field variable u(x). After obtaining the displacement vector field, one can calculate stress tensor σ, and then get von Mises stress field dem from components of
T um o Oz vu + dyy + 30
(8)
For the solution domain, have square domain of [0. 1] [0, 1] with irregular hole at the center. This hole is described by an expression polar coordinates: r = r(0). In our setup, r(0) f(a, b, 0) + GRF(A, 1, f). The first f term represents the prior void shape. The second term represents a Gaussian random field with an amplitude of = 0.05 and correlation length of / 0.6. We generate geometries using 4 types of different void priors listed in Table 3 where a represents the horizontal axis/edge of ellipse/rectangle and b represents the vertical axis/edge of ellipse/rectangle. Example of the solution dem and geometries from different void priors are shown in Figure We generate 1900 training data and 100 testing data (occupancy fields) for each type of void for geometry representation learning. Neural operators are trained with 1900 data and tested with 200 data (von Mises stress fields) from type I void only.
AirfRans (near). We directly adopt the simulation data in AiriRans Bonnet et al. (2022) which solves the Beynolds-Averagee Navier-Stokes Equations for airfoils. Unlike the other three datasets, AirfRans (near) contains two global features: attack velocity and angle, which are directly concate- nated to all queries. The original dataset consists 800 training data and 200 testing data with both geometry and physics information. We train transformer-based neural operators to predict the pressure fields around the airfoils truncated domain [-0.6, 1.6) x [-0.5, 0.5] as shown in follow the same airfoil geometry parameterization
Figure To learn the geometry representation, (with a different random seed) in AiriRans and generate 2000 geometry only datasets (occupancy fields) with 1600 for training the VAE, and 400 for testing.
16

---

Published as a conference paper at ICLR 2026
||8|
1.2
0.8
0.6
0.4
0.2
Iron core and copper coil
Magnetic flux density norm field
Figure Example of a 3D inductor data. We generate the geometries (iron core in purple, coil in yellow) by parameterizing an EE type iron core, and calculate the corresponding magnetic flux density norm field (on the right).
Inductor 3D.
We calculate the magnetic flux density field B around a 3D inductor by solving the Maxwell's equations (Eq 9 in the frequency domain using Comsol.
V B 0
V H J jwCE
V x E -jwni
where H stands for magnetic field intensity, is the electric field, J is the current density field, is material permittivity (which is assumed to be 1 everywhere), and μ is material permeability. Figure an example of the 3D inductor, consisting of an EE type iron core whose mid pillar surrounded by a copper coil within a computation domain of [070, 70] x [-70, 70] [-70, 70] mmm (normalized for VAE and neural operator training). Magnetic insulation n x A is applied to the boundary of the computation domain, where stands for the unit normal and A is the magnetic vector potential satisfying B = V x A. The copper coil contains 500 turns with current density of 1e6 A A/m?.
The iron core follows nonlinear magnetizing curve as shown in Figure The neural operators are trained to predict the steady state magnitude the magnetic flux density field |B]| at high frequency of ~ 300 kHz. The nonlinear material property curve and high excitation frequency make this 3D inductor dataset extremely challenging. We generate 3600 training data and 300 testing data by randomly selecting the geometry parameters of the iron core. We compute occupancy fields for all the data to train the VAE, while only 900 the training data are simulated to train the neural operators listed in Table
2n
Electrostatics An electrostatics dataset is built using Fenics where we solve for the electric potential ((x: from the following equation:
- (((x))V((()) = p(x:)
(10)
where c(z) is the permittivity field, and p(x) the charge density field. Both these fields are scalar fields. Here we follow the Heaviside-Loremt units, permittivity is unitless. In dataset, the whole solution domain Ω is (0, 1) x [o, is divided into two parts separated by random interface. The random interface between these parts is determined with 10 random points. These 10 points
17

---

Published as a conference paper at ICLR 2026
0.0
10000
20000
30000
40000
50000
60000
H (A/m)
Figure B-H curve of the iron core material.
Normalized potential
1.0
0.5
0.0
Type interface
-0.5
-1.0
The electric poternial field type 1 interface
Type 2 interface
Figure 8: Examples of physics data and geometry data from the Electrostatics dataset. Neural Operators only learn from electric potential fields type interface, while observing geometry data generated from types of different material interface priors.
interpolated with third order spline interpolation to produce curve shown in the right part of Figure The locations of the 10 random points are generated by adding a horizontal perturbation 4(-0.2, 0.2) types of prior locations: 10 equally distanced points from (0.3, 1) (0.7, 0) and from (0.7, 1) to (0.3, 0). A charge density of 1 and permittivity of 15 is present in the blue part, while charge density of 0 and permittivity of is in the red part. The solution of this example case is shown in the left part of Figure We generate 3800 training data and 200 testing data (occupancy fields) for each type of interface for geometry representation learning. Neural operators are trained with 1800 data and tested with 200 data (potential fields) from type interface only.
D ADDITIONAL EXPERIMENTS
In this section, we present few additional ablation studies to examine the effectiveness of the proposed latent geometry representation under different data, training, model selection, and hyperpa- rameter settings.
Performance of UPT. also evaluate UPT on the Stress and Electrostatics dataset using our framework with the results shown in Table g We implement a simplified version using transformer for encoder, decoder, and propagator, while omitting the message passing layer and inverse encoding decoding training to avoid extensive tuning. The results indicate that framework can be smoothly adapted to UPT and improve its prediction accuracy on scarce physics labels. We notice that GNOT and Transolver (query input layer) show superior performance compared LNO and UPT (query
18

---

Published as a conference paper at ICLR 2026
at output layer). A likely reason is that UPT and LNO tailored for latent temporal rollouts, whereas all four benchmarks here are stationary physics. We also expect further improvement UPT with the complete architecture and training recipe.
Table 8: Prediction error UPT (simplified implementation) on the Stress and Electrostatics datasets. The physics-agnostic pretraining framework enhances the performance of UPT.
Relative L2 on normalized data (x 110")
Model
Stress, Mesh Stress, Random Elec, Mesh Elec, Random
UPT
26.3
20.8
11.6
12.0
VPT+VAE
12.8
10.6
4.9
4.8
Scarce physics dataset. To assess the effectiveness of physics- pretraining, we conduct experiments on the AirfRans dataset in data-scarce setting. In this setting, only 200 physies samples with PDE solutions u. are used for training neural operators. The VAE is pretrained on the complete geometry dataset with 2400 airfoil point clouds. The results Table [4 show that prediction accuracy decreases compared to full-data setting in the main experiments, as expected. Meanwhile, we observe clear performance gains after adopting the pretrained encoder in most cases (except for LNO under random query). This supports our intuition that effective long the physics dataset (with PDE solution) not dense. Under rich PDE solution dataset setting, the performance gain from geometry-only pretraining may become minor.
agnostic pretraining on proxy task will be
Table Comparison of prediction errors the AirfRans dataset with scarce setting. Encoders still pretrained on all 2400 geometry data, while only 200 sets of PDE solution labels are used to train neural operators.
Relative L2 on normalized data (x 2x
Dataset
GNOT
G+VAE
Transolver
T+VAE
LNO
L+VAE
AirfRans (Mesh)
15.8
13.9
22.2
21.3
61.0
52.9
AirfRans (Random)
12.8
10.2
32.9
21.0
52.1
52.8
Effect of physies data size. We investigate the effectiveness of the latent geometry representation under different sizes of physics datasets. We expand the physics training data to 3800 for the Stress dataset (oem field on type 1 void) and the Electrostatics dataset (6 field on type interface), while using the same VAE as in the main experiments (VAE3 in Table 3). Table T shosswws the performance of GNOT trained with different amount of physics data. For both datasets, we observe significant reduction in relative error for GNOT and GNOT+VAB when physics datasets are expanded. The performance improvement from replacing geometry representation consistent across different sizes of physics data. Interestingly, the VAE used for the Stress dataset only observes 1900 random geometries generated from type void prior, but still enhances the performance of GNOT when trained with 3800 physics data.
Table 10: Prediction error of GNOT (with and without different number of physics data. We use the same VAE as in the main experiments.
pretraining). trained with
Relative L2 on normalized data (x 10 3)
Dataset
Num of physies data
GNOT
GNOT+VAL
Query method
Mesh
Random
Mesh
Random
950
18.4
17.6
15.4
12.9
Stress
1900
9.8
10.3
9.0
8.3
3800
7.5
7.2
7.1
5.7
900
6.4
6.7
5.1
5.3
Electrostatics
1800
4.2
4.6
3.3
3.4
3800
3.1
3.3
2.3
2.3
19

---

Published as a conference paper at ICLR 2026
Masked Autoencoder. We evaluate masked autoencoder (MAE) alternative pretraining method. The experimental results on the electrostatics dataset are shown in Table II where MAE(x) stands for masked autoencoder with mask ratio of We notice that MAE pretraining also achieve slightly improved accuracy compared to direct supervised learning. However, the improvement is rather marginal compared our pretraining framework with occupancy proxy task. The physics- agnostic pretraining process via MAE attempts to predict the permittivity on some randomly masked points on the electrostatics dataset. This objective doesn't 't provide much information to the encoder, as there exists no prior constraints on the permittivity field a. unlike the solution field u. Also, MAE was designed for pixel space problems rather than point clouds. Therefore, we think MAE fits better for pixel space tasks or physics-aware pretraining such learning representations of for downstream autoregressive task (predict 1 from u.).
Table 11: Comparison of prediction errors on the electrostatics dataset among direct supervised training on GNOT, our proposed pretraining strategy, and masked autoencoder pretraining.
Relative L2 on normalized data (x10 2)
Dataset
GNOT
G+ VAE G+MAE(0.3) G+MAB(0.7)
Electrostatics (Mesh)
4.2
3.3
4.1
4.2
Electrostatics (Random)
4.6
3.4
4.3
4.3
Effect of batch size. For efficient training, a large batch size of 100 is employed in the main experiments (including error estimation) and ablation studies. Additionally, we assess the proposed framework's performance under conventional batch size of 32. Table 12 presents the results of neural operators trained with identical settings as the main experiments 4 except for the smaller batch size. reduction in relative error can be observed for all experiments when trained a smaller batch size. Meanwhile, the latent geometry representation improves the performance of neural operator in most experiments. However, its impact is minimal four of the comparisons: GNOT on Stress with mesh query, GNOT on Inductor (3D) with both queries, and Transolver Inductor (3D) with mesh query. We also observe performance drop when training Transolver on the Stress dataset. A potential explanation is that simply replacing Transolver's first layer with a linear cross- crossatteention layer may be insufficient to fully leverage the latent geometry representation.
Table 12: Comparison of neural operator performance with batch size of 32. All other model/training hyperparameters remain the same as in the main experiments. Experiments demonstrating significant performance enhancement are highlighted in bolded fonts.
Relative L2 normalized data (x10 2)
Dataset
Query
GNOT
G+VAE
Trans
T+ T+VAE
LNO
L+VAE
Number of parameters
1.7-1.8 M
1.7-1.8
M
1.8-1.9 M
Mesh
6.8
6.7
6.9
7.6
13.1
9.3
Stress
Random
6.1
5.5
6.2
6.7
14.4
6.5
Mesh
4.1
3.6
7.9
6.3
20.6
18.2
AirfRans
Random
5.6
3.6
8.9
6.4
16.9
5.8
Mesh
4.8
4.9
5.6
5.6
10.7
7.5
Inductor (3D)
Random
10.7
10.6
12.2
11.7
15.3
11.9
Mesh
3.0
2.1
3.2
1.8
7.8
2.8
Electrostatics
Random
3.3
2.3
3.8
1.8
6.9
2.8
Hidden dimension neural operators. Table 13 shows the results of neural operators with hidden dimension of 256 and head number of 8, which drastically increases the number of learnable parameters. Larger neural operators achieve lower relative error compared to the main experiments in T with the learned geometry representation still outperforming point clouds in most cases. Minimal impact from the latent geometry representation is observed in two comparisons: GNOT on Stress with mesh query, and GNOT Inductor (3D) with random query. Performance drop is observed in three cases: GNOT on Inductor (3D) with mesh query, Transolver on Stress with mesh query, and Transolver Inductor (3D) with mesh query. Notably, this is the only scenario where
20

---

Published as a conference paper at ICLR 2026
applying the latent geometry representation leads to performance degradation in GNOT. A potential explanation is that the iron core only takes a relatively small occupancy volume in 3D, making difficult for the VAE (trained with BCE reconstruction loss) to capture the geometry accurately. The performance drop in Transolver has been discussed in Effect of batch size.
Table 13: Comparison of neural operator performance with hidden dimension of 256. All other model/training hyperparameters remain the same in the main experiments. Experiments demon- strating significant performance enhancement are highlighted bolded fonts.
Relative L2 on normalized data (x10-3
Dataset
Query
GNOT
G+VAE
Trans
T+VAE
LNO
L+VAE
Number of parameters
6.9 M
6.9 M
7.6 M
Mesh
7.6
7.7
7.7
8.6
22.4
10.2
Stress
Random
7.3
6.3
8.6
7.5
18.0
8.7
Mesh
6.0
4.1
9.2
8.5
25.3
13.9
AirIRans
Random
6.1
4.2
12.0
8.1
19.4
9.4
Mesh
5.4
6.1
6.7
7.2
21.5
8.2
Inductor (3D)
Random
11.4
11.3
13.4
12.0
15.9
12.1
Mesh
3.4
2.5
4.3
2.2
10.2
3.1
Electrostatics
Random
3.8
2.6
4.6
2.2
9.8
3.1
Size of latent tokens. We ablate the VAE latent configuration in pretraining stage by comparing the performance gain on Stress dataset when attached to GNOT as shown in Table where N*M in column headers indicates N tokens of dimension M. Results show that 256 latent tokens with 32 dimensions give the best performance across both mesh-based and random query strategies. Increasing token number or dimension doesn't 't yield consistent gains. Instead, the latent size appears to strike a balance between expressiveness and regularization.
Table 14: Comparison of prediction accuracy of GNOT the Stress dataset using different sizes of pretrained latent. Columns denote the latent configuration (# token * latent dimension).
Relative L2 normalized data (x 10 10-2)
Dataset
No VAE
128*32
128*64
256*32
256*64
512*32
512*64
Stress, Mesh
9.8
10.0
9.6
9.0
9.5
9.0
9.2
Stress, Random
10.3
8.4
8.3
8.3
8.6
8.8
8.6
E COMPUTE RESOURCES
We list the number of parameters and memory cost for training the VAEs and transformer- based neural operators this section. Training time per epoch not included as experiments are carried out on different GPUs (A100 or V100). To conduct experiments efficiently, we use a large batch size of 100 for all experiments. However, we also conduct extra experiments to examine the effectiveness of the proposed framework under normal batch size of 32, discussed in Appendix D Other hyperparameters are listed in section
Table 15: Number of parameters and memory consumption for physics- pretraining and operator learning at hidden dimension of 128. The difference among different experiments is negligible.
Metrie
VAE Encoder
VAE Decoder
GNOT
Transolver
LNO
Parameters (M)
1.1
8.2
1.74
1.71
1.92
Memory (GB)
11.9
29.9
32.1
7.5
21

---

Published as a conference paper at ICLR 2026
F BROADER IMPACTS
This work introduces a physics-agnostic pretraining framework (generally adaptable to arbitrary mesh) enhance the performance of neural operators. We expect positive impact in research and industry fields where partial differential equations are solved to simulate the physical world. However, we don't 't envision any misusing of methods causing any negative social impact.
22


Reviews:
[{"review_id": "CdnoIQfsH3", "reviewer": "Reviewer_En7x", "summary": "This paper introduces a two-stage operator learning framework that first learns a latent geometric representation using an occupancy prediction model, followed by conditional inference for query locations through a transformer-based neural operator. The method is validated on four 2D and 3D examples by comparing relative L2 errors across three major operator learning models. The results consistently demonstrate performance improvements over the baseline models. The paper also discusses how increased dataset variability enhances performance and analyzes the effect of different KL coefficients.", "questions": "1. A clear computational cost study is needed to show a balance of the performance win and the speed reduction since the proposed approach introduces an additional stage compared to other models.\n2. How to address problems with sharp boundaries and more chaotic dynamics, such as turbulence, with the proposed approach?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The idea presented in this paper is clear and well-motivated. The use of cross-attention for learning latent geometric representations is interesting and may have broader applicability to PDE-based surrogate modeling approaches.\n2. Table 2 provides a comprehensive comparison between the proposed method and three other approaches, enhancing the credibility of the results.\n3. The ablation studies are clearly presented and address two important aspects: the effect of KL weights and the impact of geometric data variability.", "weaknesses": "1. The paper mainly evaluates model performance in terms of relative L2 error. However, introducing Stage 1 may introduce additional computational costs, which are not measured. Moreover, the advantages of transforming the original point cloud into a latent geometric representation are not fully explored. For instance, could this help reduce inference time for temporal propagation since the propagation occurs in latent space?\n2. Table 6 shows that an AE alone suffices for the AirfRANS dataset, making VAE unnecessary. A similar analysis on the other datasets would help clarify this observation.\n3. Finally, there is no comparison with other geometry-agnostic methods, such as “Li, Zongyi, et al. Geometry-informed neural operator for large-scale 3D PDEs. Advances in Neural Information Processing Systems 36 (2023): 35836–35854.” Without such comparisons, it is difficult to assess the effectiveness of the proposed method fully.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uKIP4xwc9g", "reviewer": "Reviewer_xrAn", "summary": "The authors of this work proposed a novel deep learning framework for the prediction of quantities of interest in computational mechanics such as stress, pressure, and velocity, where the solution of PDEs depend on the geometry of the computational domain. The tested their network by choosing dataset from fluid, solid, and electrical engineering and showed the performance by reporting relative errors in L2 norm. The study includes both 2D and 3D examples. At high level, the network has two stages, in the first stage, there is an encoder that only learns the geometric feature of computational domain from a set of point clouds. In stage 2, there is operator learning to learn PDE variables. Combing the features learned from stage 1 and stage 2, the network predicts variables in query coordinates.", "questions": "Have you ever tried PointNet, PointNet++, and other point-cloud based neural networks for solving these problems? Please see the details of my explanation in the Weaknesses section.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 1, "strengths": "The article is about an important subject in computational mechanics where we would like to accelerate geometric design, depending on various parameters, and we use deep learning frameworks for this goal. So, the topic is important. The authors investigated the performance using different datasets both from fluid and solid mechanics as well as electrical engineering (rather than only one dataset or one specific application).", "weaknesses": "As I mentioned in the Strengths section, the problem has been well defined. But I strongly believe that to overcome the challenge of this problem, there have been a class of point-cloud based neural networks (e.g., PointNet, PointNet++, etc.) that have been used for computational mechanics problems. PointNet has been introduced in 2017 and the usage of PointNet in CFD was introduced in 2021 for the first time and later on, there have been so many usage of PointNet in this area. Here is a list of some papers:\n\nPointNet and PointNet ++ in computer graphics:\n\nhttps://openaccess.thecvf.com/content_cvpr_2017/html/Qi_PointNet_Deep_Learning_CVPR_2017_paper.html\n\nhttps://proceedings.neurips.cc/paper/2017/hash/d8bf84be3800d12f74d8b05e9b89836f-Abstract.html\n\nPointNet for CFD:\n\nhttps://doi.org/10.1063/5.0033376\n\nPointNet for PINNs:\n\nhttps://doi.org/10.1016/j.jcp.2022.111510\n\nPointNet for Operator Learning:\n\nhttps://doi.org/10.1016/j.cma.2024.117130\n\nPointNet for KANs:\n\nhttps://doi.org/10.1016/j.cma.2025.117888\n\nNone of these papers have been cited by the authors. So, I imagine that they have been not aware of these methodology. Because these networks handle the same issue mentioned by the authors in more efficient ways. There is only one network that learns both geometric features and also PDE variables (e.g., velocity, pressure, temperature, etc.) simultaneously. The proposed method in this article, while works, but it is very complicated in comparison. It has two stages and requires pre and post processing as explained by the authors. In simple words, the authors tried to learn geometry and variables in two different networks, while both can be learned simultaneously in one network. I like the idea; however, we need to introduce something more efficient compared to what have been developed so far; otherwise, what is the motivation for using this new method.\n\nA few other comments:\n\nI think Figure 1 is redundant, Figure 2 can be also presented better, in the main body of the article, we could have more visual results and comparison between different methods. There are some bold words in abstract, I do not think it is good to make them bold. Introduction is relatively long for ICLR and the methodology has been explained very briefly. Moreover, in the results, again there are some details about the dataset which could be moved to the supplementary sections. Numbers in tables are written using a large, unusual font sizes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yDDHyBsya9", "reviewer": "Reviewer_hYfi", "summary": "The paper proposes a two-stage training framework for neural operator learning that leverages geometry only data (like cheap, unlabeled, physics-agnostic data) to improve performance on PDE solution predictions (via expensive, label-scarce physics data). This approach combines representation learning and operator learning, allowing physics-informed models to benefit from geometry-only datasets. This is typically ignored in standard PDE modeling.", "questions": "On page 1 you comment on fixed-grid structures. Don’t you mean fixed topology?\n\nPg2 input function space is assumed to contain only geometry info. How much of a limitation do you see this as, and how might this be extended?\n\nPg4 you define \\epsilon \\sim N(0,1). Then on pg 5 you have \\epsilon as the scale param in another N(0,\\epsilon I). The latter use implies \\epsilon > 0, at odds with the first usage definition.\n\nPg6 you exclude hyperparam tuning. I get your focus on the 2-stage training – but surely one of the selling points of the work is the superior performance, so it’s important to understand better the context of the reported results. Can you comment more?\n\nPg 7 – re. comment about unnorm data sets and MSE. Can’t you present NMSE, or R^2?\n\nReferences – you should protect capitals – like {N}avier-{S}tokes etc.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "* well-motivated framework bridging geometric pretraining and operator learning. PDE-based simulations (like CFD, elasticity, electromagnetics and so on) are computationally expensive, while 3D geometries are normally cheap to obtain. \n* The occupancy field proxy is neat. It’s simple, generalizable and physics-agnostic. It seems to effectively force the encoder to learn spatial and structural priors that are transferrable to PDE learning tasks.\n* The authors test some alternative proxy tasks (like signed distance fields, shortest vectors etc) and show comparable or improved results, especially for CFD datasets.\n * There is nice integration with Transformer-based Neural Operators. The latent embeddings are seamlessly fed into three state-of-the-art transformer-based neural operators. This makes for broad applicability and makes the contribution compatible with ongoing work in neural operator transformers.\n* Demonstrated generality across problems and architectures\n* Good experiments\n* The authors are honest about limitations", "weaknesses": "* Lack of joint optimization between pretraining and PDE stages.\n* Limited exploration of theoretical connections between learned latent and PDE operator manifolds.\n* Real-world benchmarks (e.g., turbulence, multiphysics coupling and stuff like that) could further validate scalability. The latter is important yet not discussed enough.\n* The authors themselves allude to some limitations / weaknesses. Complex geometries with multiple interacting components may require multi-class or more expressive proxies. The lack of joint fine-tuning may limit performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vJpkdtLO6o", "reviewer": "Reviewer_pbnB", "summary": "Pre-training of neural operators using geometry information.", "questions": "- Training time ratio for 1st and 2nd stages?\n- LNO is the most recent, but performed the worst. Could you comment on that?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Well written. The author motivated the geometry-based pre-training strategy well.\n- Considered various benchmark problems.\n- Considered various recently proposed backbone architectures.\n- Error bars.", "weaknesses": "- line 53: This method is also limited to varying geometry problems. To say something about applicability to general PDE problems, it is necessary to show effectiveness on a non-varying geometry problem.\n- Relative $L^2$ errors are too high—for engineering applications, two to three digits of accuracy is usually recommended. However, more than 10% of error makes it difficult to see whether the improvement comes from the proposed method or optimization error.\n- The paper tackled data scarcity. However, the reader may not agree that more than 800 training data points are scarce. Especially, the AirfRans dataset provides data data-scarce scenario. Reporting the result in such a scenario may boost the credibility of this work.\n- Asserting the encoder as a learned chart for the moduli space is good, but there was no quantitative validation of it. Reporting the distance between two latent codes from two point clouds representing the same geometry would improve the claim.\n- Perhaps, further truncation of the AirfRans domain was due to failure in the original domain. (In fact, the original domain of the AirfRans dataset had been truncated already.)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)