HyPINO: Multi-Physics Neural Operators via HyperPINNs and the Method of Manufactured Solutions
Authors: Rafael Bischof
Michal Piovarci
Michael Anton Kraus
Siddhartha Mishra
Bernd Bickel
Venue: NeurIPS 2025 spotlight ()
Keywords: neural operators
physics-informed neural networks
partial differential equations
hypernetworks
Primary area: machine_learning_for_sciences

Abstract: We present HyPINO, a multi-physics neural operator designed for zero-shot generalization across a broad class of PDEs without requiring task-specific fine-tuning. Our approach combines a Swin Transformer-based hypernetwork with mixed supervision: (i) labeled data from analytical solutions generated via the Method of Manufactured Solutions (MMS), and (ii) unlabeled samples optimized using physics-informed objectives. The model maps PDE parameterizations to target Physics-Informed Neural Networks (PINNs) and can handle linear elliptic, hyperbolic, and parabolic equations in two dimensions with varying source terms, geometries, and mixed Dirichlet/Neumann boundary conditions, including interior boundaries. HyPINO achieves strong zero-shot accuracy on seven benchmark problems from PINN literature, outperforming U-Nets, Poseidon, and Physics-Informed Neural Operators (PINO). Further, we introduce an iterative refinement procedure that treats the residual of the generated PINN as "delta PDE" and performs another forward pass to generate a corrective PINN. Summing their contributions and repeating this process forms an ensemble whose combined solution progressively reduces the error on six benchmarks and achieves a >100× lower $L_2$ loss in the best case, while retaining forward-only inference. Additionally, we evaluate the fine-tuning behavior of PINNs initialized by HyPINO and show that they converge faster and to lower final error than both randomly initialized and Reptile-meta-learned PINNs on five benchmarks, performing on par on the remaining two. Our results highlight the potential of this scalable approach as a foundation for extending neural operators toward solving increasingly complex, nonlinear, and high-dimensional PDE problems. The code and model weights are publicly available at https://github.com/rbischof/hypino.

HyPINO: Multi-Physicss Neural Operators via HyperPINNs and the Method of Manufactured Solutions
Rafael B
Michal Piovarči
Michael A. Kraus²
Siddhartha Mishra
Bernd Bicke!
1Computational Design Lab, ETH Zurich, Switzerland
2 Institute of Structural Mechanics and Design, TU Darmstadt, Germany Seminar for Applied Mathematics, ETH Zurich, Switzerland *Correspondence to vabischof@ettz.ch
Abstract
We present HyPINO, a multi polti--pyyscs neural operator designed for zero-shot gener- alization across broad class of PDEs without requiring task-specific fine-tuning. Our approach combines a Swin Transformer-based hypernetwork with mixed su- pervision: (i) labeled data from analytical solutions generated via the Method Manufactured Solutions (MMS), and (ii) unlabeled samples optimized using ahysics-informed objectives. The model maps PDE parameterizations to target Physics-Informed Neural Networks (PINNs) and can handle linear elliptic, hy- perbolic, and parabolic equations in two dimensions with varying source terms, geometries, and mixed Dirichlet/Neuman boundary conditions, including interior boundaries. HyPINO achieves strong zero-shot accuracy on seven benchmark prob- lems from PINN literature, outperforming U-Nets, Poseidon, and Physice-Informed Neural Operators (PINO). Further, we introduce an iterative refinement procedure that treats the residual of the generated PINN as "delta PDE" and performs an- other forward pass to generate a corrective PINN. Summing their contributions and repeating this process forms ensemble whose combined solution progres- sively reduces the error on six benchmarks and achieves a >100x lower Le loss evalu- in the best case, while retaining forward-only inference. Additionally, ate the fine-tuning behavior of PINNs initialized by HyPINO and show that they converge faster and to lower final error than both randomly initialized and Reptile- meta-learned PINNs on five benchmarks, performing on par on the remaining two. Our results highlight the potential of this scalable approach a foundation for extending neural operators toward solving increasingly complex, nonlinear, and ligh-diinnsional PDE problems. The code and model weights publicly available at hattps://ee thub. com/ saascchof/hppin
Informed
Introduction
Neural operators have emerged a promising paradigm for solving partial differential equations (PDEs). Their ability to generalize across families of PDEs, fast inference, and full differentiability make them appealing for a wide range scientific computing tasks. In the longer term, such methods may serve as building blocks for general-purpose foundational, and multi- simulators, sometimes referred to as "world-model predictors"
53
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

However, existing neural operators are typically sample inefficient [19]. As a result, most prior work focuses on narrowly defined problem families [3T] Variations are limited to singular aspects such as
specific PDE parameters (e.g., diffusion coefficients) 66) boundary conditions [10] or domain shapes
[55]. The support for simultaneous variations of PDE operators remains limited to subdomains, such
as parametrized convection-di diffusion-reaction PDFs [52]
One way address the data requirement is by incorporating physics-informed losses. Such losses can provide self-supervision without requiring labeled simulation data 40 47 While promising, existing methods often suffer from spectral bias 48 and mode collapse [11. Moreover, purely physics-bal based training is unstable in practice [2]. Therefore, even with physics- losses, obtaining a large, labeled dataset that spans a diverse range of PDEs remains significant bottleneck.
To overcome these challenges, we propose HyPINO, a hybrid framework that combines physics- informed learning with synthetic supervised data. Our approach leverages Swin Transformer- based hypernetwork [8] 13 30 to map PDE specifications to the parameters of a target physics-infoma neural network (PINN). HyPINO enables zero-shot generalization without task-specific fine- tne-ttning. A key contribution of our work is a scalable synthetic data pipeline that generates two complementary types of training data: (i) Supervised samples generated via the Method of Manufactured Solutions (MMS) 38 by selecting target solutions and deriving the corresponding PDEs analytically. These provide direct supervision with known reference solutions. (ii) Physics-only samples constructed by randomly sampling PDE operators, source terms, and boundary initial conditions. These are trained using physics-informed losses without requiring ground-truu solutions. This hybrid training strategy allows us to cover a broad spectrum of two-dimensional linear elliptic, hyperbolic, and parabolic PDEs with mixed Dirichlet and Neumann boundary conditions on complex domain geometries, spanning wide range of phenomena in the natural and engineering sciences, including heat diffusion, wave propagation, acoustic scattering, and membrane deformation.
In addition, we introduce an iterative refinement procedure that builds an ensemble of corrective PINNs. At each iteration, the model evaluates the residual error and generates "delta" PINN to improve the solution. This ensemble refinement provides a lightweight alternative to traditional fine-tuning, requiring only inference passes rather than full backward passes.
We evaluate HyPINO on seven diverse PDE benchmarks, demonstrating improved zero-shot gener- alization compared to baselines such as U-Nets [1). Poseidon [19], and PINO (9). We also find that PINNs initialized with our method fine-tune more efficiently than those starting from random or meta-learned initializations, achieving faster convergence and lower final errors.
In summary, our main contributions are (i) a hybrid physics-informed and supervised learning frame- work for multi-physics PDE solving, (ii) scalable data generation pipeline combining random physics sampling with MMS-based supervised examples, (iii) an ensemble- based refinement mecha- nism that improves prediction quality without expensive retraining, and (iv) empirical results showing strong zero-shot and fine-tuning performance across multiple PDE benchmarks compared to SOTA.
We believe these contributions offer a practical step toward more general-purpon data-efficient neural operators for multi-physics problems and world simulator foundation models.
2
Related Work
Neural operators aim to approximate solution operators that map PDE specifications to continuous solution fields, enabling fast, mesh-free inference and generalization to unseen problem instances [15]
27
28
29
3T 34]. Recent work scales these ideas toward foundation models that ingest large corpora
of simulated data or equation specifications and promise broad tross-task transfer [17 35 (2]
Despite rapid progress, most operators still target narrow PDE families (e.g. fixed equations with varying coefficients) and depend on expensive high-fidelit solvers for supervision 52 Embedding the governing equations the loss function alleviates the need for labeled data and improves physical fidelity [40 477. While the original formulation was introduced for stand-alone Physics-Informed Neural Networks (PINNs), the same residual losses have recently been integrated into operator architectures, yielding Physics-Informed Neural Operators (PINO) that train from unlabeled residual samples 12 29). These approaches still require careful weighting of supervision terms and often struggle with stability and spectral bias for complex PDEs.

---

Biscoding
ENtlastion
BtPPNO
Deate Mid
Trakinng
E1
Unagemeed
Zero-Shat Shot Preductons
(9 Downdrea Adaptation
Interence
FIM
Liniinn
Figure 1: Overview of the HyPINO pipeline. (a) Training data includes supervised samples from MMS and unsupervised thysies-informed samples without ground truth. (b) PDEs are encoded as multi-channel and vector-based inputs and processed by HyPINO to produce task-specific PINN weights. (c) The predicted PINN maps spatial coordinates to the solution field. (d) Training combines physics-informed residual losses well as supervised losses for MMS data. (e) At inference,
HyPINO enables zero-shot prediction for unseen PDEs. (f) Downstream adaptation includes iterative refinement using residual corrections or optional, task-specific fine-tuning.
Hypernetworks generate the parameters of target network conditioned on an auxiliary input 13 In
the PDE context, HyperPINNs predict PINN weights for varying coefficients ( 24], and subsequent works extend this idea to boundary conditions, domain changes, and low-rank weight modulation [5]
10
Yet, existing models rarely support concurrent variation of multiple operators, geometries, and boundary types without task-specific fine-tuning.
36
The Method Manufactured Solutions (MMS) provides analytic ground-truth pairs by choosing a target field and deriving the corresponding source term and boundary data (38]. MMS has long served for numerical-solver verification and was recently adopted for PINN evaluation 23 and operator training [18. However, prior studies focus on single equations (e.g. Poisson); leveraging MMS for multi multi-physicss operator pre-training remains largely unexplored.
Our work situates itself at the intersection of these lines: we couple a Swin Transformer hypernetwork with mixed MMS and physics-informed supervision to produce a single model that generalizes zero- shot across diverse linear, elliptic, hyperbolic, and parabolic PDEs with mixed boundary conditions and diverse 2D geometries.
Methodology
We consider family of second-order linear PDEs defined over a bounded domain Q C R² with boundary 0 M&p L a22,, where Mdp and 00N denote the Dirichlet and Neumann boundaries. respectively. The goal is to find function u Ω + R" satisfying
Ou
[[u](x) f(x) in Ω u(x) g(x) 0n
On (x) h(() MnN,
where L is a linear differential operator involving derivatives up to second order, f: / Ω - R is known source term, and g, h are prescribed boundary functions. Our objective learn the solution operator that maps the tuple (C, f.g. h) the solution u.
3

---

3.1 PDE Parameterization
To support a wide range of linear PDEs while maintaining compatibility with modern machine learning models, we adopt parameterization that is flexible, user-friendly, and efficiently processed by State-of-the-ars architectures. The function f discretized on a uniform grid over Q, resulting in 2D array F representing its values at grid points. The boundary conditions are parametrized by creating two 2D grids per boundary type (0pp. 0(2x): (i) binary mask M indicating the presence of the boundary at each grid point, where we assign value of to the four grid cells closest to each boundary point and zero elsewhere; and (ii) value grid V storing the corresponding boundary values (g for Dirichlet or h for Neumann conditions) at those marked cells, with zeros elsewhere. Figure iillustrates a sampled PDE instance with its full parameterization. Finally, following [21], we parameterize L as Clui (x) e C2llz +yy Cg + yg where (c1, C2. C3 Ca. Cs) E R
encodes the operator coefficients.
3.2 Neural Operator Architecture
We base our model on HyperPINN [8]. a ahypernetwork-baaee neural operator that maps parametrized PDE instance the weights f of PINN up specialized that instance. Formally, the hypernetwork realizes mapping
(c, F, Mg Ms Vg V.) 0" such that ug R u,
(2)
where e denotes the vector of PDE coefficients, F the discretized source function, Mg and Ms the Dirichlet and Neumann boundary condition location grids, V. and Vn, the Dirichlet and Neumann boundary condition value grids, and u the reference solution.
The vector of operator coefficients с E R is embedded into fixed-length representation &c E Ré using a Fourier feature mapping 44 which was shown to prevent spectral bias and mode collapse, in particular in physics-informed settings [6). The grid-based inputs F, .M. Mr. Vg and V,, are concatenated and processed via series of K Swin Transformer blocks (SW.). K
ji= [30]_ After each block, we introduce a FiLM layer [39], which modulates the Swin block's output conditioned on aç:
2+11 = FiLM, (SW.,zi,, zc),
2g E Hx
11. we retain all intermediate latent represen- tations } to keep information at various semantic levels. To enable information aggregation, we flatten the spatial dimensions H; and W; and use Multi-Head Attention Pooling [25] [54, where set of trainable query vectors {2: K 1: Qa E T'xx; is defined. T corresponds to the number of
Inspired by Swin Transformer U-Net architectures
K
weight and bias tensors in the target PINN. The queries @ are then used in a multi-head attention module together with 2; reshaped into kw; E IR. XW,
P MultiHead.AAtenniion.
ku;), Ps RTC,,
The outputs {p:) are concatenated along the channel dimension to produce unified latent matrix p = [Pr |l P2 ... Pk] E TEX" C:), containing an entry of aggregated information for each weight matrix and bias vector in the target PINN. Finally, dedicated MLPs project each entry into the required dimensionality for the corresponding weight matrix or bias vector.
We define the architecture of the target PINN as MLP with Fourier feature mapping [44), which, when concatenated to the input (I, results in a dimensionality of 2N + 2, and multiplicative skip connections [45]. Fourier encodings provide spectral expressivity for modeling high- fighfrrequency components (46), while the skip connections enhance gradient propagation and, in the context of hypernetworks, have the additional benefit of enabling dynamic depth modulation based PDE complexity by allowing the hypernet to mask some layers.
For each PDE instance, the hypernetwork therefore generates the following parameter set 0*:
(Wo U, V, by, b,), {W, TT Wour, bout
(5)
where d denotes the width of the latent layers. The parameter dimensions are as follows: Wo, U. V. E Rdx and bn, bu, b, Rd; for i 1 1...., T 2, W; E dxd and b, E IRd and finally, Wont& RRidd and bont E R. Note that we use the tanh activation function due to its bounded output space, which provides stability to the hypernet's training.
4

---

(a) 80p
(b) g(z, v)
(c) 0an
(d) h(
(e) f(, v)
(l) u v)
Figure 2: Sample generated via MMS with sampled operator Clu] 00.31ug
0. 0.15uy and
sampled boundaries 0?: (a) Dirichlet boundary, (b) Dirichlet condition, (c) Neumann boundary, (d) Neumann condition, (e) source term, and (f) analytical solution.
3.3
Data Sampling
We create a synthetic dataset of PDEs by randomly drawing the differential operator L, domain S2 boundary data, source term f. and, when available, reference solution The full dataset is mix of two classes, supervised and unsupervised samples. For supervised samples, a manufactured analytical solution u chosen first. We then set f = L[u] and derive g(x) u(x) and or h(x) - om (x) by evaluating uxx) and its normal derivative on On. In addition to the physics-informe
loss, samples of this class provide the analytical solution u(x) as well as derivatives that can used for additional supervised losses during training. For unsupervised samples, the reference solution is unknown. We sample f and boundary conditions subject to constraints designed to maximize diversity and the probability of well-posedness. Training relies solely the physics-ifformed loss, as reference solutions are unavailable.
Differential operators L are formed by sampling ~ Uniform( 3.)) terms from B {u, Up. My: Mrzz Uyy? without replacement. Each selected term T, is assigned coefficient G Uniform([ 21, 2]), and the operator is defined as C[u] = -i=1 e,Ti[u].
To generate supervised samples, we use MMS, established approach for validating PDE solvers. We first construct an analytical solution u Ω - on a domain Ω C R22 by applying n ~ Uniform {6, ..... 10}) iterative updates starting from u(x, y) < 0. Each update adds term of the form dd>b(aaz + +c)+ +e, where vi E {I, sin, cos, tanh, (1+e 10-t--)-1 (1++1,1112 ++=+=)))}+ and coefficients a, E {0, Uniform 10, 10))), and c, d, Uniform (1- 2T)). Terms are incorporated using one of three rules chosen uniformly at random: additive, multiplicative, compositional.
Source term generation depends the availability of analytical solution. For supervised samples, we compute f(x = Clu) (x) via symbolic differentiation. For unsupervised samples, where is unknown, set (x) = (0, 10a), i.e., a spatially constant random source drawn from zero-mean Gaussian.
Domains Ω 1] generated via randomized Constructive Solid Geometry (CSG) [32 The outer boundary Mouuer is defined as the unit square and may represent either purely spatial or spatio-temporal domains, with y -1 marking the initial time in time-dependent PDEs. Inner boundaries 0mmmmr, are formed by subtracting randomly sampled geometric primitives (e.g., disks, polygons, rectangles) from the outer region. These boundaries enclose regions where the source term if remains active, and their role (e.g., obstacle vs. inclusion) is encoded implicitly through the boundary type.
Each inner boundary 2imner, is randomly assigned Dirichlet, Neumann, or both: w(x) -x)x) or 0u/ōn = hx(x). For supervised samples, where u(x) x) is known, we set g and h = 0u/On. For unsupervised samples, boundary values are sampled to promote compatibility with the operator L[u]. If appears standalone term, set 0 0Ω to avoid trivial or inconsistent configurations (e.g., u = f with nonzero f). If first-order terms (e.g., Ug (ny) appear alone, constant Dirichlet values used. In other cases, linear profiles are allowed, offering mild spatial variability without conflicting with the constant source term of unsupervised samples.
Despite efforts to ensure well poseeness,,,,, some unsupervised samples may still be ill- ill-posed due to incompatible boundary and source term configurations. Nonetheless, they are essential for exposing the model to realistic complexities such as interior boundaries, inclusions, and discontinuities. These are common features practical PDEs but are difficult to introduce through supervised data generated via MMS.
5

---

3.4
Objective Function
For each PDE instance (L, f. g. h) on f [-1, with Dirichlet (8hp) and Neumann (09N)
boundaries, HyPINO 4 (L, f. h) + 0* produces weights #* for a target PINN ug- Ω - R.
JR
Σ
(((ug - f(x))
(6)
xEi
is the residual loss and p(-) the Huber function [2) The Dirichlet and Neumann losses are computed similarly:
TD =
Σ
p (ug- (x)o-(x)-g(00 g(x)), JN
1
Σ
p(Vuo- (x) n(x) h(x)).
|adpl
J0ΩN
xEOΩD
XEOΩ
(7)
For PDEs with known analytical solutions u, we add a second-order Sobolev loss that penalizes emrors in function values, gradients, and second derivatives:
(k) p (V-ug- (x) - V=
Js
(8)
XEI K=
The total loss is weighted sum of the active terms:
J = AnJn + ADJD + AN + Js.
(9)
where JR is always included, TD and IN are applied when collocation points fall on 82p or agN and Ts active only when the ground-truth solution u is known.
3.5 Residual-Driven Iterative Refinement
Using a hypernetwork to generate a single PINN fixed architecture may seem restrictive, particularly in multi-physics settings where different PDEs may demand different levels of representational complexity. However, hypernetworks offer a natural mechanism for generating ensembles of PINNs inference time, which have proven effective in reducing prediction error [ 22 42). Beyond naively producing multiple independent samples, our framework for linear PDEs supports an ensemble construction through iterative refinement procedure, similar in spirit multi-stage neural networks that progressively reduce residual error 49
Given PDE instance (L,fgg,)) h). the hypernetwork generates an initial solution ( (L.fff.).h)
We compute residuals ,00),, .rD
(0) and r with respect to the PDE and boundary conditions, treat the residuals as "delta PDE" and feed them back into the hypemetwork obtain corrective PINN:
(1 = U U (0) (0) (0)
(10)
The updated solution is (1) (0)) (1). We repeat this process for 0,..., T....,T-1 T
(++1) (1) + 5m (L+1) with (1+1) = u ULLL, (0 "0
(11)
After iterations, the final prediction (T) (0) ET= Su( We refer to this model as HyPINO', where defines the number of refinement rounds.
During iterative refinement, only the small PINNs are diffecrentiaeed to compute residuals, whereas the hypernetwork remains inference mode. We use uniform weights for each fr((), though adaptive weighting (e.g., scaled residual updates) remains promising direction for future work.

---

Experiments
4.1 Training
HyPINO generates weights for target PINN with three hidden layers and 32 hidden units per layer: The full model has 77M trainable parameters. We train the hypemetwork for 30,000 batches using the Adam W optimizer with a cosine learning rate schedule from 10-4 to 10-6 and batch size of 128. Training was conducted on NVIDIA RTX 4090 GPUs for all experiments.
Training is divided into two phases. In the first 10,000 batches, all samples are supervised with known analytical solutions, using loss weights: AR 0.01, \ 1, X 0.1, A² 0.01, AD = 10, and AN = In the remaining 20,000 batches, each batch consists of 50% supervised and
50% unsupervised samples. Loss weights are updated to: AR 0.1, Ag 1, Ag 1, Ag) 0.1, AD 10, and A
4.2 Baseline Models
We compare HyPINO against three baselines, each trained for 30,000 batches with batch size 128 and an initial learning rate of 10-+: (i) U-Net [4]. which shares HyPINO's encoder but replaces the hypernetwork decoder with a convolutional decoder that directly outputs 224 x 224 solution grid. trained solely on supervised data and has 62M trainable parameters. (ii) Poseidon [19). a pretrained neural operator with 158M parameters. We use the Poseidon-B checkpoint and adapt it by changing the embedding and lead-ting time-conditioned layer normalization layers' dimensionality to match the size of parameterization. Similarly to the U-Net, Poseidon is fine-tuned only supervised data. (iii) PINO 29], a Fourier neural operator (28) with 33M parameters. We adapt to accept 5-channel grid inputs and condition the PDE operator using FiLM layers. It is trained using the same hybrid supervision and curriculum as HyPINO, including physics- informed losses.
4.3 Evaluation
We evaluate HyPINO and baseline models seven standard PDE benchmarks from the PINN
literature: (i) - ID heat equation [32), (ii) HZ 2D Helmholtz equation (2). (iii) HZ-G Helmholtz
on an irregular geometry [16]. (iv) PS-C Poisson with four circular interior boundaries [16], (v)
PS-L Poisson on an L-shaped domain 32 (vi) PS-G Poisson with a Gaussian vorticity field [19],
and (vii) WV ID wave equation [66. The exact problem statements and visualizations of the
corresponding parameterizations are provided Appendix B
Table 1: Model performance across seven PDE benchmarks. Each cell shows mean squared error (MSE) symmetric mean absolute percentage error (SMAPE) B3 Lower is better.
HT
HZ
HZ-G
PS-C
PS-L
PS-G
WV
U-Net
3.
68
6.9c-2/ 68
2.7c-2/33
3.9c3//
112
9.2c-1
159
3.7c-1 144
Poseidon
7.1c-2/47
3
1.3c-1/65
5.3e-2/93
3.5e-3/ 111
7.2c-1/ 155
8.7c-1 138
PINO
1.4e-2/38
2.0e-2/51
6.1e-2/60
1.7e-1/65
3.3e-3/51
3.1e-1/70
3.0e-1
149
3
1.3-22/7
7.2e-3/49
4.6e-2/6
64
2.8e-2/63
4.6e-3/62
2.3e-2/43
3.1e-1 (le-1/127
1
3.9c-2/78
5.1e-3/39
1.4c-1 75
1.1c-2/48
1.0c-3
1.8e-2 /38
8.5c-1 139
HyPINO
2.3c-2/42
5.7e-3 /36
/6
1.3e-1 / 64
5.6e-2 86
1.7e-4 /39
1.8c-1 /61
2.9c-1 150
HyPINO*
11
2.7e-3/31
1.6e-2/38 38
3.4e-3 / 18
1.9e-4 /6
6.6e-3/ 25
2.3e-1 134
HyPINO 10
8.0e-5/7
1.6e-3/22
1.9e-2/ 40
2.3e-3 /15
2.7c-4
5.0e-3/22
1.2c-1 96
We summarize model performance across the seven PDE benchmarks in Table I HyPINO demon- strates consistently strong results, achieving an average rank of 2.00 across all tasks, compared to 3.00 for U-Net, 2.86 for Poseidon, and 2. 2414 for PINO. It is important to note that neither Poseidon nor PINO was originally designed for the PDE parameterization chosen in this study. As such, some degree of performance degradation is expected. In contrast, HyPINO contains a dedicated embedding mechanism tailored to this parameterization, but faces the challenge of operating in a significantly less structured output space compared to the grid-based outputs of the baselines. Its competitive
7

---

zero-shot performance under these conditions is therefore noteworthy. Across all benchmarks, models trained with physics-informee objectives generally outperform those relying solely on supervised data. This indicates that incorporating physics-based losses helps mitigate the generalization gap between the synthetic training data and the evaluation tasks.
Table I further highlights the advantages of our proposed iterative refinement approach. After three refinement iterations (HyPINO*), observe substantial reductions prediction error across all but one benchmark. Notably, the MSE for PS-C and PS-G decreases by more than one order of magnitude, and an even larger improvement is observed for HT (almost two orders of magnitude). With ten refinement iterations (HyPINO ")), our model achieves state-of- but two evaluated benchmarks, outperforming the best baseline models by factors ranging from 2.1 (on HZ against Poseidon) to 173 (HT against PINO). Table 2 shows that iterative refinement leads to a progressively more accurate prediction on the challenging WV benchmark, with the model being able to extend the undulating shape continuously further from the initial condition across the time dimension. Importantly, our results indicate that iterative refinement is not specific to HyPINO but serves generally effective test-time enhancement for other physics-informed neural operators, as demonstrated by the performance of PINO and PINO
performance on all
We hypothesize that these improvements arise because the iterative procedure allows for correcting systematic biases introduced during training on synthetic data, which, despite diversity and breadth, remains composed of relatively simple basis functions. As these training- induced errors tend to be consistent, artifacts produced by the initial SyPPNNO-generatee in subsequent iterations. This residual-driver refinement yields ensembles that are significantly more effective than naive ensembles formed from independently generated target networks.
PINNs can be systematically corrected
Table 2: Comparison of predictions and errors of HyPINO after zero, three, and 10 refinement rounds across all benchmark PDEs.
HT
H7.
HZ-G
PS-C
PS-L
PS-G
WV
Reference
Prediction
Difference
Prediction
Difference
Prediction
Difference
HyPINO
HyPINO
HyPINO
10

---

HT
R7-6
Relative Earor (5.15)
R74
1.0
PS-
PS-
WV
0.8
WV
0.2
0.0
20
Reflnmment Ronnds
Figure 3: Effect of iterative refinement HyPINO predictions across benchmarks. MSE (left) and relative error (right) as functions of refinement iterations. Relative error at iteration i is the ratio of MSE at iteration that iteration O.
Figure illustraass these trends, showing mean squared error and relative error functions of the number of refinement iterations. Consistent improvements are observed with additional iterations. The performance degradation PS-L may be attributed to the already low initial error in the zeroth iteration and the small magnitudes of the solution values, resulting correction terms that fall outside the distribution encountered during training.
4.4 Fine-tuning
The parameters 6* produced by HyPINO can be used to initialize PINNs for subsequent fine-tuning on specific PDE instances. We compare the convergence behavior of HlyPLNO-initialize PINNs with those initialized randomly and with Reptile meta-learning [37], where Reptile was trained on our synthetic dataset using 10,000 outer- and 1,000 inner-loop cycles. We also evaluate ensembles generated with HyPINO and HyPINO against ensembles of equal size and architecture initialized with random weights or Reptile.
10,000 steps using the Adam optimizer, starting with learning rate of 10-1, decayed to 10-7 via cosine schedule. Figure 4 shows convergence results on the 1D Heat Equation (HT) benchmark; results for other benchmarks and ensemble comparisons are shown
PINN fine-tuning performed
in Figures 13 and 14
iiyPiNO-initializzed PINNs consistently start with lower loss and converge to lower final error on 4 out of benchmarks. On two benchmarks, they match baseline performance, and on I, they underperform. Quantitatively, a randomly initial- ized PINN requires an average of 1,068 steps to reach the initial MSE of HyPINN-iiitializzd model. For ensembles, matching the MSE of НyPINO and HyPINO requires an average of 1,617 and 1,772 steps, respectively. Reptile- initialized PINNs converge rapidly during the first 1,000 steps, which is consistent with their meta-training configuration. However, they tend to plateau earlier and converge to higher final er- rors than HyPINO initializations. These findings suggest that, in addition to strong zero-shot per- formance, HyPINO offers robust initialization strategy for training PINNs.
Heat (IF
10-3
Repille
10:4
HyPENO
10-3
10-1
10-
Figure 4: Convergence on the ID Heat Equa- tion (HT) for randomly initialized PINNs (blue), aeptile-inttializd PINNs (orange), and HyPINO- initialized PINNs.

---

5 Conclusions & Outlook
We introduce multi-physics neural operator based on hypernetworks (HyPINO), trained on synthetic data comprising both supervised samples, constructed using the Method of Manufactured Solutions, and purely physics-informed samples without ground-truth labels. To the best of our knowledge, our framework provides the highest degree of flexibility in the input space among existing neural operators: it accommodates variations in the differential operator, source term, domain geometry (including interior boundaries), and boundary / initial conditions. Our experiments demonstrate that training this synthetic dataset enables strong zero-shot generalization across a diverse set of benchmark PDEs. This suggests that multi-physiee neural operators can be learned with significantly reduced reliance on high-fidelity, labeled training data by leveraging synthetic datasets and self- supervised objectives. In addition, we propose a lightweight and effective iterative refinement strategy that significantly improves prediction accuracy. Notably, this refinement mechanism generic and can be applied to other physics-ing physics-informed neural operator frameworks as well. We also show that HyPlNO-generated parameters provide excellent initialization for fine-tur ane-ttnnng PINNS on specific PDE instances, yielding faster convergence and lower final errors compared to both randomly initialized and Reptile-initialized baselines.
Nonetheless, several limitations remain. Our current implementation is restricted to linear 2D PDEs with spatially uniform coefficients, which narrows the class of PDEs that HyPINO can currently address. However, the framework is inherently extensible. Future work will explore increasing the input dimensionality, incorporating spatially varying coefficients, supporting nonlinear PDEs, and modeling coupled systems. Some of these extensions may be achievable through modest modifications to the data generation process, the model's input encoding architecture, or extended training. Others may necessitate increased model capacity, either through scaling the architecture or improving the target networks' parameter generation process.
References
[1] Rafael Bischof and Michael A Kraus. disture-of-exxpers-eeneeet meta-learning for physics- informed neural networks. In Proceedings of 33rd Forum Bauinformatik, 2022.
[2] Rafael Bischof and Michael A Kraus. Multi-objective loss balancing for physics informed physics-iformedd deep learning. Computer Methods in Applied Mechanics and Engineering, 439:1 17914, 2025.
[3] Lise Le Boudec, Emmanuel de Bezenac, Louis Serrano, Ramon Daniel Regueiro-Espir Yuan Yin, and Patrick Gallinari. Learning a neural solver for parametric PDEs to enhance physics- informed methods. In The Thirteenth International Conference on Learning Representations, 2025. BeLL:tttts::/operrvviei hwtp://weeeeeeeeeeeeeeeeeeeeeeee net meet/forum?id=jqqV/ VVVQQS
[4] Hu Cao, Yueyue Wang, Joy Chen, Dongsheng Jiang, Xiaopeng Zhang, Qi Tian, and Manning Wang. Swin-unet: Unet-like pure transformer for medical image segmentation. In European Conference on Computer Vision, pages 205-218. Springer, 2022.
[5] Woojin Cho, Kookjin Lee, Donsub Rim, and Noseong Park. Hypernetwork- based meta-learning for low-rank physics-informed neural networks. Advances in Neural Information Processing Systems, 36: 16691111111111. 2023.
[6] Woojin Cho, Minju Jo, Haksoo Lim, Kookjin Lee, Dongeun Lee, Sanghyun Hong, and Noseong Park. Extension of physics-infomme informed neural networks to solving parameterized pdes. In ICLR 2024 Workshop on A14DifferentallEqaaion In Science, 2024.
[7] Wojciech M Czarnecki, Simon Osindero, Max Jaderberg, Grzegorz Swirszez, and Razvan Pascanu. Sobolev training for neural networks. Advances in neural information processing systems, 30, 2017.
[8] Filipe de Avila Belbute-Peres, Yi-fan Chen, and Fei Sha. Hyperpinn: Learning parameterized differential equations with physics-informet hypernetworks. The symbiosis of deep learning and differential equations, 690, 2021.
[9] Wenhao Ding, Qing He, Hanghang Tong, and Ping Wang. Pino-mbd: Physics-informed neural operator for solving coupled odes in multi-body dynamics. arXiv preprint arXiv:2205. 12262, 2022.
10

---

[10] James Duvall, Karthik Duraisamy, and Shaowu Pan. Discretization -in.eeeeddndddeeedeentee surrogate modeling over complex geometries using hypernetworks and implicit representations. arXiv preprini arXiv:2109.07018, 2021.
[11] Chi-Mao Fan, Tsung-Jung Liu, and Kuan- Liu. Sunet: Swin transformer unet for image denoising. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS), pages 2333 1133-2337.122 IEEE, 2022.
[12] Somdatta Goswami, Aniruddha Bora, Yue Yu, and George Em Karniadakis. Physics-informed deep neural operator networks. In Machine learning in modeling and simulation: methods and applications, pages
Springer, 2023.
[13] David Ha, Andrew Dai, and Quoe V Le. Hypernetworks. arXiv preprint arXiv: 1rXiv:1609.09106 2016.
[14] Patrik Simon Hadorn. Shift-deeponet Extending deep operator networks for discontinuous output functions. Master's thesis, ETH Zurich, Seminar for Applied Mathematics, 2022.
[15] Zhongkai Hao, Zhengyi Wang. Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, pages 12556-12569 PMLR, 2023.
[16] Zhongkai Hao, Jiachen Yao, Chang Su, Hang Su, Ziao Wang, Fanzhi Lu, Zeyu Xia, Yichi Zhang. Songming Liu, Lu Lu, et al. Pinnacle: A comprchensive benchmark of physics-informed neural networks for solving pdes. arXiv preprint arXiv:2306. 08827, 2023.
[17] Zhongkai Hao, Chang Su, Songming Liu, Julius Berner, Chengyang Ying, Hang Su, Anima Anandkumar, Jian Song, and Jun Zhu. Dpot: Auto-regrossive denoising operator transformer for scale pde pre-training arXiv preprint arXiv:2403.03542.220 2024.
[18] Erisa Hasani and Rachel A Ward. Generating synthetic data for neural operators. arXiv preprint arXiv:2401.0233 2024.
[19] Maximilian Herde, Bogdan Raonić, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Em- manuel de Bézenac, and Siddhartha Mishra. Poseidon: Efficient foundation models for pdes. arXiv preprint arXiv:2405. 19101, 2024.
[20] Peter J Huber. Robust estimation of a location parameter. In Breakthroughs in statistics: Methodology and distribution, pages 492-518. Springer. 1992.
[21] Tomoharu Iwata, Yusuke Tanaka, and Naonori Ueda. Meta-learning of physics-informed neural networks for efficiently solving newly given pdes. arXiv preprinl 1xXXx:23100132700 2023.
[22] Ameya D Jagtap and George Em Karniadakis. Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Communications in Computational Physics, 28(5), 2020.
[23] Ali Kashefi and Tapan Mukerji. Physics-informed pointnet: A deep learning solver for steady- state incompressible flows and thermal fields on multiple sets of irregular geometries. Journal of Computational Physics, 468:1 11510, 2022.
[24] Jae Yong Lee, Sung Woong Cho, and Hyung Ju Hwang. Hyperdeeponet: learning operator with complex target function space using the limited resources via hypernetwork. arXiv preprint arXiv:2312.15942 15949, 2023.
[25] Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention based permutation-invariae neural networks. In International conference machine learning, pages
PMLR, 2019.
[26] Shibo Li, Tao Wang, Yifei Sun, and Hewei Tang. Multi-physics simulations via coupled fourier neural operator. arXiv preprint arXiv:2501.17296, 2025.
[27] Zijie Li, Kazem Meidani, and Amir Barati Farimani. Transformer for partial differential equations' operator learning, arXiv preprint arXiv:2205.13677 2022.
11

---

[28] Zongyi LI, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differen- tial equations. arXiv preprint arXiv:2010.08899, 2020.
[29] Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. Physics-informed neural operator for learning partial differential equations. ACM/IMS Journal of Data Science, 1(3):1-27, 2024.
[30] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. arXiv e-prints, art. MiXi:22103.14030 March 2021. doi: 10.485500arXiv22003 14030.
[31] Lu Lu, Pengzhan Jin, Guofei Pang. Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218-22) 2021.
[32] Lu Lu, Xuhui Meng, Zhiping Mao, and George Em Karniadakis. Deepxde: A deep learning library for solving differential equations. SIAM review, 63(1):208- 22-20,828,.... 2021.
[33] Spyros Makridakis. Accuracy measures: theoretical and practical concern.. International journal of forecasting, 94)):577-529 1993.
[34] Tanya Marwah, Ashwini Pokle, Zico Kolter, Zachary Lipton, Jianfeng Lu, and Andrej Risteski. Deep equilibrium based neural operators for steady-state pdes. Advances in Neural Information Processing Systems, 36: 011156166177777 2023.
[35] Michael McCabe, Bruno Régaldo-Saie Blancard, Liam Holden Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Siavash Golkar, Geraud Krawezik, Francois Lanusse, et al. Multiple physics pretraining for physical surrogate models. arXiv preprint arXiv:2310.02994, 2023.
[36] Rudy Morel, Jiequn Han, and Edouard Oyallon. Disco: learning to discover an evolution operator for muli-phhysics--gnoste prediction. arXiv preprint arXiv:2504.19996, 2025.
[37] Alex Nichol, Joshua Achiam, and John Schulman. On First-Order Meta-Leaming Algorithms. arXiv e-prints, art. arv: 1.Gv:1803: 1803.02999, March 2018. doi: 10.48550kaxivv1183.029999
[38] William Oberkampf. Frederick Blottner, and Daniel Aeschliman. Methodology for computa- tional fluid dynamics code valiiaatioooi.iiiiin....... In Fluid dynamics conference, page 2226, 1995.
[39] Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville. FiLM: Visual Reasoning with a General Conditioning Layer. arXiv e-prints, art. arXiv: SECi::1709.08771 September 2017. doi: 10.48550/arxiv. 1709.0789771111
[40] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Mhysics-informan neural networks: deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
[41] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assiste Intervention-MICCN, 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234-241. Springer, 2015.
[42] Patrick Stiller, Friedrich Bethke, Maximilian Böhme, Richard Pausch. Sunna Torge, Alexander Debus, Jan Vorberger, Michael Bussmann, and Nico Hoffmann. Large-scale neural solvers for partial differential equations. In Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and Al: 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Oak Ridge, TNN, USA, August 26-28, 2020, Revised Selected Papers 17, pages 20-34. Springer, 2020.
[43] Shashank Subramanian, Peter Harrington, Kurt Keutzer, Wahid Bhimji, Dmitriy Morozov, Michael W Mahoney, and Amir Gholami. Towards foundation models for scientific machine learning: Characterizing scaling and transfer behavior. Advances Neural Information Pro- cessing Systems, 36, 2024.
12

---

[44] Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Keil., Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems, 33:7537-7547, 2020.
[45] Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 43 (5):A355-A3088, 2021.
[46] Sifan Wang, Hanwen Wang, and Paris Perdikaris. On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 384: 13938, 2021. ISSN 0045-7825. doi: https://doi.org/11..0000 hwtp:////....1111..........aaaa. 2021. 13938. URL https: wwwp sciencedirectt.....c....... com/
[47] Sifan Wang, Hanwen Wang, and Paris Perdikaris. Learning the solution operator of paramet- ric partial differential equations with physics-informe informed deeponets. Science advances, 7(40): eabi8605, 2021.
[48] Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:110768. 2022.
[49] Yongji Wang and Ching-Yes Yao Lai. Multi-stage neural networks: Function approximator of machine precision. Journal of Computational Physies, 504:1 12865, 2024.
[50] Weidong Wu, Yong Zhang. Lili Hao, Yang Chen, Xiaoyan Sun, and Dunwei Gong. Physics-informed partitioned coupled neural operator for complex networks. arXiv preprint arXiv:2410.21025. 2024.
[51] Yibo Yang and Paris Perdikaris. Physics-informed deep generative models. arXiv preprint arXiv: 18 2018.
[52] Zhanhong Ye, Xiang Huang, Leheng Chen, Zining Liu, Bingyang Wu, Hongsheng Liu, Zidong Wang, and Bin Dong. Pdeformer-Ie 1: foundation model for -ineemmnsio partial differential equations. arXiv preprint wrXxv:2407.06666 2024.
[53] Biao Yuan, He Wang, Yanjie Song, Ana Heitor, and Xiaohui Chen. High-fidelity multiphysics modelling for rapid predictions using physics-informed parallel neural operator. arXiv preprint arXiv:2502. 19543, 2025.
[54] Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transform- ers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2022.
[55] Li Zheng, Dennis M. Kochmann, and Siddhant Kumar. Hypercan: Hypernetwork-driven deep parameterized constitutive models for metamaterials. Extreme Mechanics Letters, 72: 102243, 2024. ISSN 2352-4316. doi: https://doi..or//0..o000 1016/ eml 2024. 102243, URL https
com/ http://wwe.......oooooooo..oooon
13

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes. The abstract and introduction accurately reflect the paper's contribu- tions. HyPINO is shown to generalize zero-shot across diverse linear PDEs using a Swin Transformer-based hypemetwork trained with mixed supervision. The iterative refine- ment strategy and fine-tuning results are clearly presented and experimentally validated, supporting all major claims.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper discusses several limitations throughout the text, although not in a dedicated section. It acknowledges that the unsupervised data generation procedure may produce ill-posed PDE instances due incompatible boundary and source term con- figurations (Section 3.3) It also notes that the method is currently limited to second-order linear PDEs two dimensions either one spatial and one temporal or two spatial dimen- sions-which restricts applicability to more complex, nonlinear, higher-dimensionall problems (Section 3}. Additionally, performance degradation is observed on PS-L due to low initial етror and small solution magnitudes affecting refinement efficacy (Section Finally, the adaptation baseline models to our PDE parameterization is mentioned as factor influencing comparative performance (Section 4.3)
Justification: Answer:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
14

---

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: In addition the model architecture (Section 3.2 and the data sampling procedure (Section 3.3 in the main paper, the processes are discussed in even greater detail the appendix, including training configurations and hyperparameters. Furthermore, the code for running and reproducing results is provided in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (c.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
15

---

(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should descrihe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing verifying the results. 5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
The training dataset is created online and the code is provided in the supple- mental material. Data for the evaluation benchmarks is also included and can further be obtained from the referenced sources.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cc.
guuides/CodeSubmiesse
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed run reproduce the results. See the NeurIPS code and data submission guidelines (https
/ wttppccccccccccccccccccccccccclt http://gweeeeeeeeeeee http:///eeeeooosssssoooeeeeeeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Training details are provided in Section 4.1 and, in more detail, in the appendix. Furthermore, code to reproduce the results provided in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
16

---

7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Results are reported with two metrics (MSE and SMAPE) on seven benchmarks and show clear trends. Given the conceptual difference between the compared models (acknowledged in the results, Section 4. this work should not be viewed as "delta method" claiming outperform previous work. As such, exact comparison of the reported values does not make sense.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative етror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Provided in Section 4.1 and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform. in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Yes, our work complies with the NeurIPS Code of Ethics in all respects. It advances scientific understanding by introducing generalizable neural operator for PDEs. No real-world personal data is used; all training data is synthetic. The method poses
17

---

foreseeable misuse risk, is reproducible using public benchmarks and standard tools, and has no societal impact on individuals or communities. Training conducted efficiently on standard hardware.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The work advances scientific understanding by introducing a generalizable neural operator for PDEs commonly used in engineering. As such, no direct societal impact is expected.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The work advances scientific understanding by introducing a generalizable neural operator for PDEs commonly used in engineering. Therefore, any misuse can be mostly excluded.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
18

---

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Used libraries like DeepXDE PINNacle appropriately referenced in the paper and mentioned in the relevant parts of the code.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com /dtassts has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The synthetic datasets be regenerated any time. Instructions are provided in the codebase, including how to load the trained models from the respective checkpoints. No other new assets introduced.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
19

---

The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLM was used only for writing, editing, and formatting purposes
Guidelines:
The answer NA means that the core method development in this research does not involve L.MMs as any important, original, non-standard components.
Please refer to our LLM policy (https htcp:::://earripp.../o/ooeeeeeee /cc/oonferennce//225/// for what should or should not be described.
20

---

A Methodology
A.1 Neural Operator Architecture
We build on the HyperPINN framework g and design a neural operator based on a hypernetwork that generates the weights 0* of given PDE instance. Specifically, the hypernetwork learns a mapping
Neural Network (PINN) ue, conditioned
(c. f. 9, h) - 0* such that Nig R u,
(12)
where с denotes the vector of PDE coefficients, f the interior source term, 9 and h the Dirichlet and Neumann boundary conditions, respectively, and u the true solution.
Grid embeddings. Each grid-valued input is first passed through a Fourier feature mapping 44). which augments the input with sinusoidal encodings using five exponentially increasing bands frequencies 0.1 2', E {0, 1,2,3, 4). This enhances the network's s ability to represent high-frequency content and reduces spectral bias. The Fourier mapping layer is followed by two convolutional layers with kernel size three and strides of two. For the boundary location grids M (cf. Section 3.1). compute four embeddings: 2D zB, z/y- 2 For the boundary value grids V, compute Eg (Dirichlet values) and 2/ (Neumann values). The source term yields the embedding ey.
We define the final spatial embedding 2G by
bb C 2g + z ll N θ 22 z
(13)
where @ denotes element- wise multiplication and [-|l- ||-] denotes concatenation along the channel di- mension. This composition naturally applics spatial masking to the boundary value embeddings using the boundary location masks, ensuring that information is injected only semantically meaningful locations.
Coefficient embedding. representation zc E de using a Fourier feature encoder followed by fully connected layer.
The vector of operator coefficients IR is embedded into fixed-length
Encoding. The grid embedding is processed by sequence of K Swin Transformer blocks {SW:) K Denoting by 200 E H. xW,,, the output of block SW; and 2()) = =G. we interleave conditioned on the coefficient embeddings zc- Concretely,
each block with FiLM modulation we define
39
Tx(2), Bi(z) Ie R
(14)
via small MLPs, and write
((+1) 7(()) @ (= + B((c)),
(
(15)
where "O" denotes channel-wise scaling broadcast across spatial dimensions. This design ensures that at each stage, the latent grid features are adaptively modulated by both the global operator coefficients 2c.
Inspired by Swin Transformer U-Net architectures tations from the Swin blocks { M keep information various semantic levels.
we retain all intermediate latent represen-
Pooling, To aggregate spatial information into a compact latent representation suitable for param- eterizing the target PINN, we perform Multi-Head Attention Pooling [5] 54] across the flattened outputs of each Swin Transformer block. Specifically, let z; E [BH: xC: denote the output of the i-th Fioduladdl Swin block. We reshape it into a sequence of tokens ko; E R W.C.C which serve as both keys and values in the attention mechanism.
For each layer E {1...., K)) define set of T trainable query vectors di E TxC, where T corresponds to the number of weight and bias tensors in the target PINN. We then compute the pooled representation via multi-head attention:
Pt MultiHead htw.:/eacattttttttoon Attention,
kv, ku:), Ps E T.C.
(16)
The pooled outputs { matrix,
concatenated along the channel dimension to produce unified latent
pk] E I (E C.),
p [p1
P2
(17)
21

---

This matrix p contains one latent vector per target weight or bias tensor, each embedding multi-scale information aggregated across the Swin hierarchy. To obtain the actual PINN parameters, we apply dedicated MLP to each row of p. mapping it to the appropriate shape and dimensionality required by the corresponding weight matrix bias vector.
Target PINN.
We define the architecture of the target PINN as an MLP with Fourier feature mapping 44 and multiplicative skip connections [45]. Fourier encodings provide spectral expres- sivity for modeling high-free figufrreuencc components [46 while the skip connections enhance gradient propagation and, the context of hypernetwrrks, have the additional benefit of enabling dynamic
depth modulation based on PDE complexity by masking some layers.
Given spatial input IR, the (non-trainable) encoding is defined
(xx) [sin (2 cos (2 (2) x] R22++2
(18)
where B Nx2 contains exponentially spaced frequency bands.
Following Wang et al.
45 the encoded input projected through three parallel transformations:
20 tanh + bo); Zu tanh(Ug bu), 2u tanh(V bo);
(19)
where Wi, U E (2x((2-2)) and bu by, by E Rd.. The hidden layers of the PINN are computed via:
2++1 =u @ Lanz b.) + zo @ (1 tanh b.)), i 0, ..... 2,
(20)
with weight matrices W; E IRd and biases b; E IR., Note that we use the tanh activation due to its bounded output range, which prevents exploding values during the hypernetwork training.
The final prediction obtained by a linear transformation:
u W 2T + bouts out E Flxd bour R.
(21)
For each PDE instance, the hypernetwork therefore generates the following parameter set 0*:
(Wo, U, V, bu, by),
T-2
W Y baut
(22)
A.2
Data Sampling
A.2.1 Classes of PDFs
We construct a synthetic dataset of PDE instances by systematically sampling the governing equations, the domain, boundary conditions, source terms, and (optionally) known solutions. Two classes of samples are considered:
Class I: Supervised PDEs cally, we sample:
We generate a set of PDEs with analytical solutions via MMS. Specifi-
The differential operator L.
2. The domain Ω (along with a9).
3. An analytical solution u(x).
From the chosen solution u(x), we then compute:
The source term f(x) by applying C to u.
The boundary conditions g(x) = u(x) and / or h(x) = Ou
Om (x) by evaluating u(x) and its
normal derivative on 82.
In addition to the self-supervised physics-informe loss, samples of this class provide u(x) as well as its derivatives that can be used as additional supervised losses during training.
22

---

Class I:: Unsupervised PDEs In this class, the analytical solution u(x) is not known priori. We create samples by choosing:
The differential operator L.
2. The domain Ω (and 092).
3. The source term fxx).
4. Boundary conditions, subject to constraints designed to maximize the probability of well- posedness.
Since the ground-truth solution is not available, samples from this class rely on the self-supervised physics-informed loss train the model. The full dataset consists of mix of samples from both types, with the loss containing a switch to ignore the supervised loss if no analytical solution is available.
A.2.2 Sampling Differential Operators
Considering {u, ugy My. Ugz. dyy? to be the set of all terms that can appear our differential operators, sample the number of terms n (1,,,) We then randomly select terms from B without repetition and obtain their coefficients (cf. Section 3.1 ~ ([ The sum of the selected terms multiplied by their respective coefficients constitutes the final differential operator.
A.2.3 Sampling or Deriving the Source Terms
The source term f(xx is handled differently based on whether the sample has known analytical solution. For cases with an analytical solution, u(x) is sampled (see Section (A.2.4), and the source computed by inserting into the differential operator. For samples without analytical solution, we set the source function to a constant f(x (O, 10²).
A.2.4 Sampling Analytical Solutions via MMS
We generate analytical solutions u - R. with Ω C IR and x = (z, y). by iteratively combining randomly constructed terms, as detailed in Algorithm D The number of terms drawn from discrete uniform distribution, n Uniform (6,7, 7, I0}). The initial solution is set to zero: u(x y) 0.
Each term constructed by selecting a nonlinear function sb from a predefined library:
1
1
{z, sin, cos, tanh, + 1+ x2
The coefficients and are sampled from the set f0, Uniform([110, 10))). The remaining co- efficients c, d, are sampled as c, d, Uniform([ 2xx 2x))) term then computed as d (aaz + by + c) + e, and integrated into the current state of u(x v) using one of three ran- domly chosen rules:
Addition:
u(z, v) + u(x, v) + d vlas bbr +c u(x, y) u(x, y) d blax + +c
Multiplication:
Composition:
u(x, y) +- d vaaa u(x, v) + +c) te
A.2.5 Sampling Physical Domains
We employ randomized sampling procedure based on Constructive Solid Geometry (CSG) generate complex and diverse domains.
32 to
To begin, we define the domain Ω as the bounding box [-1, 112, representing the outer boundary MRoute. This initial outer region can describe a purely spatial or spatiotemporal domain. Although we continue to use (I, y) to denote the coordinate variables, in certain PDE classes (e.g., parabolic or hyperbolic), the variable y may represent temporal dimension, with y = -1 corresponding to the initial time. We then create inner boundaries an 0immer, (i 1,2,.... n1) by randomly generating geometric shapes (e.g., triangles, polygons, disks, rectangles) and subtracting them from the outer region using CSG operations. An example of sampled domain is shown in Figure
23

---

Initialize uz, 0
Sample n ~ Uniform( {6, .,.....1......... 10})
for i 1 do
Sample a ~ {0, Uniform( (-00, 10)
Sample b ~ {0, Uniform( (00,
Sample c, d, ~ Uniform[[-2T, 2m)
Randomly select (xxx) {sin, cos, tanh, 0, x, ((x)}
Compute term + d =0ax 1.ox++00 by + cc)++ te
Randomly choose combination rule: if add then
u(x, y) < u(x, y) + term else if multiply then
u(xr,) + u(x, y) term
else if compose then
u(x, v) d (a u(x v) + +e
end end
return u(2,8) 8)
Algorithm 1: Sampling procedure for random, differentiable functions that can be used as analytical solutions with MMS.
A.2.6 Sampling Boundary Conditions
We consider two types of boundary conditions 0Q: Dirichlet and Neumann. Note that in setting, the computational domain is Ω C [-1,1]
To maximize the likelihood of obtaining well-posed PDEs, we first categorize the PDE as elliptic, parabolic, or hyperbolic. Based on this classification, the following boundary conditions are imposed on the outer boundary vuter:
Elliptic PDEs: Dirichlet conditions Monner
u( = g(x),
X E Aauter
(23)
Parabolic PDEs: Interpreting y as time, the initial condition is enforced at = -1. In addition, we impose Dirichlet conditions the spatial boundaries:
u(x) g(x), Mogtter { 1}.
(24)
Hyperbolic PDEs: Similar to the parabolic setup, we set -1 as the initial time and enforce
u(x) g(x), E ÖD {u 1]:
(25)
du
On (x) = h(x), x ag {リ = -1..
(26)
For inner boundaries (created by subtracting geometric shapes via CSG, ef, Section A.2.5, each component 0g innnnr, with E 10......, nn,, independently assigned either Dirichlet or Neumann
condition, or both:
Ou
u(x) (x())
(x)
hx)x,,
(27)
On
For samples from Class (Section conditions follow directly from:
A.2.1 where an analytical solution u(x) is known, boundary
((x) u(x),
x MΩp
(28)
Ou
h(x)
On (x).
x E 0Ωn.
(29)
24

---

Here, and h are computed by evaluating and its normal derivative at the relevant boundary segments (outer inner). For samples from Class II, we the source term f(x) zero on the boundary (see Section A.2.3) and sample boundary values in manner consistent with C(u) to ensure that the governing equation and boundary conditions remain compatible. Specifically:
If u appears standalone term C(u], we uxx) 02.
If first-order terms such My or by appear stand-alone, allow the corresponding Dirichlet boundary value u(x) to be random constant.
Otherwise, we also include linear functions as possible boundary values.
Despite efforts to ensure well-posedness in the generation of synthetic, unsupervised training samples, some configurations may still result in ill-posed problems due to conflicting boundary constraints and source functions. Nevertheless, these unsupervised samples are essential for enabling the model to learn from domains with interior boundaries. While supervised samples can also include inner boundaries, they are, by construction, overconstrained: the combination of the source term and outer boundary conditions suffices to determine the analytical solution. As a result, the model primarily learns to represent and adapt to interior boundary effects through unsupervised data, where such features introduce structural variability without the aid of explicit targets. Given the importance of accurately modeling interior boundaries practical applications, we consider this trade-off acceptable.
25

---

B Experiments
All problems are reformulated over the canonical domain [-1, 1]2. In particular, problems originally
defined over domains [az. 6x] x [gys by] are mapped to 1] through affine transformations of the form: 2(=o,) 1, ÿ by-ay by a= 1-1, 1] are the normalized coordinates.
2(y-ay) 1, where (x, y) are original spatial coordinates and
(z, v)
B.1 Heat ID (HT)
Consider the one-dimensional heat equation:
Du 8am
XE [0,1], t [0,
(30)
Ōt
2x2²
where 0.1 denotes the thermal diffuuivity constant.
Dirichlet boundary conditions are imposed as:
u(0, t) = u( t) = 0.
(31)
The initial condition is given by a periodic (sinusoidal) function:
" 0) sin
0 <<<<< < n = 1,2, 1,2,... ...
(32)
L
where L, = is the length of the domain, and n is the frequency parameter.
The corresponding exact solution is:
11
u uxx t) exp
sin
(33)
L
This benchmark problem is adapted from DeepXDE different PDE components.
32
Figure 5
shows the parameterization of the
(a) 80p
(b) g(x)
(c) 89N
(d) h(x)
(e) f(x)
(i) u(x))
Figure 5: Parameterization of the ID Heat PDE.
B.2
Helmholtz 2D (HZ)
Consider the two-dimensional Helmholtz equation:
Au(z, y) + (((xxx))) y) = f(z. (z,8) E [-1,
(34)
where & is the number.
Dirichlet boundary conditions are imposed as:
u(-1.y) u(1, y) u(z -1) u(z, 1) 0.
commonly used instance with an analytical solution is:
[x, w) (-x² (4m)² + k²) sin(fx) sin 4mry)
(35)
u(zz (2, y) sin(srx) sin(4my).
This benehmark problem is adapted from DeepXDE 32 Figure shows the parameterization the different PDE components.
26

---

(a) p
(b) ((=)
(c) 80N
(d) h(r)
(e) f(r)
(1 v(x))
Figure 6: Parameterization of the 2D Helmholtz PDE.
B.3
Helmholtz 2D Complex Geometry (HZ--)
Consider the wwoddimensional Beesoon-Blttzaam (Helmholtz-type) equation:
-Au(x, y) + ku y) ((,,)), (x, y) E Q2
(36)
where the domain Ω = [1,,,, 1]2 Ruircle consists of the square [-1, 1] with four circular regions
removed.
The source term is defined as:
f(11 x2) = A #222 sin((1)71) (in sin(22)7222
(37)
with parameters μ4 1, #2 = 8, and = 10.
Dirichlet boundary conditions are imposed as:
u(r y) = 0.2, (I, 4) E Mree
(38)
1.0, (x, v) E 8Ω Oaginel
where Mee denotes the outer rectangular boundary, and Meciel 1 0R; are the boundaries of the interior circles.
The circles defining the removed interior regions are given by:
R1 {(z, v) (x 0.5) (y 0.5) < 0.2"}.
R₂ {(x.y) v) (r 0.4) (y 0.4) < 0.4").
R3 {(z, v): (r + 0.2) (y+ 0.7) < 0.1%).
R4 = {(x, v): (r+ 0.6)
0.5) < 0.3
This benchmark problem is adapted from PINNacle different PDE components.
16
Figure 7
shows the parameterization of the
(a) 0
(b) g(x)
(d) h(x)
(c) f(x)
u(x))
Figure 7: Parameterization of the 2D Helmholtz-type Poisson-Boltzaann) PDE with complex geometry.
B.4 Poisson 2D Circles (PS-C)
Consider the two-dimensional Poisson equation: -Au(x, y) O, (x.y) E St
(39)
where the domain is defined as rectangle with four interior circular exclusions:
Prec U
Ra
with Sree = 1-0.5 0.5)²
i=1
27

---

and the circular regions R given by:
R1 = {Ir (x 0.3) +v- <0.1")
R2 {1r,))
(x+ +0--03) <01))
(40)
R3 {(r 0.3) (y+ (3)+
R4 = {(r,y) (x + 0.3) + (y + 0.1
Dirichlet boundary conditions applied follows:
0, (1r, y) E RRa
u(x,y) v)
(41)
1 (x, y) E 0e
This benchmark problem is adapted from PINNacle 16 Figure 8 different PDE components.
shows the parameterization of the
(a) 80p
(b) g(x)
(c) 80,
(d) h(x)
(c) f(x)
(f) x(x)))
Figure 8: Parameterization of the 2D Poisson PDE with circular inner boundaries.
B.5
Poisson 2D L-Domain (PS-L)
Consider the two-dimensional Poisson equation:
Byy = 1, (x,y) E Ω
(42)
where the domain is an L-shaped region:
1=1-1,112110.11
Dirichlet boundary conditions are applied as:
u( v) 0, (x. v) ao.
(43)
This benchmark problem is adapted from DeepXDE 32 Figure 9 different PDE components.
shows the parameterization of the
(a) 80p
(b) g(x)
(d) h(x)
(c) f(x)
(f) u(x))
Figure 9: Parameterization of the 2D Poisson PDE L-shaped domain.
B.6
Poisson 2D Gauss (PS-G)
Consider the two-dimensional Poisson equation: -Au(z, y) = f(x, v), (z, v) E (0,1)?,
(44)
with homogeneous Dirichlet boundary conditions:
u( y) 0, (z, v) 02.
(45)
28

---

The source term f is defined as superposition of a random number N of Gaussian functions:
N
(x (z )² + (y My,2)²
f(x,y) = Σ exp
(46)
207
where G00m(0.4), Mz My. 000 1}. and 0; (00025, 0.1).
We select a sample from the dataset introduced in 19 Figure 10 shows the parameterization of the different PDE components.
(a) p
(b) g(x))
(c) 802
(d) h(x)
(e) f(x))
((()))
Figure 10: Parameterization of the 2D Poisson PDE with Gaussian superposition vorticity field.
B.7
Wave 1D (WV)
Consider the one-dimensional wave equation:
8uu
0uu
0, (z, E [0,
[0, 1].
(47)
d12
Oz
Dirichlet boundary conditions are imposed as:
u(0, = u(1, =0
(48)
The initial conditions are given by:
u(x,0) sin(Tx)) sin sin(4
(49)
Ou
Öt (x, 0) = 0.
(50)
The corresponding exact solution
u(,, t) = sin( (Tz) cos (2 + sin(4rz) cos(88t).
(51)
2
This benchmark problem is adapted from PINNacle [16 Figure 11 shows the parameterization of the different PDE components.
(a) 80p
(b) g(x)
00x
(d) ((x))
(e) f(x)
(1 (())
Figure 11: Parameterization of the ID Wave PDE.
B.8
Baseline Models
We compare our method against three baselines. All models are trained for 30,000 batches with batch size of 128 and an initial learning rate of 10-4
U-Net 41
convolutional encoder network that shares the same input encoding architec-
ture HININO, but replaces the Hansformer-based decoder with purely convolutional upsampling

---

stack that directly outputs a solution grid of shape (224 x 224), matching the resolution of the imput tensors. is trained exclusively on supervised PDEs with analytical solutions. using batch size of 128, an initial learning rate of 10-1, and for 30,000 training batches. The U-Net has a total parameter count of 62M.
Poseidon 19]. A large pretrained operator network with approximately 158M parameters. We use the accept all grid-based
Poseidon-B checkpoint and adapt by changing the input dimensionality inputs, Additionally, the read-time-conditioned layer normalization layers-originall designed to condition on ID time input- are modified condition on the 5D vector of differential operator coefficients. Poseidon is fine-tuned exclusively on supervised samples, using the same training setup as the U-Net (30,000 batches, batch size 128, initial learning rate 10 4).
PINO 29 Fourier neural operator 28 architecture with 33M parameters, trained with joint physics-informed and supervised losses computed in Fourier space. We adapt the model to accept 5-channel grid inputs and condition on the PDE operator using FiLM layers. It follows the same hybrid supervision and training curriculum HyPINO, including physics-informed losses, and is trained for 30,000 batches with a batch size of 128 and an initial learning rate of 10-4.
30

---

B.9 Evaluation
100
C00
02
(a)
(b)
(c)
(d)
(c)
Figure 12: Visual progression of iterative refinement across different samples. Each row shows: (a) HyPINO prediction (9), (b) 1st Refinement (1) (c) 2nd Refinement (2) (d) Final prediction (0) + (1) (2) and ground truth (e).
31

---

B.10 Fine-tuning
We investigate the utiliity of SEPPINO-generane PINN parameters 8* as a prior for rapid adaptation to specific PDE instances. We compare three initialization strategies: (i) randomly initialized PINNs, and (ii) PINNS initialized via Reptile meta-learning [37]. For Reptile, we use 10,000 outer-loop steps on our synthetic dataset and 1,000 inner-loop updates per sample with an inner learning rate of 0.01.
PINNs, (ii)
In addition to single-network performance, we evaluate the effect of initialization on ensemble- methods. We compare ensembles of size and 10 generated using HyPINO (denoted HyPINO* and HyPINO"") against ensembles of the same size and architecture initialized either randomly or via Reptile. For the latter, we replicate the Reptile-initialized weights across all ensemble members. Note that a HyPINO' ensemble consists of the base PINN as well as refinement (or delta) PINNs:
(0) t= ( thus creating an ensemble with + experts.
Convergence behavior across all PDE classes reported in Figures 13 and 14
Bus 1E3
1558
2007
IbTEN0
Belriiaee 20 1821
Sdabilo 2D 250
1931
200
30-
Becake
Decubee
Titeaadth 2tt--tt--eeeeeee Glnratry (IC44)
Hebriddo 13 Crepte Cereutty 2006)
Bobrlddo 23- Creple Cenmatty (056
Taets
Prisen Crder (9543
Pobace 20 Chele (P9-2)
Polore 20 Chica
206
1200
(b)
(c)
Figure 13: Convergence of PINNs when fine-tuned on each of the benchmark PDE problems. We
compare the convergence of different ensemble sizes: (a) single PINN, (b) ensemble of size 4 (c) ensemble of size 11, where an ensemble of size is an ensemble of randomly initialized PINNs (blue), PINNs initialized via Reptile (orange), or PINN initialized via HyPINO followed by i refinement rounds (green).
32

---

1558
1532
2007
30*
(99)
(0/WW0
10WWWN
I9NN 1LTE50
11500
AN17
10-
(a)
(b)
(c)
Figure 14: Convergence of PINNs when fine-tuned on each of the benchmark PDE problems. We
compare the convergence of different ensemble sizes: (a) single PINN, (b) ensemble of size 4 (c) ensemble of size 11, where an ensemble of size is an ensemble of randomly initialized PINNs
(blue), PINNs initialized via Reptile (orange), or one PINN initialized via HyPINO followed by i refinement rounds (green).
Resolution Invariance Ablation
Discretization-inaaiia is important property for neural operators. While the output of HyPINO is a continuous PINN that can be evaluated at arbitrary spatial coordinates, the input PDE parameteri- zation (source function and boundary masks/values) is discretized to match the Swin Transformer's 's input resolution. Following prior work [19], this limitation can be mitigated by demonstrating test-time resolution invariance when varying the input grid resolution and resizing it to (224 x 224).
fixed-size grid (224 224)
We performed this ablation the Helmholtz benchmark (HZ) by changing the source function resolution between 28 and 448. The results are shown in Table
Table 3: Resolution invariance ablation on the Helmholtz benchmark (HZ). Each cell reports SMAPE across different input grid sizes, resized to 224 224.
28
112
140
SMAPE
38.04
35.78
35.91
36.00
36.05
36.05
196
224
280
336
392
448
SMAPE
36.05
36.04
36.05
36.03
36.04
36.04
Between resolutions 56 and 448, SMAPE varies by less than 0.3, which indicates approximate invariance. Only at very coarse resolutions (28 28) does the performance begin to deteriorate.
33

---

L-BFGS Fine- Tire-Tnnnng with Different Initializations
Our initial choice evaluate fine-tuning performance using the Adam optimizer was motivated by its wide adoption in the PINN literature. To test whether HyPINO initializations also benefit second- order optimization, we conducted additional fine-tuning experiments using L-BFGS, chosen for its broad adoption and ease of use within PyTorch. All runs used standard I.-BFGS hyperparameters without tuning.
Table 4: Iterations required to match the initial MSE of HyPINO-initializee PINN.
HT
HZ
HZ-G
PS-C
PS-L
PS-G
WV
Random Init
20
N/A
36
35
Reptile Init
4
22
211
22
65
9
27
On PS-C and PS-L, Reptile requires 22 and 65 L-BFGS steps, respectively, match HyPINO's starting error, while random initialization needs 36 and 34, On HZ-G, Reptile requires 211 steps, while random never reaches HyPINO's 's initial accuracy.
Table 5: Final MSE after L-BFGS fine-tuning.
HT
HZ
HZ-G
PS-C
PS-L
PS-G
WV
Random Init
2.93c-9
1.15e-7
2.89c-1
3.18c-4
7.05e-5
5.69c-4
2.68e-2
Reptile Init
2.69e-9
2.18e-7
3.55e-2
9.34e-4
8.66e-5
5,68e-4
3.80e-4
HyPINO Init 1.62e-9
1.52e-7
1.74e-2
8.19e-5
6.87e-5
5.69e-4
1.94e-2
The results show that HyPINO initializations remain effective with L-BFGS. HyPINO achieves the lowest final MSE on four benchmarks (HT, PS-C, PS-L., HZ-G) and is competitive on PS-G. Only on WV does Reptile achieve the best result, while HZ, random slightly outperforms HyPINO. These differences are especially meaningful given the high cost of L-BFGS iterations.
34


Reviews:
[{"review_id": "5byFKsUHuE", "reviewer": "Reviewer_9qpF", "summary": "This paper uses a hyper-network (based on the SwinTransformer Architecture) that predicts the parameter of a PINN architecture for different PDEs. This is a new way to combine operator learning and PINNs where the operator learning part is really a hyper-network that predicts the parameters of PINN for a given PDE.\n\nThe authors show that their methodology of first predicting the parameters of a PINN rather than directly training a PINN from scratch is out perform PINN base baselines. The authors further provide a delta trick (where the error in the proposed solution and the true solution is defined as another PDE that is predicted as a PINN), that iteratively reduces the error of HyPINO over datasets, often achieving ~100x L2 error 1D heat equation,  Helmholtz equation as compared to other operator based baselines. \n\nMost of the experiments performed by the authors are on time-independent PDEs and hence the authors are also able to use methods of manufactured solutions, to generate synthetic training data to train the model.", "questions": "From the way I understand, the hyper-network takes as input a grid based solution (since its a Swin Transformer). However, is the PINN also evaluated on the same points, or any randomly sampled point in the domain? I think this a bit unclear, and maybe should be made a bit more explicit in the methods section.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "I think that this is quite an interesting work, where the authors are able to compete with operator based baselines such as PINO, and at the same time able to achieve the same property of a PINN, i.e, ideally the ability to sample from any point in the domain. \n\nFurthermore, the delta PINN trick of progressively improving the solution of the PDE is quite interesting. \n\nA potential negative here is that the experiments are mostly shown for steady-state PDEs, and it would be interesting to see how the method performs for time-dependent dynamics. \n\nFurthermore, it will be interesting to see how well the work does on 3D PDEs, since ideally it can pretty simply bring down the compute cost if this methodology works well. However, I think that the paper introduces an interesting methodology. \n\nFew baselines and works to consider (that at least should be cited since they are relevant) are: \n\n- DISCO: learning to DISCover an evolution Operator for multi-physics-agnostic prediction\n- Deep Equilibrium Based Neural Operators for Steady-State PDEs\n\nI think the former is very similar in approach that they try to use a hyper-network to somehow compress the dynamics of the model, and the latter is a neural operator for a steady state PDEs with a similar data generation methodology.", "was_revised": true, "final_justification": "I will keep my score since the reviewers have addressed most of my concerns in the rebuttal. I think that this is an interesting paper and methodology that can be useful for the community.", "rebuttal": "We thank the reviewer for taking the time to carefully evaluate our work and for the constructive feedback. We appreciate your insights and address your points below.\n\n[W1:] While it is true that most of the experiments focus on steady-state PDEs, our evaluation also includes the 1D wave equation (WV), a time-dependent hyperbolic PDE. HyPINO demonstrates competitive zero-shot performance on this benchmark (MSE: 2.9e-1), outperforming UNet and Poseidon, and matching PINO.\n\n[W2:] The reviewer is correct in stating that the current work is limited to PDEs in (at-most) two space dimensions. However, our framework is very general and can be readily extended to 3-D PDEs, which we aim to consider in future work. \n\n\n[W3:] Thank you for highlighting these relevant works. We will include and discuss them in the related works section of the final version.\n\n[Q1:] Indeed, the choice and sampling procedure of collocation points used to evaluate the generated PINNs could be explained further. The target PINNs are evaluated on 4,096 randomly sampled points, where 2,048 lie within the domain, and 2,048 are sampled on the boundaries. The reasons for this are twofold: first, sampling on the entire grid (over domain and boundary) would yield 224*224 collocation points, which would not fit into memory. Second, by sampling explicitly on the boundary, we ensure better coverage and a stronger training signal for Dirichlet and Neumann boundary conditions. If we would only rely on points from the sampled grid, very few would lie exactly on the boundary, especially for complex shapes, which could lead to weak or noisy training signals for the boundary conditions.\nThis sampling process will be clarified and more detailed in a camera-ready version of the paper, if accepted."}, {"review_id": "qbX8UkyjZG", "reviewer": "Reviewer_NMyN", "summary": "This paper proposes a transformer-based hypernetwork architecture for predicting the solutions of PDEs with a target main network, the weights of which are generated by a hypernetwork that takes as input the PDE coefficients, source function, and boundary functions, coded as matrices. The training data contains only manufactured solutions and physics information, and in particular contains no expensive simulations. The target main network is an MLP with Fourier-feature embedding and multiplicative skip connections. An important limitation of the method is that currently it can only handle simple linear PDEs (up to second order).", "questions": "The method appears extensible to nonlinear PDEs, except perhaps for the residual-driven refinement process. Was this attempted, and the results weren't satisfactory? The restriction to low-order linear PDEs is the reason I didn't give 4 to quality.\n\nThe way that the binary mask for the boundary conditions is generates needs to be better explained, e.g., I couldn't understand what this means: \"a value of 1 to the four grid cells closest to each boundary point and zero elsewhere\".\n\nWhat constitutes a \"neural operator\" is a bit controversial. Theoretically, they act on infinite-dimensional function spaces, but in practice, they need to be discretized. Approaches in the literature say that a true neural operator must be \"discretization-invariant,\" or \"discretization-convergent.\" Could the authors comment on this issue and how this affects their approach? Have different discretizations (grid sizes) at the input of the hypernetwork been tried at training and testing times?", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n+ The training data contains no expensive simulations but exploits manufactured solutions and physics-regularization. \n+ The construction of the training data set is very general, allowing source terms, Dirichlet and Neumann boundary conditions, and complex geometries.\n+ The main network is a small PINN, which is therefore efficient and meshless.\n+ The resulting main network can be further fine-tuned and the \n+ The use of analytical solutions allows the inclusion of a Sobolev loss term, which as far as I know, is novel in the physics-informed machine learning area.\n\nWeaknesses:\n\n- The main obvious limitation is the low-order linear PDE constraint.\n- It is difficult to set up a fair comparison against the Poseidon and PINO approaches, as they use different training data, model size and training time needs to be taken into account, etc. Therefore, the results in Table 1 needs to be considered with care.\n- The iterative refinement process relies on superposition, so it can't be extended to nonlinear PDEs.\n- The novelty in the architecture itself is low, as both hyperPINNs and Swin transformers are known. But this is a minor weakness.", "was_revised": true, "final_justification": "I am overall satisfied with the author's responses and will keep the score as is.", "rebuttal": "We thank the reviewer for taking the time to carefully evaluate our work and for the constructive feedback. We appreciate your insights and address your points below.\n\n[W2:] All baselines were fine-tuned on our synthetic dataset using a similar training budget (same number of steps, batch size, and optimizer settings). While the UNet and Poseidon are trained without the physics-informed objective, PINO was trained with the same settings as HyPINO. We agree that the FNO architecture in PINO was never intended for the type of PDE parametrization used in this work (i.e., binary indicator grids for the boundaries), which might make it less effective than a Swin Transformer. Conversely, HyPINO must predict a full set of PINN weights, which is more challenging than predicting a solution grid. We acknowledge the difficulty of direct comparison in the evaluation Section 4.3 on line 250. \n\n[W3:] It is correct that the iterative refinement process relies on linearity. That said, some non-linear PDEs can be linearized, making the method also applicable to those.\n\n[Q1:] Our framework can be extended to 3D, non-linear, and higher-order PDEs with only minor modifications, such as adding non-linearities like $u u_x$ to the set of selectable terms when sampling the differential operator, or adding a third variable and changing the dimensionality of the inputs to the Swin Transformer and target PINNs. However, more computational resources would be necessary to train such a model. We want to highlight that, to the best of our knowledge, HyPINO is the first neural operator trained on such a diverse set of linear, 2D PDEs with mixed Dirichlet and Neumann boundary conditions on complex domains. \n\nWhile our current framework focuses on linear PDEs, this still captures a broad and impactful class of PDE problems in engineering and the natural sciences, often sufficient for many modelling applications or as first steps into non-linear modelling approaches. Moreover, we have experimented with linearization techniques (e.g. the Cole–Hopf transformation for the 1D Burgers equation) as a practical extension strategy, which enables HyPINO to handle certain nonlinear PDEs. However, we agree this work should trigger novel research directions, such as including non-linear examples directly in the training pipeline.\n\n\n[Q2:] Indeed, the part about constructing the binary mask for the boundary conditions could use some additional information. We sample continuous boundary points directly on the CSG-defined shapes and need to convert them into a grid-based discretization for the Swin Transformer. For each point on the boundary, we find the enclosing grid cell it lies in and set the four corner points of that cell to 1 in the binary mask (indicating that the boundary passes through this region). For example, a boundary point at $(0.5, 0.5)$ activates the grid points $(0,0)$, $(1,0)$, $(0,1)$, and $(1,1)$. This gives the binary masks $M_g$ and $M_h$. The corresponding boundary values are stored in $V_g$ and $V_h$ at the same locations. This can be clarified in the final version.\n\n[Q3:] We agree that discretization-invariance is a key property for neural operators. While the output of HyPINO is a continuous PINN that can be evaluated at arbitrary spatial coordinates, the input PDE parametrization (source function and boundary masks/values) is discretized on a fixed-size grid (224×224) to match the Swin Transformer’s input resolution. Following prior work [1], this limitation can be mitigated by demonstrating test-time resolution invariance when varying the input grid resolution and resizing it to (224x224). We performed this ablation on the Helmholtz benchmark (HZ) by changing the source function resolution between 28 and 448:\n\n$$\n\\\\begin{array}{|c|c|c|c|c|c|c|c|c|c|c|c|c|}\n\\\\hline\n\\\\textbf{Grid Size} & 28 & 56 & 96 & 112 & 140 & 168 & 196 & 224 & 280 & 336 & 392 & 448 \\\\\\\\\n\\\\hline\n\\\\textbf{SMAPE} & 38.04 & 35.78 & 35.91 & 36.00 & 36.05 & 36.05 & 36.05 & 36.04 & 36.05 & 36.03 & 36.04 & 36.04 \\\\\\\\\n\\\\hline\n\\\\end{array}\n$$\n\nBetween resolutions of 56 and 448, SMAPE varied by less than 0.3, which shows approximate invariance. Only at very coarse resolutions (28x28) does the performance start deteriorating. This experiment can be added to the appendix.\n\n[1] Herde, Maximilian, et al. \"Poseidon: Efficient foundation models for pdes.\" Advances in Neural Information Processing Systems 37 (2024): 72525-72624."}, {"review_id": "9SIEyYDojT", "reviewer": "Reviewer_ydBg", "summary": "This paper proposes a novel neural operator architecture that integrates PDE residual loss into the training process, enabling self-supervised learning using only initial and boundary conditions. Inspired by HyperNetworks and Swin Transformers, it introduces a modular and scalable design with enhanced generalization and reduced reliance on labeled PDE solutions. A differentiable sampling strategy allows the model to backpropagate through PDE solvers to generate supervision signals. The method achieves strong performance across several multi-physics PDE benchmarks, outperforming baseline neural operators with significantly fewer training samples.", "questions": "1. Resolution Equivariance: Can the method generalize to different spatial resolutions at inference time? Since Swin Transformers rely on fixed-window self-attention, resolution equivariance—a core property of neural operators—may not hold. Please clarify and, if possible, include supporting experiments or theoretical discussion.\n\n2. Ablation Studies: The architecture includes both HyperNetwork modulation and Swin-style attention. Could the authors provide ablation results isolating the effect of each component on performance and generalization?\n\n3. Computational Overhead: Backpropagating through PDE solvers may introduce significant training costs. Please quantify the runtime and memory overhead compared to standard supervised neural operators.\n\n4. Related Work: Consider citing and discussing relevant works such as MPP, DPOT, PDE-Former, OFormer, and Transolver to better contextualize the novelty and contribution of the method.", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper presents a novel neural operator framework that combines architectural innovations with a self-supervised PDE-informed training scheme. Its use of PDE residual loss to generate supervision signals from initial and boundary data significantly reduces dependence on labeled PDE solutions, a major practical advancement. The integration of ideas from HyperNetworks and Swin Transformers leads to a scalable and modular design with strong empirical generalization. The method is validated on a diverse set of benchmark PDE problems, demonstrating superior performance with limited data. The paper is clearly written, technically sound, and makes a strong case for its significance and originality.\n\nWhile the contributions are substantial, several concerns merit further clarification and improvement:\n\n1. Resolution Equivariance Unclear: A core property of neural operators is resolution equivariance—the ability to generalize across discretizations. However, the architecture incorporates Swin Transformers, which operate over fixed local windows. It is unclear whether NEMO can generalize across resolutions, and no experiments directly evaluate this. If the model is not equivariant, it would be more accurate to frame it as a physics-informed transformer rather than a neural operator.\n\n2. Training Efficiency and Complexity: The differentiable sampling approach involves backpropagation through PDE solvers, which could be computationally intensive. A more detailed analysis of training time and computational requirements would help assess the method’s practicality.\n\n3. Ablation and Component Justification: While the architectural design is rich, the paper lacks deeper ablation studies. For instance, what is the contribution of HyperNetwork-style modulation versus Swin-style self-attention? How critical is the differentiable sampling step versus simply adding residual supervision at predefined grid points?\n\n4. Missing Citations: The paper overlooks relevant prior work in PDE pretraining and transformer-based operators. Notable omissions include MPP, DPOT, and PDE-Former (for physics-based pretraining), as well as OFormer, GNOT, UPT, and Transolver (for transformer-based operators). These omissions weaken the related work discussion and make it difficult to fully assess the novelty in context.", "was_revised": true, "final_justification": "The authors have addressed my concerns, and I will keep my score.", "rebuttal": "We thank the reviewer for taking the time to carefully evaluate our work and for the constructive feedback. We appreciate your insights and address your points below.\n\n[Q1:] We agree that resolution-equivariance is a key property for neural operators. While the output of HyPINO is a continuous PINN that can be evaluated at arbitrary spatial coordinates, the input PDE parametrization (source function and boundary masks/values) is discretized on a fixed-size grid (224×224) to match the Swin Transformer’s input resolution. Following prior work [1], this limitation can be mitigated by demonstrating test-time resolution invariance when varying the input grid resolution and resizing it to (224x224). We performed this ablation on the Helmholtz benchmark (HZ) by changing the source function resolution between 28 and 448:\n\n$$\n\\\\begin{array}{|c|c|c|c|c|c|c|c|c|c|c|c|c|}\n\\\\hline\n\\\\textbf{Grid Size} & 28 & 56 & 96 & 112 & 140 & 168 & 196 & 224 & 280 & 336 & 392 & 448 \\\\\\\\\n\\\\hline\n\\\\textbf{SMAPE} & 38.04 & 35.78 & 35.91 & 36.00 & 36.05 & 36.05 & 36.05 & 36.04 & 36.05 & 36.03 & 36.04 & 36.04 \\\\\\\\\n\\\\hline\n\\\\end{array}\n$$\n\nBetween resolutions of 56 and 448, SMAPE varied by less than 0.3, which shows approximate invariance. Only at very coarse resolutions (28x28) does the performance start deteriorating. This experiment can be added to the appendix.\n\n[1] Herde, Maximilian, et al. \"Poseidon: Efficient foundation models for pdes.\" Advances in Neural Information Processing Systems 37 (2024): 72525-72624.\n\n[Q2:] In HyPINO, Swin blocks are used to encode the discretized inputs (i.e., the source fields, domain geometry, and boundary conditions) into a latent representation. To condition this representation on the PDE operator, we apply FiLM layers after each Swin block. These layers modulate the intermediate latent features conditioned on the PDE coefficients $c \\in \\mathbb{R}^5$. This is a lightweight and common mechanism for global conditioning, consistent with prior work in neural operators [1], and better suited than cross-attention in our setting, because the conditioning signal is a 1D vector rather than a sequence of tokens. \n\nIn the decoder, we adopt a hypernetwork-style design: pooled latent features are passed through MLPs to generate the weights and biases of the target PINN. \n\nThe Swin encoder, the FiLM-conditioning, and the hypernetwork decoder serve complementary roles, which makes isolated ablation of individual components less straightforward. However, while designing the architecture, we experimented with different Transformer backbones, MLP widths, and activation functions. We found these architectural choices to have relatively minor influence on overall performance compared to the quality of the dataset and the choice of loss functions, such as the addition of the physics-informed loss, whose effectiveness is evidenced by the superior performance of HyPINO over the UNet baseline that shares the same encoder. We therefore mostly adopted standard architectural settings from prior work.\n\n[Q3:] We would like to emphasize that HyPINO does not backpropagate through any external numerical PDE solver. Our framework is entirely neural: HyPINO is a hypernetwork that maps a PDE specification $(L, f, g, h)$ to the weights $\\theta^\\star$ of a small target PINN $u_{\\theta^\\star}$. This PINN approximates the PDE solution as a continuous function over the domain. Residuals are computed by applying the differential operator $L$ to $u_{\\theta^\\star}$ via automatic differentiation, and the training objective is to minimize the residuals.\n\nAt inference time, our residual-based refinement strategy also avoids solver-level differentiation. We compute the residual of the current PINN and pass it through the hypernetwork in forward mode only to obtain a corrective delta-PINN. The updated prediction is then formed by summing the PINN outputs. Only the small PINNs are differentiated to compute residuals whereas the hypernetwork remains frozen.\n\nRegarding computational costs, to obtain the loss of a predicted PINN $u_{\\theta^\\star}$, we evaluate it on a set of collocation points and compute the necessary first- and second-order derivatives to obtain the residuals. These higher-order derivatives are computed efficiently via automatic differentiation of the small PINN only (3 layers, width 32). The resulting residual loss is then backpropagated to update the hypernetwork. Importantly, this gradient only flows through the Swin-based encoder at the final step after residuals have been computed. Thus, while residuals require evaluating derivatives up to second order, the computationally expensive hypernetwork is only differentiated once per batch.\n\n\n[Q4:] Thank you for pointing out these relevant works. We will include and discuss them in the related works section of the final version."}, {"review_id": "UOF5wUv3Lb", "reviewer": "Reviewer_7tHo", "summary": "The manuscript provides a hybrid neural operator approach combining synthetic supervised learning data generated by automatic differentiation with a physics-informed loss. The method is called HyPINO and aims to predict the parameters of a network approximating the solution of a PDE based on a discretization of the data. The method is tested on a variety of two-dimensional PDEs for zero-shot accuracy. Further, it is investigated whether the output of HyPINO can be used as an initialization strategy for physics-informed training. The results compares favorable to several baselines.", "questions": "+ Can you point out the key algorithmic innovation(s)? Is this the combination of the existing ideas on neural operators to achieve improved performance? \n+ Can you incorporate more state-of-the-art optimizers for PINNs when fine-tuning? \n+ In the subsection regarding the loss function, the supervised loss is not mentioned. However, from my understanding, you are using both a supervised and physics-informed loss during optimization. Can you clarify this?", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n+ The manuscript addresses an important and timely problem in scientific machine learning. \n+ The proposed method performs well compared to the considered baselines. \n\n**Weaknesses**\n\n+ The manuscript proposes a whole pipeline, which makes it hard to distill the conceptual and algorithmic innovations. In particular, the combination of supervised and physics-informed losses for neural operators has been around for a couple of years, the method of manufactured solutions is classic, and the hypernetwork HyPINN was used before. As there are many components, an ablation study would provide much improved insight to the contribution of the individual components of the overall pipeline. \n+ When finetuning the PINN's parameters, Adam is used. However, it is more or less consensus that second-order optimizers are far superior in physics-informed learning, in particular Gauß-Newton type methods. \n\n**References**\n\n*Physics-informed neural operator for learning partial differential equations*, Z Li, H Zheng, N Kovachki, D Jin, H Chen, B Liu, K Azizzadenesheli, A Anandkumar, ACM/JMS Journal of Data Science, 2024\n\n*Achieving High Accuracy with PINNs via Energy Natural Gradients*, J Müller, M Zeinhofer, ICML 2023\n\n*Challenges in training PINNs: A loss landscape perspective*, P Rathore, W Lei, Z Frangella, L Lu, M Udell, ICML 2024", "was_revised": true, "final_justification": "I have raised my score based on the reply, but I am still recommending a rejection. \n\nI believe that the manuscript has potential with many different ideas, but the technical elaboration and scientific testing of the proposed methodologies should be improved.. In particular, to two main methodologies proposed in the manuscript—the use of MMS for NOs and iterative refinements—miss a thorough evaluation, including ablation studies, and are only demonstrated for a specific neural operator.", "rebuttal": "We thank the reviewer for taking the time to carefully evaluate our work and for the feedback. We appreciate your insights and address your concerns below.\n\n[Q1:] While our approach builds on prior work, the contributions of HyPINO go significantly beyond a simple combination thereof. Neural operators such as FNOs and CNOs typically require large datasets of high-fidelity simulation data for training. The introduction of the physics-informed loss in PINO reduced this dependency significantly. However, some supervised data is still necessary, as training solely with physics-informed objectives is often unstable due to spectral bias and optimization challenges [1]. This remaining dependency on simulation data is problematic when designing multi-physics solvers, as generating and combining data for diverse physical problems in a single dataset is challenging. This is why we propose to leverage the method of manufactured solutions (MMS) to obtain an (potentially) infinite number of diverse training samples. To the best of our knowledge, HyPINO is the first neural operator trained on a dataset spanning such a diverse set of linear, 2D PDEs with mixed Dirichlet and Neumann boundary conditions on complex domains. \n\nThis infinite dataset also allows us to revisit the previously underexplored HyperPINN design, which is a more data-intensive neural operator than CNOs or FNOs because it generates a continuous target function parametrized by a PINN as opposed to the grid-based outputs of CNOs and FNOs. The benefits of HyperPINNs are that (i) the output PINN is continuous and can be evaluated at arbitrary points on the domain, (ii) exact analytical derivatives can be obtained directly via automatic differentiation, and (iii) the computation of these derivatives is computationally inexpensive because the gradients for computing the residual only need to be backpropagated through the lightweight PINN rather than the large Swin Transformer. While previous work has recognized these benefits and used HyperPINNs as backbones of their neural operators, their scope was typically restricted to minor variations of a single equation or physical system (e.g. different source terms for the same PDE, or the same equation on multiple geometries). Through the combination with the diverse, infinite MMS-dataset we are able to unlock the potential and scale HyperPINNs to true multi-physics neural operators. \n\nFurthermore, while HyPINO shows strong zero-shot performance for a diverse set of benchmark PDEs, consistently outperforming other baseline operators, it also naturally supports task-specific fine-tuning. This is done by taking the weights generated by HyPINO as initialization and training the resulting PINN with standard physics-informed objectives using analytical derivatives to compute and minimize the residual. This task-specific fine-tuning is impossible with most other neural operator designs because they do not produce a parameterized solution that is separate from the model. \n\nFinally, we introduce the iterative refinement process, a training-free method that leverages superposition to improve the predictions of physics-informed neural operators for linear PDEs. Given an initial PINN solution from HyPINO, we can compute its PDE residual (i.e. how much it violates the PDE). This residual function can be fed back into the hypernetwork to generate a “delta-PINN” representing a correction to the original solution. Adding the delta-PINN’s predictions to the original PINN’s predictions yields an updated solution that cancels out much of the previous residual. This procedure can be iterated multiple times, thus creating an ensemble of self-correcting networks. This refinement does not require any retraining of the hypernetwork. It is a post-processing technique that uses the trained neural operator model repeatedly in inference mode. To the best of our knowledge, this iterative residual refinement idea with delta-PINNs is also entirely novel and possibly even applicable to other physics-informed neural operator designs, such as PINO, to improve their predictions on linear PDEs.\n\n[1] Goswami, Somdatta, et al. \"Physics-informed deep neural operator networks.\" Machine learning in modeling and simulation: methods and applications. Cham: Springer International Publishing, 2023. 219-254.\n\n\n[Q2:] We agree that second-order optimizers such as L-BFGS or Gauss-Newton methods were shown to improve convergence in physics-informed training. In fact, many state-of-the-art PINN setups adopt a two-stage scheme with Adam followed by a second-order optimizer.\nHowever, in our experiments, the goal was not to reach state-of-the-art performance on each benchmark PDE, but to isolate and assess the impact of HyPINO’s initialization quality. To do so, we deliberately chose a minimalistic and widely used training configuration so that improvements could be directly attributed to the initialization. As shown in Figure 4, HyPINO-initialized PINNs (a) start with lower error, (b) have well-behaved convergence curves, and (c) reach lower errors faster than random and Reptile initializations.\nWhile we recognize that techniques such as Sobolev losses, dynamic loss weighting, adaptive sampling, and second-order optimizers can improve PINN performance, our aim was to evaluate HyPINO initializations in a clean and controlled setting. Including such enhancements would have shifted the focus from the quality of the initialization itself and introduced confounding factors that would have made a direct comparison difficult.\nThat said, our method is fully compatible with more advanced PINN training strategies. If desired, these can easily be added on top during downstream adaptation. While we did not run experiments with second-order optimizers or other advanced PINN techniques, the benefits of HyPINO initializations are likely to carry over, as these strategies are complementary and largely independent of the initialization scheme.\n\n[Q3:] Our training objective indeed combines both physics-informed and supervised terms when analytical solutions are available via MMS. Specifically, the loss used for training is $ J=\\lambda_R J_R+\\lambda_D J_D+\\lambda_N J_N + J_S $, where $J_R$ is the residual loss over interior collocation points, $J_D$ and $J_N$ are Dirichlet and Neumann boundary losses, and $J_S$ is a Sobolev supervised objective computing a Huber loss on function values, first, and second derivatives between the generated PINN and the analytical solution at sampled collocation points. This is detailed in Section 3.4 (Eq. 8). The Sobolev loss [2] was shown to significantly improve the accuracy and generalization of neural networks, including PINNs [3]. However, computing higher-order derivatives required for the Sobolev loss is typically computationally expensive and thus impractical in many applications. In our setting, the necessary derivatives are already obtained as part of the residual evaluation for the physics-informed loss, which allows us to incorporate the Sobolev loss without additional computational overhead.\n\n[2] Czarnecki, Wojciech M., et al. \"Sobolev training for neural networks.\" Advances in neural information processing systems 30 (2017).\n\n[3] Son, Hwijae, et al. \"Sobolev training for physics informed neural networks.\" arXiv preprint arXiv:2101.08932 (2021).\n\nWe thank the reviewer again for their comments and hope to have addressed their concerns satisfactorily. If so, we kindly request the reviewer to upgrade their assessment accordingly, while being at their disposal for any further questions about our paper."}]

Author rebuttal:

Meta-review: (1) Summary of this work: This paper presents HyPINO, a hyper-network (based on the SwinTransformer Architecture) that predicts the parameters of a PINN architecture for different PDEs. To enhance the generalizability, HyPINO is optimized with data generated by MMS and unlabel data optimized by PINNs. Besides, an iterative refinement procedure is presented to finetune the generated PINN parameters for new tasks. HyPINO demonstrates strong performance in diverse PDEs.

(2) Strengths and weaknesses: Although the HyPINN framework is new, this paper presents a good unification of MMS-generated data and PINNs. The iterative refinement procedure is interesting. I think the whole framework can be inspiring for future exploration of generalizable PINNs. One main concern, as pointed out by Reviewer 7tHo, this paper lacks a detailed ablation study, since there may be manual and unexplained designs, such as swin Transformer.

(3) Summary of rebuttal: During the rebuttal, the authors have made a detailed analysis of optimizers, as well as the resolution-invariant property. Although the novelty of using MMS for PINO is argued by one Reviewer, I think introducing MMS to this community is valuable and the authors have provided a reasonable way to utilize the generated data.

**Final decision:** This paper presents a promising and effective framework to enable generalizable PINNs, which has been highly acknowledged by 3 of 4 reviewers. After carefully checking all the reviews and discussions, I think the authors have provided sufficient evidence to address the reviewer's concerns. Thus, I recommend an acceptance (spotlight).
Decision: Accept (spotlight)