PDEfuncta: Spectrally-Aware Neural Representation for PDE Solution Modeling
Authors: Minju Jo
Woojin Cho
Uvini Balasuriya Mudiyanselage
Seungjun Lee
Noseong Park
Kookjin Lee
Venue: NeurIPS 2025 poster ()
Keywords: SciML
meta-learning
data compression
Primary area: machine_learning_for_sciences

Abstract: Scientific machine learning often involves representing complex solution fields that exhibit high-frequency features such as sharp transitions, fine-scale oscillations, and localized structures. While implicit neural representations (INRs) have shown promise for continuous function modeling, capturing such high-frequency behavior remains a challenge—especially when modeling multiple solution fields with a shared network. Prior work addressing spectral bias in INRs has primarily focused on single-instance settings, limiting scalability and generalization. In this work, we propose Global Fourier Modulation (GFM), a novel modulation technique that injects high-frequency information at each layer of the INR through Fourier-based reparameterization. This enables compact and accurate representation of multiple solution fields using low-dimensional latent vectors. Building upon GFM, we introduce PDEfuncta, a meta-learning framework designed to learn multi-modal solution fields and support generalization to new tasks. Through empirical studies on diverse scientific problems, we demonstrate that our method not only improves representational quality but also shows potential for forward and inverse inference tasks without the need for retraining.

PDEfuncta: Spectrally-Aware Neural Representation for PDE Solution Modeling
Minju Jo
Woojin Cho
Uvini Balasuriya Mudiyanselage'
Seungjun Lee
Noseong Park
Kookjin Lee
SSoroonne University, TelePIX, SSCAI, Arizona State University, Brookhaven National Laboratory, 'KAIST
minnju420gmm com, woojin. wowinnyyymmmmma..mm com, ubalasur@asu edu, slee19@bn1 gov, noseongkkaist. kookjin leeGasu. edu
Abstract
Scientific machine learning often involves representing complex solution fields that exhibit high-frequency features such as sharp transitions, fine-scale oscillations, and localized structures. While implicit neural representations (INRs) have shown promise for continuous function modeling, capturing such high-frequency behavior remains a challenge eesicngageeeeciallll when modeling multiple solution fields with a shared network. Prior work addressing spectral bias in INRs has primarily focused on single-instance settings, limiting scalability and generalization. In this work, we propose Global Fourier Modulation (GFM), a novel modulation technique that injects high-frequency information at each layer the INR through Fourier-basse reparameterization. This enables compact and accurate representation of multiple solution fields using low-dimensional latent vectors. Building upon GFM, we introduce PDEfuncta, meta-learning framework designed to learn multi-modal solution fields and support generalization to new tasks. Through empirical studies diverse scientific problems, demonstrate that method not only improves representational quality but also shows potential for forward and inverse inference tasks without the need for retraining.
Introduction
Solving partial differential equations (PDEs) is essential progress in many science and engineering applications, including fluid dynamics, seismic inversion, climate modeling, and mechanical design. Traditionally, these PDE problems are addressed using numerical methods (e.g.
finite-oolume (2). finite-element 3). which are reliable but relatively slow and computationally expensive for nigh-dimensional high-resolution data. Scientific Machine Learning (SciML) 4 5,677 8 offers a more efficient alternative by combining physics-basee models with data-driven approaches approximate solutions quickly and accurately. One critical challenge within SciML applications involves the efficient compression and functional representation of large-scale scientific datasets, which contain complex structures, sharp transitions, and fiih-ffreyuncy content ( 10] To handle such data is critical - not only for reducing storage and memory costs but also for enabling fast and flexible adaptation in downstream tasks such as operator learning, inverse modeling.
promising strategy to address these challenges is the use implicit neural representations (INRs), which represent data continuous functions by mapping coordinates X' (e.g. spatial, temporal
'Equal Contribution
CCorresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

positions) to physical quantities f(x). These methods inherently offer mesh-free and resolution- invariant representations, making them particularly suitable for complex scientific data. Nevertheless. standard INR architectures based on multilayer perceptrons (MLPs) suffer from a well-
limitation called spectral bias 11 12 13 where the neural network tends to learn low-frequency components more rapidly than high-frequency ones.
This problem has been rigorously formalized in prior theoretical work. In particular, according to Theorem in M.. MLPs with tanh activation, gradients associated with high-frequency components are smaller than those for low-frequency components.
Although various approaches, including SIREN [4], Fourier feature networks (FFN) [15], and
wavelet implicit neural representations (WIRE) [6]. have demonstrated improved performance, mitigating this spectral bias problem, by proposing advanced nonlinear activation functions, these methods typically focus training single INR for single data instance.
To resolve this weakness, we turn to the Functa paradigm [17], which functionalizes discrete data date-reprssnntinn each sample as continuous function and compresses en-
Low freq.
High freq.
tire dataset by modulating a single INR with low-dimensional latent vectors. Motivated by this success, we ask whether a similar strategy functionalize discretized PDE date-thereby collapsing massive simulation outputs into lightweight latents while restor- ing their functional character and use this reduced representation in performing down- stream tasks (e.g.. g., super-resolution, learning forward/inverse PDE operators). However, naïvely adopting the Functa paradigm the PDE domain is expected to bring novel challenges, namely, spectral bias and no established mechanism describing forward/inverse problems. See Figure for example; existing INR modulation techniques (e.g.. Shift, Scale, FiLM) struggle to capture high-frequency components of the PDE solution.
40
35-
(Ours)
PSNR
30
(1.1)
12.21
(3.3)
(4:41
15.5)
(6.6)
(7.7)
(0.81 (9.9) (10.10)
(al,
a2)
Figure 1: [Spectral bias problem] Experimental re- sults on Helmholtz equations
[
Spectral bias of INR with modulation methods. Existing modulation approaches 17 18 apply instance-specific modulations directly in the latent space, without explicit control over spectral characteristics. Consequently, these modulation methods inadequately represent PDE data with complex, high-frequency features. To effectively overcome spectral bias within modulated INR settings, we propose novel modulation method based on Fourier reparameterization, frequency- aware method that explicitly injects spectral priors into the latent space.
In this paper, we address these challenges by introducing spectrally balanced and resolution-invariant framework for learning scientific data. We make the following key contributions:
We propose Global Fourier Modulation (GFM), modulation technique that extends Fourier reparameterization to the modulated INR setting. Our GFM injects fixed Fourier bases into each layer of the shared INR backbone, enabling to learn high-frequency components that other modulations fail to capture.
We present PDEfuncta, reversible architecture that represents onputt solution field pairs using one shared latent modulation vector. This design supports bidirectional inference.
Through various experiments benchmark PDE datasets, we empirically demonstrate that our method achieves high reconstruction accuracy, generalizes across parameter ranges and resolutions, and outperforms existing modulation baselines.
Our approach also bridges the gap between NR-based compression and operator learning. While
22 23 are designed to map between function spaces, they are limited to one-way inference. In contrast, our PDEfuncta leverages the functional continuity and Fourier reparameterization based modulation deliver a compact, flexible, and bidirectional inference for learning function space mappings in the latent space.
classical neural operators [19 21

---

2 Related Works
Implicit Neural Representation INR methodologies aim receive coordinates input and map them to corresponding values. Two types of INR, namely Physics-informed Neural Network (PINN) (24] and Neural Radiance Fields (NeRF) (25), have the goal of solving PDE problems and representing 3D scenes, respectively. Since inference is possible through input coordinates, it is feasible represent at the desired resolution and express a single data point function. According to [5], Fourier feature mapping can maximize the performance of INR, showing equivalence to the shift-invariant kernel method from the perspective of NTK (26). On the other hand, as seen in [14 [6] 271, many studies have focused on proposing activation functions for INR to enhance both the expressiveness and complexity of the INR model.
Data to Funeta An INR is a network that continuously represents a signal or field in its domain. INR methods like DeepSDF (28) and NeRF model data as continuous functions. Recent work has explored treating data as functions to improve learning in function space. For example, Functa [17] shows that one can encode each data sample like image, as function (parameters of INR) and learn tasks over these representations. In this paradigm, each dataset sample is encoded as low-dimensional latent vector that modulates a shared INR network. Such functional representations offer substantial benefits, including resolution independence, improved generalization, and efficient data compression. Our work leverages these ideas by employing modulated INRs to encode PDE solution fields and enable direct learning of operators in continuous function spaces, significantly improving efficiency and flexibility in scientific machine learning applications.
Neural Operator Neural operator is one of the methods to approximate solutions to PDEs, aiming to map input and output function spaces. Fundamental research in this field, known the deep operator network (DecpONet) (29). is composed of branch network and a trunk network, each used to learn the PDE operator. In this process, DeepONet requires a fixed discretization of the input function. On the other hand, there are studies that learn operator by approximating kernel integral operations. In this domain, a promising study known as Fourier neural operator (FNO) (20 maps function spaces using convolution operation in the Fourier domain. Furthermore, investigations involve training models in the wavelet domain B0 BT and utilizing message passing in graph neural networks for operator learning [32 (3). These approaches are highly effective in learning PDE data and can perform computations rapidly during dynamic simulations. However, a drawback of these methods is the requirement for input and output data to have consistent sampling point. Subsequently, solutions such as the R4 (55,, which feature a GNN-based encoder-decoder structure, have been proposed to address problems beyond fixed rectangular domains. Additionally, there DINO [22] and CORAL (36) based on implicit neural representation, which enables learning and inference even in situations where the sampling ratios of input and output data differ.
3 Global Fourier Modulation for Mitigating the Spectral Bias Problem
INRs model data as continuous functions f(X'; 0) that map spatio-temporal coordinates X' to solution fields, offering cooluution-ivvaiian representations. However, they require separately trained set of parameters 0 = (W, b} for each data sample, which limits scalability. The Functa framework 17 37 addresses this scalability issue by introducing modulated INRs, where single shared INR network is used across all samples, and each instance is customized via a sample-specific latent code z. This latent vector is passed through a lightweight modulation network g(2; 2; T) to produce set of layer-wise modulation vectors {a* He where π = { bmod are the parameters of The latent vector z is learnable and initialized as zero vector, forming a set known as Functaset.
However, modulated INRs still inherit the spectral bias of MLPs, where low-frequenc, components leameed more rapidly than high-frequency ones. To mitigate this, we propose Global Fourier Modulation (GFM), which is a novel modulation strategy that explicitly injects frequency-awareness into the network. Its design and theoretical foundation are detailed next.
3.1 Spectral Bias and Fourier Reparameterization
INRs are well known for exhibiting spectral bias, the tendency of neural networks to approximate the low-frequency components of target function much earlier than the fregufnryyency components.
3

---

Global Fourier Modulation (GFM)
W 8 AW ).).
(Host( Pr. +0 High
2
"0"""""*" +8haa
g
+
(("" [/)=a"_
7(W" +
Latent modulation vector(s)
Low
Figure 2: Overall pipeline of GFM. The GFM framework injects fixed Fourier bases into each layer of the shared INR, enabling effective learning of both low- and high- fregh-freuency components.
Theorem 1 from [ provides a theoretical explanation for this phenomenon, showing that low- frequency components dominate the gradient dynamics initialization. As result, model struggle to capture fine-scale structures, which are essential in many physical systems and PDE- based data.
To mitigate this issue, prior works has proposed frequency awarre parameterizations that explicitly embed spectral priors into the model architecture [3) A representative approach reparameterize each weight matrix as W = S.4, where S E MRXD is trainable coefficient matrix and & E IR Dx M a fixed Fourier basis composed sinusoidal functions. This formulation restricts the function class to lie within predefined frequency subspace, making high-frequency modes more accessible during training. also improves the conditioning of the Neural Tangent Kernel (NTK), leading to a more balanced eigenvalue spectrum and better convergence complex signals.
Theorem (Theorem 2 in (8)) Let W E MMM denote the weight matrix of a hidden layer in an trainable coefficient
MLP. Suppose that it is reparameterized as W 5. 4, where S E RxDD matrix and & E IR DxM is fixed Fourier basis matrix. Then, for any two frequencies FL F2 > 0, any 2 and fixed index i, for m I...., M there must exist a set of basis matrices such that:
aL(Fi)
(F2)
(L((F)
(L(F2)
(LI(F)) (L(F2)
2 max!
3- c,
(1)
dsim
isc.
Ow,,1
@wi,1
Ow,,
Ow,,M
where 8tm and Wam denote elements of S and W, respectively.
Theorem formalizes the gradient dynamics under Fourier reparameterivation. It shows that the gradient ratio between high-freqeency reparameterization than standard weights, up to a small error c. This indicates that high-frequency structures receive relatively stronger gradient signals during training, improving the model's ability to learn fine-scale patterns.
f and low-frequency components greater under Fourier
Construction of the Fourier Basis. The fixed basis matrix oh RDxM is constructed by evaluating phase-shifted cosine functions over a uniform spatial grid {pm 1M in the range [-Tmax/"2 2,, Tmax/22 where Tmax is determined by the smallest frequeney used. Each row cor- responds to unique combination of frequency и and phase shift allowing spectral diversity across layers. Specifically, each basis row in the k-th layer is defined as:
(2) where is selected from mixture of low and high frequency components, and is phase shift uniformly sampled from [0, 2n.. This diverse basis configuration enables the network to better capture both smooth and fighffrequency features. Full construction details are provided in Appendix H
t [cos(w Pm + 4) }) Jm for d = d......
3.2 Global Fourier Modulation for Modulated INRs
mitigates spectral bias in standard INRs by pro- jecting weights into frequency-aware subspace, it is fundamentally restricted to single-instance settings. In contrast, modulated INRs aim to generalize across datasets by injecting instance-specific information through modulation. However, prior modulation techniques- such as Shift, Scale, and FiLMM-ppraal solely in the activation space and lack explicit frequency control. This limitation makes them inadequate for modeling signals with rich high-frequency content (see Figure
While Fourier reparameterization
effectively

---

Function space (A)
Lateot modulation vectoria)
Function space (L4)
1f90 160 (a(1))
m.)
R0:
7692/12A Rhin,,
7(P 80
E [0,3g]
f E (To, Tul
Figure 3: Proposed method: PDEfuncta. PDEfuncta leverages GFM to represent paired function spaces using a shared latent vector, supporting bidirectional inference between input and output fields. This architecture enables efficient compression and reversible mapping for scientific datasets.
To address this gap, propose GFM, novel modulation strategy that integrates reeqeencyawwarenee directly into the weight space. GFM extends Fourier reparamcterization to multi-sample settings by combining shared Fourier basis @ (cf. Section 3.1 with sample-specifie modulation vector a² = &. *.*... Here, #.W E Id modulates the spectral components of the weight matrix, while k. E I2d adjusts the bias term. At each layer k, the weight matrix computed as:
W S xh
Sh
(R + k.W 1
where R* E RR is shared learnable base matrix and 1 E IR-M is a vector of ones for broadcasting. This construction enables direct, sample-specific modulation in the frequency domain. To clarify how GFM differs from previous approaches, we first revisit the general update rule for modulated INRs:
h k+1 (v' ht bb;aa))) b*)));a*)))
(4)
where h E IRd is the hidden representation at layer ks y is an activation function and ((;; is modulation function. Existing modulation methods apply i in the activation space as follows:
Shift : (((( h²th + ;²);a" = (W* t + b*) + a
(5)
Scale : (((( n²t + a²) } (W* h² + b²) o
(6)
FiLM:
((W( A² a (W* 1tt + bt) o t. + t.b.
These approaches adjust values per instance but leave the weight matrices unchanged, limiting their capacity to modulate frequency content explicitly. In contrast, GFM redefines the transformation as:
GFM (Ours): : ((W* h + b²);; a Φ+) = (R* k.W 1 tb h + 8 + a.b.
(8)
While the primary effect arises from spectral modulation via #.W the additional bias term k.6 enhances representational flexibility and fine-grained control over output activations.
By conditioning weights on shared Fourier basis and modulating them in the frequency domain, GFM inherits the benefits of Fourier reparameterization- -uccii as improved NTK conditioning and stronger high-frequency response (Theorem [ while scaling to dataset-level representations. This design enables traguency-targeted, sample-specifer adaptation in parameter-efficie efficiee-efficien manner.
PDEfuncta: From discrete to continuous solution representations
Applying our proposed GFM to the Functa framework effectively overcomes the spectral bias com- pared to conventional modulation strategies (see Table and Figure/6lnn Section 5). As demonstrated theoretically Section 3 GFM significantly improves the compression and accurate reconstruction of scientific data with high-frequency components. While effectively modeling individual datasets valuable, scientific applications frequently involve data pairs defiedd on two distinct yet related function spaces (A, 24). A representative example is full waveform inversion (FWI) 39 10), where paired data samples exist as seismic velocity fields and corresponding seismic waveforms (cf. Ap- pendix D). In this section, leveraging Functa's 's functionalization and compression capabilities, we introduce PDFfuncta, a novel framework designed to jointly represent these paired function spaces through a shared latent modulation vector
5

---

4.1 Overall Architecture
Our proposed method, as depicted in Figure 3 represents functions the function space A and function space U through respective neural networks fa(;; ;2; 0a) and fu(X; These two INR models receive spatio-temporal coordinates ho [x, as input and have the following structure:
h²- = >(Wo, 110 + +b), h h++ "W#*(W** k htt + +b"); bba tt), ==1,
(9)
0) = Wh-1, hut1 + betl
Here, 0a = (W 01 and 0. (Wi b are the weight and bias of the INRs and fu. respectively, which common model parameters used for samples in the dataset. In contrast, a* represents the modulation vector for the k-th layer, serving as model parameters that vary depending on the data follows:
(o
ma); ga(2) ma) Wrriod + m
(10)
{ow)k=1 = gu (u(2;; mu); 9u mm) W 2 + A.mod
The latent modulation vector 2 consists of parameters learned through auto auto-decoding 28 from data sample, and one generated for each data sample. The weights and bias parameters constituting a linear layers, Ta [Wrmod [mmod] Tu [W Mmood], receive the 2 as input and output modulation vectors {a k 1 km j.(i.e Eq (T0), these two modulation vectors are applied to each layer of the INR networks fa and fu. enabling the neural network represent various data samples. We extend Fourier reparameterization to support multiple data instances via GFM.
/h (a) = 5hh o² = (R + a k.W
11).00 Φ², (a( = S th (R², + a. R.W
)1)- Th (11)
where (R R is base coefficient matrix (shared across all instances) and & a. k.K.W. W Jk=1 are a modulation term derived from a shared latent modulation vector z. In the simplest instantiation, the GFM method adds {a a RR" each row of (R. Rakk= RR MxD (or an equivalent broadcasting) so that { a adjust the coefficients associated with the Fourier basis vectors.
4.2 How to train
Our training method requires steps to learn the functaset that matches to train/testset during both the training and testing phases. We nested loop structure in the training phase (cf. Algorithm I following functa framework: an inner loop that focuses on construction loop that updates the global model parameters. Both loops the same loss function IL, which is the sum of reconstruction errors from fa and Su as follows:
functaset, and an outer
L = (Lx; (a", (a('; ; 0a 9u (2t1 + Lx, (u', fu ; Pn gu (2(====m)))))
(12)
While both loops utilize the same loss function Lx (t) = (x - 2 the inner loop updates parameters for each individual sample, whereas the outer loop updates the parameters based on the batch loss Σi=1 Lj. In the inner loop, sample K samples from each batch and update modulations of the selected samples. This bilevel approach mimics a specialized form of Model- Agnostic Meta-Leaat Letannea (MAML) [40] that focuses on learning only subset of weights, facilitating efficient adaptation and learning across diverse datasets [1]
8
Experiments
In this section, we empirically evaluate the proposed GFM and PDEfuncta frameworks. Our ex- periments focus on four main goals: (i) validating GFM's ability to compress and reconstruct PDE solution fields under single-INR modulation; (ii) evaluating whether the learned latent space enables generalization to unseen parameter configurations; (iii) demonstrating PDEfuncta's 's ability to perform bidirectional inference across paired function spaces; and (iv) comparing PDEfuncta against neural operator baselines on complex geometry benchmarks.

---

Table 1: Comparison with existing modulation methods (PSNR\/MSEL)
Convection
Helmholtz #1
Helmholtz #2
Navier-Stokes
Muramoto-Sivashing
Modulation
PSNR
MSF
PSNR
MSE
PSNR
MSF
PSNR
MSE
PSNR
MSE
Shift 17
13.225
0.3849
25.467
0.0341
18.249
0.1426
25.767
0.0092
16.962
7.9715
Scale 17
27.410
0.0067
25.669
0.0472
18.183
0.1355
31.469
0.0024
20.395
3.8306
FiL.M
22
27.330
0.0125
33.611
0.0133
22.771
0.0762
31.904
0.0022
22.297
2.3309
GFM
42.474
0.0004
39.139
0.0006
29.033
0.0056
34.754
0.0011
29.827
0.4198
5.1
Experimental Setup
For each dataset, we report reconstruction performance using Peak Signal-to-Noise Ratio (PSNR) and Mean Squared Error (MSE). For neural operator tasks, we use the T,2-norm of prediction error, which is the standard metric in literature. All modulation based experiments use 20 20-dimensional latent vector z to modulate the shared INR. Further details are provided Appendix E
Datasets.
We evaluate the performance of our proposed GFM using four scientific datasets: the con- vection equation (Eq. (33). Helmholtz equation (Eq. (14), ), Navier-Stokes (NS) equation (Eq. (17). and Settamoto-Sivsshiine
(KS) equation (Eq. (66). To assess capability of PDEfuncta for bidi- rectional inference, we use the OpenFWI dataset [T]] which contains pairs of samples across two distinct function spaces. Lastly, we explore performance of our proposed method in challenging
reural-operator scenario, using the Airfoil (Eq. (2T) and Pipe flow (Eq. (22) datasets. Detailed descriptions of each dataset are provided in Appendix D
Baselines. We compare our GFM against three representative modulation methods: Shift, Scale, and FiLM. Additionally, we provide comparisons with Spatial Functa (37) in Appendix [ To evaluate
the neural operator capability of PDEfuncta, we consider established baselines including FNO 20].
UNet 42), Geo-FNO 35), FFNO 21, CORAL (36 and MARBLE 23
5.2 Compression and Reconstruction with Single-INR modulation
We begin with the standard single-INR setting, where a single INR is modulated by a sample-specific latent codes to com- press and reconstruct individual continuous solution fields. Ex- periments are conducted on four PDE datasets known for high-
1.0
Shift
FILM
0.8
Scale
0.6
0.4
frequency details: 1D convection equations (3 {1, ........ 50)).
2D Helmholtz equations (#1: 41, a2 E fl..... 5): #2:
0.0
a1 02 E {4.... 10)), 2D incompressible NS, and KS equa- tions. Detailed experimental setup is listed in Appendix E
1000
2000
3000
4000
5000
Epoch
Figure 4: [Kuramoto-Sivashinkee Sivashinsky]
As summarized in Table I GFM achieves the highest PSNR and lowest MSE across all datasets. Notably, it improves PSNR by up to 15dB over the strongest baseline, FiLM, the convection equation, and by 3-7dB on the others. Figure 4 ssowws the training loss curves up to 5000 epochs on the KS equation for four modulation methods: Shift, Scale, FiLM, and our proposed GFM. GFM shows the fastest and most stable convergence, reaching the lowest final MSE among all baselines. This also highlights that GFM can successfully compress and reconstruct high-frequency spatiotemporal dynamics using only a 20-dimensional latent code. We also present qualitative reconstruction results in Figure 6) The top row shows KS error maps, and the bottom row shows Helmholtz reconstructions. GFM shows the most visually accurate outputs with minimal error, while others show noticeable distortion.
Training loss (MSE) over epochs.
5.2.1 Evaluating Functional Representation in Latent Modulation Space
To evaluate whether the learned latent modulation space captures meaningful functional representations, we design two generalization experiments. These experiments test the model's 's ability to interpolate beyond the discrete set of PDE coefficients seen during training. We report results on the convection equation, where we test on unseen coef- ficients 8r E (1.5, ...,..., 49.5) that are excluded from the training set (3 {1,2..... 50])). In both cases, we aim to recover the full solution for 3 using
Table 2: Results on unseen coefficients
Setting
Setting 2
Method
PSNR
MSE
PSNR
MSE
Shift
8.427
0.5305
13.307
0.4533
Scale
9.125
0.4186
9.197
0.5421
FILM
10.613
0.3954
11.927
0.3268
GFM
32.586
0.0017
30.830
0.0033
7

---

(a) Ground Truth
(b) Shift
(c) Scale
(d) FiLM
(e) GFM
4.5
430
-33
(1) Ground Truth
(g) Shift
(h) Scale
(i) FiLM
(i) GFM
Figure 6: Top: Error maps for the Saaaott--ivvshhnsky equation. Bottom: Helmholtz equation
(a1 6.0, 02 9.0) reconstructions. GFM achieves the most accurate reconstructions.
only its inferred latent code 2, without updating the shared INR. All results are obtained using the pre-trained model from Section 5.2 Additional details provided in Appendix F22
Setting 1: Latent Interpolation. We assume no access to the target solution for the unseen coefficient 3 Its latent code 2g- is estimated via cubic interpolation between the latent codes of nearby training coefficients. This setting evaluates whether the latent space encodes smooth, functionally meaningful transitions between parammeer-conddtioned solutions.
Setting Latent Fitting from Partial Observation. We assume partial access the target solution over & E J0, 0.5] and optimize only the latent code Zạ-- to fit the observed segment while freezing the shared INR. We evaluate reconstruction performance on both the observed segment and the unobserved future i E [0.5, 1J.
Our experimental results, which are summarized in Table and Figure empirically demonstrate that, even when trained only on finite discrete set of PDE coefficients, the proposed method can effectively generalize unseen continuous parameter values via latent space interpolation. This indicates that the latent modulation vectors capture a smooth, functional mapping aligned with the underlying parametric dependence the PDE family. In Setting 2, where partial observations the solution are available, our method successfully recovers the full spatiotem- poral structure of the solution by optimizing only the latent code while keeping the INR fixed. The model not only fits the observed interval accurately but also extrapolates reliably to the unobserved region, highlighting the adaptability and efficiency of the latent modulation space. Further, this property is essential for scientific applications where one frequently encounters novel or intermedi- ate parameter regimes that including settings where partial observations are available for 3 are discussed in Appendix F22
M M,5
(a) GTO = 24.5)
(b) Setting
(c) Setting
86ss Unnce
(d) GT(3 = 19.5) (e) Setting
(l) Setting 2
Figure [Convection] Reconstruction results for
unseen / = 24.5, 49.5 on Setting and 2.
explicitly present in the training Additional experiments,
5.3 Bidirectional Inference
A core contribution of the PDEfuncta framework is its ability to perform bidirectional functional inference between two function spaces A and U4 via joint latent representation. This capability enables unified modeling of forward and inverse mappings between parametric PDE function spaces a setting frequently encountered in scientific applications such as aerodynamic design. this section, we evaluate this bidirectional capability in two key aspects: (i) reconstruction fidelity seen paired instances using diverse modulation strategies and (ii) generalization to unseen PDE instances with complex geometries in neural operator settings.

---

Table 3: Comparison with existing modulation methods (PSNRT/MSEL)
FWI (A)
FWI (t()
Navier Stokes ((A)
Navier Stokes (2))
Modulation
PSNR
PSNR
MSF
PSNR
MSF
PSNR
MSE
Shift
25.879
0.0102
24.995
0.0126
28.134
0.0155
26.306
0.0219
Scale 17
25.256
0.0117
29.978
0.0039
38.784
0.0013
38.747
0,0013
FiL.M 22
25.292
0.0115
30.026
0.0035
42.649
0.0005
43.105
0.0004
GFM
28.757
0.0054
32.761
0.0021
43.166
0.0004
43.792
0.0003
(a) GT(Airfoil)
(b) Pred:
(c) Pred: g
(d) GT(Pipe)
(e) Pred: g
(f) Pred: Gt
Figure 7: PDEfuncta reconstructs both forward G A U and inverse G : U - A mappings between function spaces for unseen airfoil and pipe flow samples.
5.3.1 Reconstruction Fidelity on Seen Samples: Modulation Comparison
We evaluate PDEfuncta's ability to compress and reconstruct paired data from seen samples using shared 30-dimensional latent vector. Experiments are conducted on the FWI and 2D incompressible Navier-Stokes datasets, covering distinct scientific domains. In FWI, A denotes seismic data and UI is the corresponding velocity map. For Navier-Stokes, A consists of velocity fields at early time steps (t E [o, 14), and L/ later timesteps (t E 29). This setup enables bidirectional inference between physical states over time or domains. We compare four modulation strategies: Shift, Scale, FiLM, and our GFM. As shown Table GFM achieves
best reconstruction performance across both directions. On FWI, it reconstructs velocity maps from seismic data with PSNR of 32.76 dB. Navier-Stokes, shows the lowest MSE of 0.0003 for velocity reconstruction at E [15, 29]. These results highlight PDEfuncta's s ability to learn accurate bidirectional mappings between paired fields. Additional experimental results and analyses are provided in the Appendix G
5.3.2 Generalization to Unseen Instances: Neural Operator Comparison
Finally, we compare PDEfuncta with neural operator baselines on two complex geometry benchmark datasets extending it to neural operator, i.e., Euler-NACA (Airfoil) and Pipe. The EuCA dataset predicts airflow (lf) over held out airfoil shape (AA), whereas the pipe benchmark dataset considers incompressible flow (Lf) in unseen pipe cross sections(A). In both cases, we aim to learn a pair of bidirectional solution operators between function spaces: the forward mapping g A + M and the inverse mapping g 1 - A.
Table 4: Test -2-error (J)
Method
Eukr-NACA
Pipe
0.0385
0.0153
UNet
0.0505
0.0298
Goo-ENO
0.0158
0.0066
0.0062
0.0073
CORAL
0.0059
0.0120
MARBLE
0.0058
0.0103
PDEfuncta
0.0051
0.0081
We evaluate both G and G on unseen samples. Table reports L2 errors for forward mapping G. and PDEfuncta achieves the lowest error on the Airfoil task and competitive performance on the Pipe dataset. As shown in Figure PDEfuncta demonstrates strong inference performance in both forward and inverse mappings. These results demonstrate that the proposed model generalizes well across domains and supports consistent bidirectional inference between geometry and solution spaces.
6 Conclusion
We presented GFM, simple prraamttteriztiiinn that handles spectral bias in modulated INRs by conditioning every layer through fixed Fourier basis. Theory and experiments show that this approach can learn high-frequency components well as low-frequee frwffregenccy components, and provides reliable compression scientific machine learning data. Building on GFM, we introduced PDEfuncta, a bidirectional novel architecture that encodes paired fields with single latent vector. Taken together, GFM and PDEfuncta offer lightweight, resolution free characteristic, unified frameworks for functionalized data and spectrally balanced modeling of PDE systems.

---

7 Acknowledgement
K. Lee acknowledges support from the U.S. National Science Foundation under grant IIS 2338909. This work also supported by Samsung Research Funding & Incubation Center of Samsung Electronics under Project Number SRFC-IT2402-08, and by Samsung Electronics Co., Ltd. (No. G01240136, KAIST Semiconductor Research Fund (2nd)).
References
[1] Randall J LeVeque. Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems. SIAM, 2007.
[2] Randall J Veque. Finite volume methods for hyperbolic problems, volume 31. Cambridge university press, 2002.
[3] Junuthula Narasimha Reddy. An introduction the finite element method. New York, 27(14), 1993.
[4] Nathan Baker, Frank Alexander, Timo Bremer, Aric Hagberg, Yannis Kevrekidis, Habib Najm, Manish Parashar, Abani Patra, James Sethian, Stefan Wild, al. Workshop report on basic research needs for scientific machine learning: Core technologies for artificial intelligence. Technical report, USDOE Office of Science (SC), Washington, DC (United States), 2019.
[5] George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422 -4022 2021.
[6] Kamyar Azizzadeneshelii Nikola Kovachki, Zongyi Li, Miguel Liu-Schiaffini, Jean Kossaifi, and Anima Anandkumar. Neural operators for accelerating scientific simulations and design. Nature Reviews Physics, ((5:330-3228, 2024.
[7] Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. Scientific machine learning through physics-infonme inffrredd neural networks: Where we are and what's ' next. Journal of Scientific Computing, 92(3):88, 2022.
[8] Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes, Journal of Machine Learning Research, 24(89):1-97, 2023.
[9] Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Daniel MacKinlay, Francesco Alesiani, Dirk Pflüger, and Mathias Niepert. Pdebench: An extensive benchmark for scientific machine learning. Advances in Neural Information Processing Systems, 35:1 59961611,,, 2022.
[10] Chengyuan Deng. Shihang Feng, Hanchen Wang, Xitong Zhang, Peng Jin, Yinan Feng. Qili Zeng, Yinpeng Chen, and Youzuo Lin. Openfwi: Large-scale multi-structural benchmark datasets for full waveform inversion. Advances in Neural Information Processing Systems, 35:6007-6020, 2022.
[11] Zhiqin John Xu. Understanding training and generalization deep learning by fourier analysis. arXiv preprint arXiv: 1800v 04099, 2018.
[12] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks, International conference on machine learning, pages 5301-5310, PMLR, 2019.
[13] Zhi-Qin John Xu, Yaoyu Zhang, and Tao Luo. Overview frequency principle/spectral bias in deep learning. Communications on Applied Mathematics and Computation, pages 1-38, 2024.
[14] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances neural information processing systems, 33:7462-7473, 2020.
10

---

[15] Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Keil., Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems, 33:7537-7547, 2020.
[16] Vishwanath Saragadam, Daniel LeJeune, Jasper Tan, Guha Balakrishnan, Ashok Veeraraghavan, and Richard G Baraniuk. Wire: Wavelet implicit neural representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18507-18516, 2023.
[17] Emilien Dupont, Hyunjik Kim, SM Eslami, Danilo Rezende, and Dan Rosenbaum. From data to functa: Your data point a function and you can treat like one. arXiv preprint 2r01v:2201.12000 2022.
[18] Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018
[19] Lu Lu, Pengzhan Jin, and George Em Karniadakis. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXív: 1910.03/93 2019.
[20] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differen- tial equations. arXiv preprint arXiv:2010.08895, 2020.
[21] Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. In The Eleventh International Conference on Learning Representations, 2023.
[22] Yuan Yin, Matthieu Kirchmeyer, Jven-Yves Franceschi, Alain Rakotomamonjy, and Patrick Gallinari. Continuous pde dynamics forecasting with implicit neural representations. arXiv preprint arXiv:2209. 148555 2022.
[23] Honghui Wang, Shiji Song, and Gao Huang. Gridmix: Exploring spatial modulation for neural fields in pde modeling, In The Thirteenth International Conference on Learning Representations, 2025.
[24] Maziar Raissi, Paris Perdikaris, and George E. Karniadakis. Physics-informie neural networks: deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
[25] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan Barron, Ravi Ramamoor- thi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1)999-106, 2021.
[26] Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018.
[27] Liu Ziyin, Tilman Hartwig, and Masahito Ueda. Neural networks fail to learn periodic functions and how fix it. Advances in Neural Information Processing Systems, 33:1583-15000 2020.
[28] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 165-1744 2019.
[29] Lu Lu, Pengzhan Jin, Guofei Pang. Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218-222 2021.
[30] Gaurav Gupta, Xiongye Xiao, and Paul Bogdan. Multiwavelet-base operator learning for differential equations, Advances in neural information processing systems, 34:24048-24066 2021.

---

[31] Tapas Tripura and Souvik Chakraborty. Wavelet neural operator: neural operator for parametric partial differential equations. arXiv preprint arXiv:2205.022191 2022.
[32] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arX1v:2003.034855 2020.
[33] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W Battaglia. Learning mesh-based simulation with graph networks. arXiv preprint arXiv:2010.03409, 2020.
[34] Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. Journal of Machine Learning Research, 24 2023.
[35] Zongyi Li, Nikola Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Otta, Moham- mad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Arizzadeneshelii et al. Geometry-nformee neural operator for aare--ccae 3d pdes. Advances in Neural Information Processing Systems, 36, 2024.
[36] Louis Serrano, Lise Le Boudec, Armand Kassai Koupai, Thomas X Wang, Yuan Yin, Jean-Noöl Vittaut, and Patrick Gallinari. Operator learning with neural fields: Tackling pdes on general geometries. Advances in Neural Information Processing Systems, 36:70581-7061 2023.
[37] Matthias Bauer, Emilien Dupont, Andy Brock, Dan Rosenbaum, Jonathan Richard Schwarz, and Hyunjik Kim. Spatial functa: Scaling functa to imagenet classification and generation. arXiv preprint arXiv:2302.033 000.0333., 2023.
[38] Kexuan Shi, Xingyu Zhou, and Shuhang Gu. Improved implicit neural representation with fourier reparameterized training. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 15985-25994 2024.
[39] Jean Virieux and Stéphane Operto. An overview of full-waveform inversion in exploration geophysics. Geophysícs, 746::WCCI-WCC2, 2009.
[40] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. In International conference on machine learning, pages 1126-1135 PMLR, 2017.
[41] Luisa M Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast context adaptation via meta-learning. 2019.
[42] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. intervention-MICCA. 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234-241. Springer, 2015.
Medical image computing and computer-assisted
[43] Ricardo Buitrago, Tanya Marwah, Albert Gu, and Andrej Risteski. On the benefits of memory for modeling time-dependent PDEs. In The Thirteenth International Conference on Learning Representations, 2025.
[44] Seungjun Lee and Taeil Oh. Inducing point operator transformer: A flexible and scalable architecture for solving pdes. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 153-161, 2024.
12

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificcton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: All claims in the abstract and introduction clearly reflect the paper's ' contribu- tions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
13

---

Justification:
The paper discusses potential limitations in the Appendix B
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof'
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
Theoretical results are stated clearly with underlying assumptions in Section 3
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Detailed information, including dataset descriptions and training/testing proce- dures, is fully explained to enable reproducibility (See Appendix D)
Guidelines:
14

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution a new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Data and code used in experiments are publicly available with explicit instruc- tions included in supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nis.cc/public//uust http://///giiieee/eeeuuubiiiiiii
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
15

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experimental setup and details, including data splits, hyperparameter, training algorithm are provided in main paper and Appendix [
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Error bars provided standard deviations multiple runs, as detailed in Appendix G
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Experimental setup and computer resources details given in Appendix H
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification:
All research activities conform to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Both positive and negative societal impacts are discussed in Appendix
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
17

---

Justification:
The paper does not release high-risk data or models
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-us should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Detailed license and data information provided in Appendix
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, haperswithcode com/ datasets has curated licenses for some datascts. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification:
All new assets are provided in the supplementary material.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/moodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
18

---

Answer:
[NA]
Justification:
This work does not involve human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: No large language models were used as a core component.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non ntn-standard components.
Please refer to LLM policy (https://neurr) httpa/pppp..e//oooeeeenres////// for what should or should not be described.
19

---

A
Symbol Definitions
Table summarizes the key symbols and notation used throughout the paper for clarity and reference.
Table 5: Definitions symbols.
Symbol
Description
Input and function space variables
X
Input coordinate (spatial and/or temporal position).
I,
t
Spatial and temporal components of the input coordinate X.
(x(
INR modeling function space AA (with parameters 0.).
(x)
INR modeling function space (with parameters 0.).
A,U
Input and output function spaces
G.G
Function space mapping A U and U + A
Neural network architecture parameters
Set of network parameters (weights and biases of INR).
Da On
Network parameters fa and fu, respectively. Number of hidden layers in the INR.
L
he
Initial input to the network.
ba
Hidden feature vector layer ls (ho is input, hC+1 is final hidden layer output).
w*
Weight matrix of layer in the network.
6*
Bias vector of layer & the network.
Nonlinear activation function applied hidden layers.
Dimension each hidden layer (length Ay by).
Fourier reparameterization and spectral terms
S
Trainahle coefficient matrix for layer & under Fourier reparameterization. Fixed Fourier basis for layer
4%
Pm
mn-th point in uniform spatial grid used to construct 4x.
Tmax
Maximum coordinate value for the Fourier basis sampling interval.
(wiows Whigh
Frequency component used in constructing the Fourier basis.
Phase shift applied in the Fourier basis construction.
D
Number of Fourier basis functions per layer.
Number of sample points per basis function.
Fi, F2
High and low reference frequencies (Fi > F3 > 0) used in Theorem
Small positive constant (error tolerance in Theorem
Modulation and training variables
R
Shared base coefficient matrix for layer k. Modulation vector for layer k Components as modulate weight and bias at layer respectively. Modulation function applied at layer k
a
R.W. k.b (a)())
Latent modulation vector (sample-specifi)
9o (z; To)
Mapping function space A: generates (at) k= from latent z.
gu (23 mu)
Mapping for function space Lt: generates [a from latent Learnable parameters of the modulation networks Ja and gu.
Tas Wu.
Minner; Touter
Learning rates inner/outer loop Training loss function.
Speeciic parameters (experiment settings)
3
PDE coefficient in the ID convection equation. PDE coefficients in the 2D Helmholtz equations.
01,
20

---

B Limitation and Broader Impact
Our proposed GFM method allows explicit control over the ratio of high-frequence and low-frequency components in the Fourier basis during weight construction. While optimal performance may require careful hyperparameter tuning to select the appropriate frequency ratio, this flexibility enables our framework to be adapted to wide range of scientific problems and data characteristics. Additionally, our proposed methods, GFM and PDEfuncta, enable efficient compression and representation of multiple PDE solution datasets, offering significant advantages in terms of memory efficiency. These methods can facilitate effective storage and transmission of scientific data, potentially making large- scale modeling and simulation more accessible across various domains.
C Comparison with Existing Neural PDE Solvers
PDEfuncta demonstrates clear advantages over traditional Physics-Informed Neural Networks (PINNs) and Neural Operators. (1) PINNs, while effective in integrating physical laws directly into training, often struggle with scalability and generalization to complex, eigh-frequency PDE solutions. (2) Neural Operators, including methods like FNO and DeepONet, address scalability typically require consistent discretizations and are limited to one-way function space mappings. PDEfuncta overcomes these challenges by leveraging Global Fourier Modulation, providing a com- representation capable of accurate forward and inverse mappings
pact, efficient, and spectrally-awerr without retraining. Its shared latent modulation vector further enhances generalization across varied parameter spaces, demonstrating clear advantages in efficiency, generalization, and bidirectional inference capabilities compared to both PINN and Neural Operator frameworks.
D Dataset Description
We provide detailed descriptions of all benchmark PDE datasets used in the main paper, including the governing equations, boundary/initial conditions, parameter ranges, and scientific context for each.
D.1
Convection Equation
The convection dataset is based on the one-dimensional convection equation with a controllable advection speed parameter 3. The PDE governs the evolution of field u(x, , () as:
Ou at
Du 3 dr
0, E E [0,
(13)
where 3 the convection coefficient. In our experiments, ,3 varies within range (e.g. 3 E [1, 50) to represent solution fields of different characteristic speeds. We impose periodic boundary conditions on the spatial domain Q. An initial condition u(x, 0) is 1 + sin(z)). It is a fundamental model for wave propagation and convective transport.
D.2 Helmholtz Equation
The Helmholtz dataset involves a steady-state wave equation in two spatial dimensions. We generate solutions of the form u(x, y) by choosing forcing functions that yield analytic solutions. The PDE given by:
Pulz,y) Or2
Pu(z, v)
+
+ k2 uu,( v) q(z,)) = o,
Oy²
(14)
(x (-(agTt)2 (a2T)² ks2 sin (ag πτ) sin(azty),
u(x,y) =² sin(a Tw) sin(agTy),
(15)
where the wavenumber and q(x,,) 1/) is a source term. The coefficients ai and a2 are PDE
coefficients that control the number of oscillations of the solution in the and y directions, respectively. In our dataset, we consider two difficulty settings: Helmholtz #1 with (a1, E (1....,55 5) (limited frequencies), and Helmholtz #2 with {a), (2) E {1....,10) 103.
21

---

D.3
Kuramoto-Sivashinsky Equation
The dieeeiinnsionll Kuramoto Sivashiisk equation 4 is prototypical model for spatio-temporal
chaos and pattern formation. Following the experimental setup in 43 we use the data without normalization to preserve natural scale.
+ udx + Usz + VUlZER
0,
(f,x) E [0. 7) x [0, L.
20
(16)
u(0, z) =0(2), uo
Σ
A.sin((2 L) + dz),
==0
where u(z, is the evolving scalar field, and 0s viscosity (or hyperviscosity) parameter. In our simulations, we take periodic boundary conditions on the domain [0. L] (so that derivatives in are periodic), which is natural given that initial conditions are composed of Fourier modes.
D.4 Navier-Stokes Equation
We use the 2D Navier-stokee equation data employed in the neural operator related studies 220 21 44 This equation represents the dynamics of incompressible fluid within a rectangular domain and expressed as follows:
Ow(at t)
uu( + v
x E (0, ()) 1 (0,7)
(17)
w(x,t) 1) V t(x,t),
x E (0, 1)2; (0,7)
(18)
V u(xx O,
* (0, 1)2 t E (0,7)
(19)
where is the velocity field and w is the vorticity. the viscosity and forcing term. The initial condition wx((:) is generated according wh p where μ ~N(0, 73/2 4.7)
We consider the spatial domain x E 1)² with the following boundary condition:
[x1 x2) = 0. (sin (sin x2)) + cos(2m(1) + +2)))
(20)
D.5 Euler's Equation (Airfoil)
In our study, we employ the Euler's equation (airflow), which is pivotal component of the discussed benchmark data in 22]. The governing equation as follows:
Ops + V (ppv) = O, Öppv Øt
+( =0 aF
Ōt + V (EE+ VVV 0, (21)
where Pf. p. v and E are the fluid density, the pressure, the velocity vector and the total energy respectively. The viscous effect is neglected. The far-field boundary condition is specified as Pro 1, Pso = 1.0, Mx = 0.8, АoA = 0, where Mo represents the Mach number and АоA denotes the angle of attack. In our experimentation, use total of 1,000 training data samples and 200 test data samples. The computational grid utilized is the C-grid mesh with about 200 x 50 quadrilateral elements.
D.6 Pipe
The pipe dataset (35 is derived from simulations of viscous incompressible flow in a pipe, governed by the Stoker-Stokes equations for velocity and pressure.
ov 0
+ (v
V)v = - - Vp
(22)
V V
where is the velocity field, p is the pressure, and is the kinematic viscosity. The domain is cylindrical pipe, long tube with circular cross section, though our dataset is represented on a mesh (in the format of a 129 x 129 grid for cross-sectional plane, indicated by the input size).
22

---

D.7 Seismic Wave Equation
The OpenFWI dataset [T0] is collection of seismic forward modeling simulations used for full- waveform inversion research. Each data sample consists of a subsurface velocity model (a spatial map of wave propagation speeds) and simulated seismic wavefields recorded from sources propagating through that medium.
1
00p
V2p
(23)
22 012
where p(a,, 2, f) is the pressure wavefield (or partiele displacement) at spatial location (ar, 2) and time t, and v(x, 2) the velocity model (spatially-varying wave speed). The term s(2, 2, t) represents the seismic source (for example, Ricker wavelet point source on or near the surface). In our dataset, ? a 2D slice the Earth's subsurface (with x horizontal and z depth).
E Detailed experimental setup
This section provides the detailed experimental setup, including hyperparameter choices and compu- tational cost measurements. Experiments are conducted on a system running UBUNTU 18.04 LTS, PYTHON 3.9.7, PYTORCH 1.13.0, CUDA 11.6, 19 CPU, and NVIDIA RTX A5000.
E.1 Hyperparameters
For all experiments, we fixed the backbone INR to SIREN in order to isolate and compare the effects of different modulation methods. Across all models and experiments, set Tinner = 0.01 and nouterr 0.0001. All networks use a total of 5 layers, each with hidden dimension M = 256. The modulation mapping is implemented latent code 2 is fixed to 20 for all settings. For GFM, additional hyperparameter settings are required construct the Fourier basis (ef. Appendix H. Specifically, the number of phase shifts (Mphase) is fixed to 32 for all datasets, while the numbers of high-frequency and low-frequency components (Mhigh, Thw set as follows: 128 and 32 for Convection and FWI, 128 and 128 for Helmholtz, 64 and 16 for Parramoto-Sivasinne
two-layer MLP with hidden dimension 512, and the dimension of the
and 8 and 128 for Navier-Stoker The batch size and the number of training epochs for each dataset are as follows: batch size 32 and epoch 1,000 for Convection, batch size 16 and epoch 5,000 for Helmholtz, bach size 32 and epoch 5,000 for Kuramoto- -iinntto-ivassissky batch size 10 and epoch 1,000 for Navier-Stokes, and batch size 16 and epoch 10,000 for FWI.
E.2 Computational cost
To assess the computational requirements of different modulation methods, we report peak GPU memory usage and average training time per epoch for each model, measured on single NVIDIA A5000 GPU, As representative case, we provide results for the Helmholtz42 setting under the single-INR modula-
Table 6: Computational cost
Method
Memory (GB)
Time (sec)
Shift
8.15
0.20
Scale
12.15
0.24
FiLM
12.15
0.29
tion configuration (cf. Section 5.2 and Appendix[], Results are summarized in T GFM achieves a strong trade-off between model complexity and com-
SpatialFuncta
8.83
0.59
GFM (Ours)
8.28
0.27
putational efficiency, with memory consumption and training speed close to Shift, yet substantially better than Scale, FiLM, and SpatialFuncta. Notably, SpatialFuncta exhibits much higher training time per epoch (0.59s), which due to the extra computational cost of applying modulation every spatial location, rather than globally, despite moderate memory usage. Other experimental settings showed similar trends both memory and runtime.
23

---

More Experimental Results on Experiments with Single-INR modulation
This section provides additional analyses and experimental results to supplement Section 5.2 We present training loss curves for every dataset, detailed evaluation of the latent modulation space through interpolation analyses, and statistical comparisons across all benchmarks.
F1
Training loss curves
Figure presents the training loss (MSE) curves for Shift, Scale, FiLM, and GFM across all PDE benchmarks considered in Section 5.2 As discussed in Section GFM consistently achieves faster and more stable convergence compared to other modulation strategies. This advantage is especially notable for challenging cases such as Helmholtz#2 and Sarramooo-Sivashngg exhibit slow unstable training. These results further demonstrate the effectiveness of GFM in robust optimization and in mitigating the spectral bias typically observed in modulated INRs.
where baselines
1.4
Shift
FILM
0.25
Shift
RIM
0.25
Shift
FLM
1.2
0.20
0.20
1.0
00.15
0.0.8
0.6
0.10
0.10
0.4
0.05
0.05
0.00
0
200
400
600
800
1000
1000
2000
3000
4000 5000
1000
2000
4000 5000 Epoch
3000
Epoch
Epach
(a) Convection
(b) Helmholtz#1
(c) Helmholiz42
1.0
FiLM
Scale
GFM
Scale
GFM
0.6
0.6
0.4
0.0
0.0
5000
Epoch
Epoch
(d) Navier-Stokes
(c) Kuramoto-Sivashine
Figure Training loss (MSE) curves for Shift, Scale, FiLM, and GFM on (a) Convection, (b) Helmholtz#1, (c) Helmholtz42, (d) Navier-Stokes, and (e) Kuramoto-Sivashinst GFM consistently achieves faster and more stable convergence across benchmarks.
F.2 Additional Analysis of Latent Modulation Space
To further analyze the properties of the Iearned latent modulation space, we present additional qualitative and quantitative results for the generalization experiments described in Section 1.2.1 Specifically, we visualize the dynamics of the interpolated latents for the convection equation and compare interpolation results across different modulation strategies.
Figure shows the cubic interpolation trajectories of the latent code 2 for the convection equation. Each subplot corresponds to one dimension of the 20-dimensional latent modulation vector. Blue dots indicate latents obtained from the seen (training) coefficients, while crosses denote interpolated latents for unseen test coefficients, estimated via cubic interpolation. This visualization suggests that the latent space encodes smooth and coherent functional variations as the PDE parameter is varied. The interpolated latents for unseen coefficients generally follow the manifold traced by the points, indicating that the latent space may provide a functionally meaningful representation aligned with the underlying parametric dependence of the PDE family.
To further investigate the effect latent interpolation across different modulation strategies, Figure presents the reconstruction results for an unseen coefficient (3 30.5) obtained via cubic interpola- tion of the latent code for each modulation method (Shift, Scale, FiLM, and GFM). The ground truth solution is shown in Figure 10 (). and Figures ()) show the corresponding reconstructions. These results highlight that only GFM produces an accurate reconstruction that closely matches the ground truth, whereas Shift, Scale, and FiLM all fail to capture the solution structure.

---

30
(a) Latent modulation #1
(b) Latent modulation #2
(c) Latent modulation #3
10
20
20
30
30
40
50
B
(d) Latent modulation #4
(c) Latent modulation #5
(f) Latent modulation #6
10
20
20
30
40
(g) Latent modulation #7
(h) Latent modulation #8
(i) Latent modulation #9
UN NNLN
10
20
30
40
50
10
20
30
40
20
30
40
50
(i) Latent modulation #
(k) Latent modulation #11
(I) Latent modulation #12
10
20
30
40
50
10
20
30
40
20
30
40
50
B
(m) Latent modulation #13
(n) Latent modulation #
(o) Latent modulation 15
10
20
30
40
50
10
20
30
40
50
20
30
40
50
B
(p) Latent modulation #16
(q) Latent modulation #17
(r) Latent modulation 418
10
20
30
40
50
10
20
30
40
50
B
(s) Latent modulation #19
(1) Latent modulation #20
Figure 9: Dynamics Latents (convection equations)
F.3 Statistical Comparison Across Modulation Methods
Table 7 provides the mean and standard deviation of PSNR across three trials for each modulation method and PDE benchmark. While Table [ in the main paper presents the mean values, this supplementary table highlights the consistency and robustness of the results by reporting the standard deviations.
25

---

(a) GT (3=30.5)
(b) Shift (8=30.5)
(c) Scale (3=30.5) FiLM (3=30.5)
GFM (3=30.5)
Figure 10: [Convection] Predicted solution snapshots for unseen coefficient 30.5 from interpolated latent codes using different modulation methods. (a) Ground truth, (b) Shift, (c) Scale, (d) FiLM, and (e) GFM. For each method, the latent code for 3 30.5 is obtained via cubic interpolation between neighboring training latents.
Table 7: Comparison with existing modulation methods with mean and standard deviation (PSNR1).
Modulation
Convection
Helmholtz: #1
Helmholtz #2
Navier-Stokes
Kuramoto-Sivashinsky
Shift 17
13.225=005997
25 25.467:0.995
18.249:10.01
25.767=00.35
16.962:10.819
Scale 17
27.410=20.00
25.66911:492
18.183
31.469=0.507
20.395=10.997
FiLM
22
27.330+1.255
33.611+0.506
22.771+-0533
31.944+0.502
22.297++1890
GFM
42.474=0.412
39.139+0.530
29.033++0556
34.754
29.827=00290
G More Experimental Results on Bidirectional Inference
Function space (A)
F Function space (44)
0,
f=( 0.,)
Laleni modulation vectur(s)
?(W: ((
"0 Mioo +5
9a ma)
7WW1h 32)
5(R + K
Visualization of the PDEfuncta pipeline for the Airfoil dataset. Two modalities in airfoil data represented separate INRs fa and fus and jointly compressed and reconstructed through a shared latent modulation vector 2.
Figure 11:
This section provides supplementary explanations for Section 5.3 First, we include schematic illustration to visually describe how the Airfoil dataset is used in the unseen neural operator setting (cf. Section 5.3.2). Figure illustrates the overall architecture of PDEfuncta applied to the Airfoil dataset. The two modalities modalities-geomeete geometry and flow field-are modeled using separate INRs fa and fa, while a shared latent modulation vector enables joint compression and bidirectional reconstruction between them. Additionally, present FWI reconstruction results for seen samples using different modulation methods in Figures [2 and
26

---

(a) Velocity
(b) Seismic #1
Seismic #2
Seismic #3
Seismic #4
Seismic #5
Figure 12: Ground truth for representative FWI sample. (a) Velocity map and (b-f) correspond- ing seismic data.
(a) Velocity
(b) Scismic #1
(c) Scismic #2
(d) Scismic #3
(c) Scismic #4
(f) Scismic #5
gg) Velocity
(h) Scismic #1
(i) Seismic #2
( Seismic #3
(k) Seismic
()) Seismic #5
(m) Velocity
(n) Seismic
(o) Seismic
(p) Seismic
(q) Seismic
Seismic #5
(s) Velocity
Seismie
(u) Seismic
(v) Seismic
(w) Seismic
(x) Seismic
Figure 13: FWI bidirectional reconstruction results for seen samples using different modulation methods. Each row shows velocity map and its corresponding seismic data reconstructed by different modulation method: (a-f) Shift, (g-l) Scale, (m-r) FiLM, and (s-x) GFM.
27

---

H
Implementation Details of Fourier Basis Construction
This section provides additional details regarding the construction and implementation of the fixed Fourier bases used in Global Fourier Modulation (GFM), supplementing Section 3.1 of the main paper. The fixed Fourier bases for each layer are constructed following the procedure introduced in [8). In all experiments, the fixed Fourier basis & for each layer is constructed by combining a set of frequencies and phase shifts (cf. Equation 3.1) ensure spectral diversity, we employ Vlw low-frequency components (Wlow Niow? 3.... 13) and 7hiigh high-frequency components Mhhhae phase shifts uniformly spaced over [0, 2m Here, 7luw, Thigh, and Pphase are hyperparameters, and the total number basis vectors per layer thus D (Noow + (higgh) Niphase- Each basis vector evaluated at uniformly spaced M points within the interval [-Tmax/2. mmmx//2 /2). where max 2T/oow The number of sampling points M is chosen match the input dimension of the corresponding layer, ensuring compatibility with weight reparameterization. In our implementation, these bases are precomputed, fixed throughout training, and not updated by gradient descent.
2
(Whigh {1, 2, .-., Rhigh )) For each frequency,
I Meta-learning based Training Algorithm
In this section, we outline the meta-learning-bet based training and inference procedures for PDEfuncta. Algorithm| describes both the training and inference phase, where both network parameters and sample-specific latent vectors are jointly optimized via nested inner-outer loop.
Algorithm Training and Inference the proposed method
1: /* Training */
2: Input: Apatio-tempora grid X' (x,
3: Randomly initialize 0 = (0., 0,), (Taa J and set Ztrain + (zero vector)
4: while not done do
5: Sample batch of output (at, u² Jee
6: Sample K examples from batch B
7: /* Inner loop #/
8: for
K do Lj Lx, (a', (x Onr Do (z Ta))) + Lxx ful,,,(;;00 ( (Etrain T.)))
9:
10: "train + ziraen Vinner Atrain
Lj
11: end for
12: 1* Outer loop #/
13:
Lg Eieo (Ixx, (a fa(l'; Ba, Ia (itraan ma))) + Ix (u², fu(((; Our Iu (straini mw))))
14: +- Mouter VOLB
15:
Mouter V LB
16: end while
18: #* Inference (g AA */
Input: New sample test from space A, coordinates X, trained parameters m
20: Output: Corresponding sample pred in space 2
A* Latent Fitting */
22: while not converged do
La =Lx(a fa(''' Pa. Ja (atestt; Ta)))
24: Stest &- Ztest
Menfer V seaes La
end while
26: 1* Prediction */
27: urred [a'(X; Onr Iu (Frest nu))
return upred
29: A Note: Inverse Problem (gl U + A) is symmetric: optimize Ztest using Ly
Lx(u ee fa(X (22660 ou Iu (Ztest; n ))),, then predict ared [a(';; Oa, Iu (a(etest Ra)) */
28

---

J Comparison with Spatial Functa
Table 8: Performance comparison (PSNR1/MSEL) between Spatial Functa and GFM on Convection,
Helmholtz, and Sirasiii-SS equations.
Convection
Helmholtz #1
Helmholtz
Kuramoto-Sivashing
Modulation
PSNR
MSE
PSNR
MSE
PSNR
MSE
PSNR
MSE
SpatialFuncta
37
20.390
0.0739
28,908
0.0068 25.976
0.0141
20.728
3,4297
GFM
42.474
0.0004
39.139
0.0006
29.033
0.0056
29.827
0.4198
1.4
Soutsffunct
0.25
Spatiffuncta
0.25
Spatiaftnnts
1.0
Spatiafundia
0.20
0.20
2.0
1.0
0.15
0.15
0.6-
0.4
0.00
0.05
230
450
400 Epoch
800
1000
1000
2003
3000
0030
5050
1000
2003
3000
4000
30%0
0.0
0
1309
2:03
2000
4:00
3000
Epoch
(a) Convection
(b) Helmholtz #1
(c) Helmholtz #2
(d) Kuramoto -Siassiiik
Figure 14: Train loss (MSE) curves comparing Spatial Functa and GFM across epochs for PDE benchmarks (Convection, Helmholtz#1, Helmholtz22, Kuramoto-Sivachinsk))
To evaluate the performance of our proposed GFM in comparison with Spatial Functa [77. we conduct experiments under the same conditions as those reported in Table Unlike global modulation approaches such as Shift, Scale, FiLM and GFM, which inject instance-specific information via low-dimensional global latent vector (e.g., z E R200 in our experimental setup), Spatial Functa utilizes a high- ligg--imensional spatially structured latent code (e.g., E 88x8816 in our implementation) designed modulate local regions of the underlying function space. Note that these latent dimensions are representative of the settings used in our experiments and may vary depending on the specific model architecture or dataset. Due to the structural difference latent space dimensionality, direct comparison under fixed latent size is infeasible. We therefore present Spatial Functa as separate baseline, following the original implementation and latent dimension, and report results SSvaassisii benchmarks. We exclude Navier-Stokes since its 3D-coordinates are incompatible with the 2D latent grid arrangement used by Spatial Functa.
on the convection, Helmholtz, and
Table reports the reconstruction accuracy for both methods, and Figure 14 shows the training loss curves. Figures [5 [6 provide the reconstructed solutions on convection and Helmholtz dataset, respectively. Notably, in Figures 15 and [6] we observe that as 8 and aj increases, the reconstruction quality of Spatial Functa (second row) deteriorates significantly, whereas GFM (third row) maintains high fidelity even in challenging high-frequency regimes. While Spatial Functa achieves moderate improvements over basic global modulation methods such as Shift, Scale, and FiLM (cf. Table IT. our GFM method consistently achieves superior PSNR and lower reconstruction errors across all considered datasets. This empirical advantage highlights the effectiveness of explicitly injecting frequency-awan priors into the modulation space, allowing GFM to better capture both global structure and fine-scale, which global modulation was previously thought to insufficient.
We also observe that Spatial Functa requires significantly higher computational cost compared to global modulation methods. The use hiih--iimesiooall spatial latent codes results in longer training times (see Section E.2). In contrast, GFM requires less computational cost, leveraging compact latent codes for faster training and inference.
29

---

32
80
60
(a) 8 10
(b) 0 20
(c) 1 30
(d) 40
(c) B = 50
10
(g)
(h) 3 30
(i) 3
i B 50
18
81
112
66
68
80
02
04
02
(k) B
=
10
()) / = 20
(m) / 30
(n) 8 40
(0) 3 = 50
Figure 15: Reconstruction results for Spatial Functa and GFM on the convection equation with {10, 20, 30, 40, 50). Rows represent ground truth (top). Spatial Functa (middle), and GFM (bottom).
(a) a1=2.0 02=9.0
(b) 01=4.0 02=9.0
(c) 01=600 02=9.0
(d) a10000 02=
(c) 01=00.0 02=9.0
(1) a1=2.0 92=
(g) 01=.0, ag=9.0
(h) a1=6.0 02=9.0
(i) a1=8.0, a2=9.0
(j) a1=10.0, 02=9.0
(k) 01=2.0, 09=9.0 0,0.00 09. (m) 01 =600, a2=9.0
(n) a1=8.0, 0z=9.0 (0) =1000.0, ag=9.0
Figure 16: Reconstruction results for Spatial Functa and GFM on the convection equation with (0), a2) = {2.00, 9.0), (4.0, 9.0), (6.0, 9.0), (8.0,90 9.0), (.0.0,900). Rows represent ground truth (top), Spatial Functa (middle), and GFM (bottom).
30


Reviews:
[{"review_id": "PNuxHiekzX", "reviewer": "Reviewer_FpZz", "summary": "In this work, the authors propose a novel modulation technique for Implicit Neural Representations (INR), called Global Fourier Modulation (GFM) that injects high-frequency information at each layer of the INR through Fourier-based reparameterization. This enables compact and accurate representation of multiple solution fields using low-dimensional latent vectors. Moreover, the authors adopt GFM to define PDEfuncta, a meta-learning framework designed to learn multi-modal PDE solution fields and support generalization to new tasks. The authors demonstrate that the method not only improves representational quality but also shows potential for forward and inverse inference\ntasks without the need for retraining.", "questions": "Why did the authors focused on a discrete set of parameters to test PDEfuncta?\nHow can one identify the correct latent space? What is the suggested size of it and how does it affect PDEfuncta?", "limitations": "The manuscript lacks benchmarks with broadband solution to be approximated. The experiments presented in the manuscript are relatively low frequency and it would be interesting to perform a stress-test on broader-band exercises", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "PDEfuncta shows the highest PSNR and lowest MSE across all datasets for 1D benchmarks. The major weakness relies on the auto-encoding of the input data to identify the latent space. This could represents a bottleneck depending on the chosen technique. While optimal performance may require careful hyperparameter tuning to select the appropriate frequency ratio, this flexibility enables GFM\nframework to be adapted to a wide range of scientific problems and data characteristics.", "was_revised": true, "final_justification": "Considering the authors' rebuttal to my initial comments, the main question I raised were answered. I am still a little doubtful about the choice of the beta values, because for larger intervals, the solution is quite degraded (at least in terms of MSE). I feel that the choice of these beta values is crucial for the correct training and usage of PDEfuncta, and that there is room for improvement in how to effectively sample beta values to achieve superior performances. Therefore, I maintain my initial ratings.", "rebuttal": "**Q1) The major weakness relies on the auto-encoding of the input data to identify the latent space. This could represents a bottleneck depending on the chosen technique.**\n\nThank you for the insightful comment. An alternative approach to circumvent the latent bottleneck is to train separate INRs individually, for example, one per $\\beta$ value when solving the convection equation. While this can enhance expressivity, it incurs a substantial computational cost. This trade-off has also been widely discussed in the computer vision community, motivating approaches like Functa, which offer significantly better efficiency while still delivering comparable performance. A central challenge for these methods, however, lies in improving their expressivity, which our work aims to address.\n\n---\n\n**Q2) Why did the authors focused on a discrete set of parameters to test PDEfuncta?**\n\nIn the context of parameterized PDEs, solution snapshots—which we aim to represent using INRs—can vary significantly with different values of the input parameters, such as $\\beta$ in the convection equation. For instance, lower beta values produce smoother, less oscillatory solutions, while higher values lead to highly oscillatory ones. Drawing an analogy to computer vision tasks, we can think of a solution snapshot at a given beta as an image, and assume that there exists a collection of such images corresponding to a discrete set of beta values. Therefore, our use of a discrete parameter set follows standard practices in PDE benchmark datasets and allows for systematic training and evaluation.\n\nWhile our model is trained on discrete $\\beta$ values, we emphasize that PDEfuncta is not limited to memorizing those specific samples. To assess its ability to generalize to unseen, continuous parameter values, we conducted an additional interpolation experiment under the same setup as Section 5.2.1 (Setting #1). Specifically, we increased the spacing of the training $\\beta$ values to intervals of 2 and 3 (from the default of 1), and kept the test $\\beta$ values fixed at the midpoints (e.g., $\\beta$ = 1.5, 2.5, ...). This setting makes interpolation more challenging by forcing the model to infer from sparser training support. Results are as follows:\n\n| Interval | PSNR ↑  | MSE ↓ |\n|--------|-----------|----------|\n| 1 (default) | 32.59 | 0.0017 |\n| 2 | 26.90 | 0.0057 |\n| 3 | 21.29 | 0.0230 |\n\n[Table 1] Interpolation performance with increasing gaps between training $\\beta$ values.\n\nThese results support our claim that PDEfuncta learns a smooth and structured latent space that generalizes beyond the discrete training set, capturing functional variations across continuous parameter ranges.\n\n---\n\n**Q3) How can one identify the correct latent space?**\n\nEvaluating the \"correctness\" of a learned latent space is challenging, as there is no ground-truth latent code for comparison. To address this, we designed diagnostic experiments (cf. Section 5.2.1) to assess the functional behavior of the latent space. In particular, we consider the latent space meaningful if it satisfies the following two properties:\nLatent interpolation: Interpolating between latent codes of nearby training parameters yields accurate predictions for intermediate, unseen parameter values.\nPartial generalization: A latent code inferred from partial observations should enable reconstruction of the full solution for unseen parameter instances, without updating the shared INR.\nThese two capabilities indicate that the latent space captures smooth and structured functional variation aligned with the underlying PDE family.\n\n---\n\n**Q4) What is the suggested size of it and how does it affect PDEfuncta?**\n\nWe thank the reviewer for raising this important point. In the main paper, we fixed the latent code dimension to 20 across all tasks for simplicity. To evaluate the sensitivity of PDEfuncta to the latent dimensionality, we conducted additional experiments with smaller (10) and larger (40) latent codes.\nThe table below summarizes the results for Helmholtz #1 (uni-directional) and Navier-Stokes (bi-directional):\n| Code Size | PSNR (Helmholtz) ↑ | PSNR (NS-A) ↑ | PSNR (NS-U) ↑ |\n| --------- | ------------------ | ------------- | ------------- |\n| 10 | 38.42 | 42.70 | 43.15 |\n| 40 | 39.75 | 43.79 | 43.78 |\n\nThese results indicate that PDEfuncta is robust to the choice of latent code dimension. Even with a small code size (i.e., 10), the model performs competitively, and increasing the dimension provides only modest gains. This shows that PDEfuncta captures functional variation efficiently without requiring large latent capacity.\n\n---\n\n**Q5) The manuscript lacks benchmarks with broadband solution to be approximated. The experiments presented in the manuscript are relatively low frequency and it would be interesting to perform a stress-test on broader-band exercises**\n\nWe thank the reviewer for the constructive feedback. While the experiments in the paper include datasets with nontrivial frequency content—such as the Kuramoto–Sivashinsky (KS) equation, which is widely used to benchmark high-frequency behaviors in PDE learning—we agree that broader-band scenarios offer a stronger stress test for evaluating spectral generalization.\n\nTo further support our claims, we conducted two additional experiments:\n\n1. KS with varying Reynolds numbers: We varied the viscosity ($ν$ $\\in$ $\\[0.5, 0.6, ..., 3.0\\]$) to change the Reynolds number and produce solutions with different turbulence levels and spectral densities.  \n2. Convection with extended $\\beta$ range (1–100): By sampling $\\beta$ from a wider range, we induce sharper gradients and more oscillatory behavior in the solution fields.\n\n| Method | PSNR ↑ | MSE ↓ |\n|--------|--------|--------|\n| Shift  | 18.02  | 7.754  |\n| Scale  | 20.03  | 4.682  |\n| FiLM   | 22.02  | 3.215  |\n| Ours   | 29.86  | 0.613  |\n\n[Table 2] Reconstruction performance on KS with varying Reynolds number\n\n| Method | PSNR ↑ | MSE ↓ |\n|--------|--------|--------|\n| Shift  | 14.82  | 0.342  |\n| Scale  | 22.42  | 0.159  |\n| FiLM   | 23.69  | 0.086  |\n| Ours   | 35.65  | 0.0013 |\n\n[Table 3] Reconstruction performance on Convection with $\\beta$ $\\in$ $\\[1, 100\\]$\n\nThese results further validate that PDEfuncta can handle broadband signal regimes and remains effective under severe spectral challenges."}, {"review_id": "77JbT5OVnp", "reviewer": "Reviewer_5L3v", "summary": "This paper proposes to combine FiLM-like conditioning with Fourier-based weight construction to achieve latent representations that are transferable across dataset samples, which are subsequently integrated in a Deep-O-Net like model to predict PDE solutions, called implicit neural representation (INR). For each data sample, a latent vector is constructed, which is used to produce scaling and bias factors to modulate networks weights. Based on that, the authors further extend to use the same latent weights to decode both the input and output of a PDE mapping. Experiments are conducted on diverse 1-step mapping PDE problems.", "questions": "- From Algorithm 1 in Appendix I, it seems that no extra model is used to compute the latent vector z (such as an auto-encoder), is this correct? Is this the standard way for training INRs?\n\n- Would the proposed method be extensible to PDEs without periodic boundaries?\n\nTypo:\n\n- Line 171: the first $\\alpha^{k.W}$ should use comma between $k$ and $W$.", "limitations": "- Limitation is discussed in Appendix B.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The claims that GFM creates better high-frequency reconstructions seem to be supported by experiments.\n\n- The proposed method outperforms baseline models, including Shift, Scale, and FiLM.\n\nWeekness:\n\n- Based on my understanding, while GFM exploits the structure of Fourier bases, which filters over space and time for both global and local patters, the baseline methods (Scale, Shift, FiLM) do not. For example, from the bottom row of Figure 6 it seems that the baseline models barely work for the demonstrated sample. As multi-resolution processing is known to be critical to model PDEs [1], a stronger baseline model would also make use of some global-local processing, such as convolutions at multiple scales.\n\n- The latent interpolation experiment in 5.2.1 appears to be rather easy as the tested beta values are quite dense within the training set betas.\n\n[1] Towards Multi-spatiotemporal-scale Generalized PDE Modeling, https://arxiv.org/abs/2209.15616", "was_revised": true, "final_justification": "The authors clarified my concerns. However, my knowledge of INRs restricts me from providing a higher rating. Thus, I will maintain my rating as accept.", "rebuttal": "**Q1) Based on my understanding, while GFM exploits the structure of Fourier bases, which filters over space and time for both global and local patterns, the baseline methods (Scale, Shift, FiLM) do not. For example, from the bottom row of Figure 6 it seems that the baseline models barely work for the demonstrated sample. As multi-resolution processing is known to be critical to model PDEs [1], a stronger baseline model would also make use of some global-local processing, such as convolutions at multiple scales.**\n\nThank you for your constructive suggestion! While we acknowledge that the suggested method could serve as a valuable baseline for comparison, adapting it to the INR framework may be nontrivial. We note that Figure 6 illustrates the reconstruction results of INRs with different modulation techniques, not neural operators results. We will expand the discussion on this aspect—particularly regarding local and global feature representations—by citing the referenced work in the revised manuscript.\n\n---\n\n**Q2) The latent interpolation experiment in 5.2.1 appears to be rather easy as the tested beta values are quite dense within the training set betas.**\n\nWe thank the reviewer for pointing this out. The default interpolation setting in Section 5.2.1 indeed uses densely sampled training parameters (interval = 1), which could make interpolation relatively easier.\n\nTo more rigorously evaluate PDEfuncta's generalization ability, we designed additional interpolation experiments, where interpolation evaluation is performed under increasingly challenging conditions. Specifically, during test time, we restrict the available latent codes to those corresponding to training $\\beta$ values sampled at coarser intervals (e.g., 2, 3, 5), and perform interpolation only using those. This allows us to assess the model’s ability to interpolate under limited latent support.\n\n| Interval | PSNR ↑ | MSE ↓   |\n|----------|--------|---------|\n| 1        | 32.59  | 0.0017  |\n| 2        | 26.90  | 0.0057  |\n| 3        | 21.29  | 0.0230  |\n| 4        | 14.91  | 0.1035  |\n| 5        | 11.75  | 0.2593  |\n\n[Table 1] Latent interpolation under reduced support\n\nWhile the performance gradually degrades as the available latent support becomes sparser, the model still produces reasonable reconstructions. Notably, even at interval 5, PDEfuncta outperforms Shift, Scale, and FiLM baselines evaluated under the easier interval-1 setting (see Figure 5 and Table 2 in the main paper). This further supports the robustness of our learned latent space and its ability to generalize under limited information.\n\n---\n\n**Q3) From Algorithm 1 in Appendix I, it seems that no extra model is used to compute the latent vector z (such as an auto-encoder), is this correct? Is this the standard way for training INRs?**\n\nYes, the reviewer is correct. As shown in Algorithm 1, our method does not use an explicit encoder (such as in a VAE or autoencoder) to compute the latent vector \\( z \\). Instead, we adopt an *auto-decoder* approach, where each z is treated as a learnable parameter initialized randomly and optimized directly via gradient descent.\n\nThis approach is widely used in implicit neural representation frameworks such as DeepSDF [1] and Functa [2]. It provides more direct control over per-instance representations and is particularly suitable for high-fidelity reconstruction tasks as in our PDE scenarios.\n\n[1] Park, Jeong Joon, et al. \"Deepsdf: Learning continuous signed distance functions for shape representation.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.\n\n[2] Dupont, Emilien, et al. \"From data to functa: Your data point is a function and you can treat it like one.\" International Conference on Machine Learning (pp. 5350-5364). (2022).\n\n---\n\n**Q4) Would the proposed method be extensible to PDEs without periodic boundaries?**\n\nThank you for the insightful question. We would like to clarify that the proposed method does not require periodic boundary conditions. While the term \"Fourier\" appears in the GFM design, it refers to a reparameterization of the MLP weights in a frequency-aware space, and does not assume periodicity of the input domain.\n\nImportantly, several of our evaluated benchmarks involve non-periodic boundaries. For instance, both the Airfoil and Pipe datasets include non-periodic boundaries, yet PDEfuncta achieves strong performance on these tasks (see Table 4). These results demonstrate the method's ability to generalize beyond non-periodic boundary settings.\n\n---\n\n**Q5) [Typo issue] Line 171**\n\nWe thank the reviewer for pointing this out. As noted, the correct notation should be $\\alpha^{k, W}$ instead of $\\alpha^{k.W}$. We will fix this typo in the revised version."}, {"review_id": "t3AmzDkgWk", "reviewer": "Reviewer_CbRV", "summary": "The authors introduce Global Fourier Modulation (GFM) which addresses the challenge of spectral bias in implicit neural representations (INRs) when modeling complex solution fields of partial differential equations (PDEs). They further propose PDEfuncta, a meta-learning framework that learns bidirectional mappings between paired function spaces.  Experiments demonstrate substantial improvements in reconstruction accuracy and generalization to unseen parameters and resolutions without retraining.", "questions": "* Neural Operators have been successfully applied to spatiotemporal evolution applications. The applications in the paper seem to be all steady state examples. Can PDEFuncta be extended to spatiotemporal evolutions?\n* Can the authors provide frequency power spectrum plots of the GFM reconstructions and report the fourier error metrics[1] for the evaluated methods?\n\n[1] Appendix B in \"PDEBench: An Extensive Benchmark for Scientific Machine Learning\", Takamoto et al., NeurIPS 22", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths:\n* Bidirectional Operator Learning: PDEfuncta supports both forward and inverse mappings in a unified latent space, matching or exceeding neural operator baselines which gives it a clear advantage over PINN and Neural Operators\n* GFM seems to provide a huge gain in terms of PSNR as compared to Shift, Scale or FiLM\n### Weaknesses:\n* The mitigation of spectral bias has not been shown empirically (e.g. with the help of frequency power spectrum plots)", "was_revised": true, "final_justification": "Good paper for mitigating spectral bias in operator learning. Finds use in many physics problems with high frequency information like turbulence modeling and fine-scale phenomena like vortices.", "rebuttal": "**Q1) The mitigation of spectral bias has not been shown empirically (e.g. with the help of frequency power spectrum plots). Can the authors provide frequency power spectrum plots of the GFM reconstructions and report the fourier error metrics[1] for the evaluated methods?**\n\n**A1)** We thank the reviewers for highlighting the importance of explicitly evaluating spectral bias. To address this, we performed an additional frequency-domain analysis on the Kuramoto–Sivashinsky (KS) dataset using Fourier RMSE, which measures the root-mean-square error between the ground-truth and reconstructed solutions in the frequency domain. Specifically, we computed the error within three frequency bands: low, mid, and high.\n\nThe results are summarized below:\n\n| Method | High ↓ | Mid ↓ | Low ↓ | Avg ↓ |\n|--------|--------|--------|--------|--------|\n| Shift  | 6.396  | 0.011  | 0.055  | 2.806  |\n| Scale  | 4.335  | 0.015  | 0.073  | 1.898  |\n| FiLM   | 3.432  | 0.016  | 0.074  | 1.511  |\n| Ours   | 1.386  | 0.055  | 0.086  | 0.664  |\n\n[Table 1] Fourier RMSE across frequency bands on KS dataset\n\nThese results confirm that our GFM approach significantly reduces reconstruction error in the high-frequency regime compared to baseline modulation methods. While the low and mid-frequency components are reconstructed comparably across methods, GFM exhibits clear advantages in preserving high-frequency content, directly addressing the concern of spectral bias.\n\n-----\n\n**Q2) Neural Operators have been successfully applied to spatiotemporal evolution applications. The applications in the paper seem to be all steady state examples. Can PDEFuncta be extended to spatiotemporal evolutions?**\n\n\n**A2)** We thank the reviewer for this observation. PDEfuncta is readily extendable to spatio-temporal evolution tasks. In our Navier–Stokes experiment (see Figure 3), we model a temporal operator by treating two contiguous time windows as paired function spaces:\n\n$\\mathcal{A}$: $t \\in [0, T_a]$ is represented by an INR $f_a$,\n\n$\\mathcal{U}$: $t \\in [T_a, T_u]$ is represented by a second INR $f_u$\n\nBoth networks share the same latent vector $z$. Consequently, the latent map $G : \\mathcal{A} \\rightarrow \\mathcal{U}$ learns the flow operator that advances the solution by $\\Delta t=T_u−T_a$. This procedure can be applied to long-horizon roll-outs on the Navier-Stokes dataset and other dynamical benchmarks."}, {"review_id": "coS5H9QRiR", "reviewer": "Reviewer_hdMN", "summary": "The authors propose Global Fourier Modulation (GFM) to fix spectral bias in modulated INRs. They reparameterize network weights using fixed cosine bases (randomized in phase) + sample specific modulation coefficients.\nThey build upon the Functa idea, to represent paired function spaces (shapes, temporal fields,...) with shared latent vectors for bidirectional inference.", "questions": "+ How does GFM compare to other methods in terms of frequency metrics (powerspectrum, nmse,...)? If you bin low/high frequencies, how does it compare? Can you show empirical evidence for spectral bias mitigation?\n+ Did you do a fair hyperparameter search for the other methods/architectures? In Table 1 the MSE of the KS-equation is over 7 for Shift; this seems unusually high for normalized data.\n+ What is your main point you want your reader to take away?\n    - GFM is good to mitigate spectral bias? \n    - Exploration of shared latent modulation vector behaviour\n    - Bidirectional inference possibilities\nA clear focus on one topic helps the overall reading-flow. It was hard to follow in a first read. Currently, it is necessary to read into prior work to understand the methods presented.\n+ Most of the used PDEs are toy datasets (Convection, Helmholtz), in regards to how much higher frequencies influence the PDE. Given that we know very little about the Reynoldsnumber of the datasets, it is hard to assume how much of a challenge the given PDEs are. Can you evaluate your method on dataset like Kolmogorov-Flow or different setups of the KS equation with set higher Reynoldsnumbers (so know to expect turbulences in the data)?\n- Given the claim that your method can compete with NO and PINNs, there should also be more extensive experimental setups, that should be in the main paper (not appendix). There is a single table in the main paper (Table 4), which also indicated that the presented method is beatable by other methods. Can you show a more extensive anlysis of the PDEFuncta part of the paper? Compare on complex dataset the capabilities and to modern baselines.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n- Novel idea to enable bidirectional inference between paired function spaces via shared latents (meta-learning framework)\n- Creative extension of the cosine reparametrization to modulate INRs\n- Methods shows consistent improvements on their selected datasets\n- Shows potential for unified forward/inverse operator learning\n\n### Weaknesses\n- Only a theoretical explanation for mitigation of spectral bias (no frequency domain analysis or frequency related metrics for empirical validation of claims).\n- More extensive comparison to other spectral bias mitigation methods, to show the direct comparision on spectral metrics (SIREN, WIRE,...).\n- Only limited scope of experiments shown, while claims are high (like solves spectral bias, the overall comparison to neural operators (NO) or physics informed neural networks (PINN) is weak from an experimental standpoint).\n- Typing/Spelling mistakes in text/figures; Table and figure descriptions are weak/need more context\n- Hard to read/follow/understand (missing connected overall story). Focusing on one topic (GFM or PDEFuncta) could have benefited the storyline.\n\nThe paper has interesting ideas, but validation of the spectral improvements are missing (spectral analysis), as well as the validation of beating most existing state of the art models. Experimental setups and results are mostly banished to the appendix. Also missing an extensive comparison to other spectral methods (SIREN,...).\nThe empirical improvements might exist trough the proposed method, but we do not know if they come from better spectral properties or just a better reparameterization in general.", "was_revised": true, "final_justification": "The authors have been addressing some of my concerns via experiments/tables, however given that no images/plot could be shared within the rebuttal period, there are still open questions (comparison of powespectra).\nIn the believe that the authors will address the other issues I raised, in the camera-ready version I am going to reevaluate my given score.", "rebuttal": "**W1 & Q1) How does GFM compare to other methods in terms of frequency metrics ... Can you show empirical evidence for spectral bias mitigation?**\n\nThank you for the question. To empirically evaluate how GFM compares to other methods in the frequency domain, we conducted a Fourier RMSE analysis on the KS dataset, following prior approaches for assessing spectral bias.\nWe computed the reconstruction error across low, mid, and high frequency bands and compared GFM to Shift, Scale, and FiLM-based modulation. The results are presented below:\n| Frequency Band | Shift | Scale | FiLM | Ours (GFM) |\n|----------------|-------|--------|------|------------|\n| **High**       | 6.396 | 4.335  | 3.432 | **1.386** |\n| **Mid**        | 0.011 | 0.015  | 0.016 | **0.055** |\n| **Low**        | 0.055 | 0.073  | 0.074 | **0.086** |\n| **Avg**        | 2.806 | 1.898  | 1.511 | **0.664** |\n\nAs shown, GFM achieves the lowest average error and significantly improves accuracy in high-frequency bands, offering clear empirical evidence of spectral bias mitigation.\n\n---\n\n**W2) More extensive comparison to other spectral bias mitigation methods, to show the direct comparision on spectral metrics (SIREN, WIRE,...).**\n\nThank you very much for your valuable suggestion. As described in the supplementary source code, PDEfuncta utilizes SIREN as its backbone architecture. To provide a more extensive comparison, we have conducted additional experiments using alternative backbones, which are Wavelet Implicit Neural Representation (WIRE) [1] and Fourier Feature Mapping (FFM) [2]. The experiments were performed using the Convection dataset with coefficients $\\beta$ ranging from 1 to 50.\n\n### FFM Backbone\n| Modulation | PSNR | MSE |\n|------------|------|------|\n| Shift      | 21.127   | 0.0322  |\n| Scale      | 23.813 | 0.0175  |\n| FiLM       | 23.219 | 0.0186  |\n| **Ours (GFM)** | **29.879** | **0.0044** |\n\n### WIRE Backbone\n| Modulation | PSNR | MSE |\n|------------|------|------|\n| Shift      | 27.492 | 0.0401 |\n| Scale      | 32.894 | 0.0029 |\n| FiLM       | 29.282 | 0.0047 |\n| **Ours (GFM)** | **33.047** | **0.0025** |\n\nNote that our proposed GFM modulation is theoretically grounded specifically in the characteristics of the SIREN backbone and hyperparameter settings are optimized accordingly. However, as clearly shown in the table, even with FFN and WIRE backbones, GFM consistently outperforms other modulation techniques. This underscores GFM's effectiveness in mitigating spectral bias across different architectures.\n\n\n*[1] Saragadam, Vishwanath, et al. \"Wire: Wavelet implicit neural representations.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.*\n\n*[2] Tancik, Matthew, et al. \"Fourier features let networks learn high frequency functions in low dimensional domains.\" Advances in neural information processing systems 33 (2020): 7537-7547.*\n\n--- \n\n**W3 & Q5) Given the claim that your method can compete with NO and PINNs, there should also be more extensive experimental setups, … Can you show a more extensive anlysis of the PDEFuncta part of the paper? Compare on complex dataset the capabilities and to modern baselines.**\n\nWe sincerely thank the reviewer for highlighting this important concern. The primary objective of our PDEfuncta framework is to accurately represent high-frequency content frequently observed in PDE solution snapshots. Our experiments confirmed that PDEfuncta successfully captures these complex high-frequency details, especially when modeling parametric PDE solutions with modulated INRs.\n\nWe additionally demonstrated PDEfuncta's capabilities on downstream tasks such as neural operator scenarios, where it produced results comparable to modern neural operator baselines—achieving the best performance on the NACA dataset. However, we agree that to fully realize its potential as a general neural operator, further enhancements are necessary, and we will include this point explicitly in our discussion.\n\nLastly, we would like to respectfully clarify that we have not claimed direct comparability to PINNs. We appreciate your pointing this out, and we will ensure this distinction is clearly communicated in our revision.\n\n---\n\n**W4) Typing/Spelling mistakes in text/figures; Table and figure descriptions are weak/need more context**\n\nWe acknowledge the importance of clear and detailed captions for tables and figures. Accordingly, we will significantly enhance their descriptions, adding more contextual information to aid comprehension and effectively highlight our results. These improvements will be reflected in the camera-ready version.\n\n---\n\n**W5) Focusing on one topic (GFM or PDEFuncta) could have benefited the storyline.**\n\nWe sincerely thank the reviewer for this valuable feedback. To clearly establish the connected storyline, we introduced Global Fourier Modulation (GFM) specifically as a modulation method addressing the spectral bias issue, which was an essential prerequisite step towards extending our framework to PDEfuncta. We acknowledge the reviewer's suggestion and will explicitly clarify in our revision why GFM is crucial and how it naturally leads into PDEfuncta.\n\n---\n\n**Q2) Did you do a fair hyperparameter search for the other methods/architectures? In Table 1 the MSE of the KS-equation is over 7 for Shift; this seems unusually high for normalized data.**\n\nThe Kuramoto–Sivashinsky (KS) dataset used in our experiments is not normalized, which naturally leads to relatively high MSE values. The raw solution fields contain large-magnitude, high-frequency variations due to the chaotic nature of the dynamics, so MSE values in the range of 7 are not unexpected in this setting.\n\n---\n\n**Q3) What is your main point you want your reader to take away?**\n\nWe sincerely appreciate the reviewer’s insightful and detailed comments. Below, we address each specific point:\n\n**Q3-1) GFM is good to mitigate spectral bias?**\n\nWe fully agree that mitigating spectral bias is central to our study. As outlined in the manuscript, Global Fourier Modulation (GFM) directly addresses spectral bias, a prevalent limitation in standard implicit neural representations (INRs), by explicitly injecting frequency-aware priors into the modulation of INRs. The theoretical justification for this approach is provided clearly in Theorem 1, where Fourier reparameterization notably increases the gradient contributions for high-frequency components. Empirically, this improvement is demonstrated through rigorous experiments across various scientific PDE benchmarks (Figure 1, Figure 6, and Table 1), particularly highlighting GFM's superior capability to accurately capture high-frequency details compared to existing modulation methods.\n\n**Q3-2) Exploration of shared latent modulation vector behaviour**\n\nIndeed, in modulated INRs, latent vectors (z) typically serve as simple representations or compressed encodings of data. Our contribution extends this notion significantly by investigating the dynamics of these latent vectors in relation to PDE parameters. Specifically, in Appendix F, we provide detailed empirical analyses demonstrating that variations in latent vectors correspond closely to PDE coefficients. Further, we introduce PDEfuncta, which uniquely represents paired scientific datasets through a shared latent vector, thereby facilitating bidirectional inference within scientific machine learning scenarios.\n\n**Q3-3) Bidirectional inference possibilities ... it is necessary to read into prior work to understand the methods presented.**\n\nIn our camera-ready version, we commit to enhancing the related work and proposed method sections to better contextualize PDEfuncta within the literature. Furthermore, Appendix I will be expanded to explicitly include comprehensive details about the inference process, thereby clearly illustrating the training-inference transition. Again, we are grateful for your valuable feedback, which will greatly enhance the readability and clarity of our manuscript.\n\n\n---\n\n**Q4) Most of the used PDEs are toy datasets (Convection, Helmholtz), in regards to how much higher frequencies influence the PDE. ... Can you evaluate your method on dataset like Kolmogorov-Flow or different setups of the KS equation with set higher Reynoldsnumbers?**\n\nAs demonstrated in prior work (e.g., [3]), approximating PDE solutions using neural networks presents its own set of challenges (e.g., spectral bias), even for problems that are traditionally considered tractable in numerical analysis. Examples include the convection equation with higher beta values and the Helmholtz equation with shorter wavelengths. Nevertheless, we greatly appreciate the reviewer’s suggestion and have conducted additional experiments to assess the applicability of the proposed method to more complex and challenging problem settings.\n\nTo this end, we conducted an additional experiment using the Kuramoto–Sivashinsky (KS) equation, where we varied the viscosity parameter $\\nu$ to induce different levels of turbulence. In our original setup, all training samples used a fixed viscosity $\\nu = 0.1$. In the new experiment, we generated 100 training samples using five distinct viscosity values:\n$\\nu$ $\\in$ {0.05, 0.10, 0.15, 0.20, 0.25},$\\quad$ $\\text{20 samples per value}.$\nThis setup results in a mixture of chaotic (low $\\nu$) and smooth (high $\\nu$) solutions, enabling a more comprehensive stress test.\n\n| Method | PSNR ↑ | MSE ↓ |\n|--------|--------|--------|\n| Shift  | 18.02  | 7.75   |\n| Scale  | 20.03  | 4.68   |\n| FiLM   | 22.02  | 3.21   |\n| **Ours** | **29.86**  | **0.61**   |\n\nOur model continued to perform robustly under this more complex and diverse setting, demonstrating its ability to generalize across multiple turbulence regimes. \n\n\n*[3] Krishnapriyan, Aditi, et al. \"Characterizing possible failure modes in physics-informed neural networks.\" Advances in neural information processing systems 34 (2021): 26548-26560.*"}, {"review_id": "CXbN4HGhHD", "reviewer": "Reviewer_tDdd", "summary": "This paper deals with the high frequency feature of solutions of some PDEs, e.g., Helmholtz equations. The proposed approach is based on two components: 1. The Functa framework [1] of modulating implicit neural representations (INR); 2. Fourier reparameterization [2]. The latter technique is known for capturing high frequency features with theoretic guarantee.\n\nIn experiments, the authors designed two sets of study: 1. Compression and reconstruction; 2. Bidirectional inference. The proposed method outperforms baselines in both ways.\n\n[1] Emilien Dupont, Hyunjik Kim, SM Eslami, Danilo Rezende, and Dan Rosenbaum. From\ndata to functa: Your data point is a function and you can treat it like one. arXiv preprint\narXiv:2201.12204, 2022.\n\n[2] Kexuan Shi, Xingyu Zhou, and Shuhang Gu. Improved implicit neural representation with\nfourier reparameterized training. In Proceedings of the IEEE/CVF Conference on Computer\nVision and Pattern Recognition, pages 25985–25994, 2024.", "questions": "N.A.", "limitations": "N.A.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strength**\n\nQuality: The paper demonstrates GFM's superior accuracy, improving PSNR and reducing MSE on 5 types of problems, e.g., convection, Helmoltz1&2, Navier-Stokes, KS (Table 1), and shows competitive performance against neural operator baselines on Euler-NACA and pipeline datasets (Table 4).\n\nClarity: The paper is logically organized with clear problem statements and effectively uses figures (e.g., Figure 2 for GFM pipeline) to illustrate concepts and results.\n\nSignificance: This work features bidirectional inference. It tackles the significant problem of spectral bias in INRs and introduces the PDEfuncta framework, enabling novel multi-modal and bidirectional functional inference between paired function spaces.\n\nOriginality: The Global Fourier Modulation (GFM) is a novel contribution extending Fourier reparameterization to modulated INRs, and its integration into the PDEfuncta framework for bidirectional inference is a novel architectural design.\n\n**Weakness**\n\n1. The main concern is about scalability. The proposed method belongs to INR category which is good at capturing details and high frequency feature. However, the architecture of INR mostly relies on MLP, and in the scenario of PDE, how scalable is such method? This paper does not include related analysis.\n2. In section 4.2, the optimization variables of inner loop and outer loop are not mentioned, causing a little confusion. The variable of inner loop should be $z$, and other variables will be optimized in outer loop according to Algo. 1 in Appendix. A simple explanation in Section 4.2 could be more convenient to readers.", "was_revised": true, "final_justification": "After checking other reviews, I didn't find any unresolved major concern. However, my opinion about the limitation on scale also remains, as the author admitted MLP being the main type of architecture. Therefore, my score remains to 4.", "rebuttal": "**Q1) The main concern is about scalability. The proposed method belongs to INR category which is good at capturing details and high frequency feature. However, the architecture of INR mostly relies on MLP, and in the scenario of PDE, how scalable is such method? This paper does not include related analysis.**\n\nCurrently, coordinate-based neural networks (e.g., INRs, PINNs, NeRFs) predominantly use MLPs as their backbone, since alternative architectures such as CNNs, RNNs, or Transformers typically assume specific structural priors in the data that are not present in coordinate-based representations. Recent advances in network design have introduced the Kolmogorov–Arnold Network (KAN) [1], inspired by the Kolmogorov–Arnold representation theorem. KANs have been proposed as lightweight and potentially more expressive alternatives to MLPs. However, their use in this context is still at an early stage and requires further investigation. Given this current landscape, we adopt MLPs in our work and explore the mentioned direction through additional numerical experiments. The results of the additional experimentation are as follows. \n\n[1] Liu, Ziming, et al. \"Kan: Kolmogorov-arnold networks.\" arXiv preprint arXiv:2404.19756 (2024).\n\n---\n\n**Q2) In section 4.2, the optimization variables of inner loop and outer loop are not mentioned, causing a little confusion. The variable of inner loop should be , and other variables will be optimized in outer loop according to Algo. 1 in Appendix. A simple explanation in Section 4.2 could be more convenient to readers.**\n\nWe will enhance the explanation of the meta-learning algorithm in Section 4.2 by clearly specifying the optimization variables used in the inner and outer loops. Specifically, we will state that the inner loop optimizes only the latent modulation vector z individually for each data sample, while the outer loop optimizes the shared parameters of the INR networks $\\theta_a$, $\\theta_u$, and the modulation network parameters $\\pi_a$, $\\pi_u$. This clarification will align the main text with Algorithm 1 in the Appendix I, improving readability and comprehension."}]

Author rebuttal:

Meta-review: The paper addresses the spectral bias problem in implicit neural representations (INRs) for learning to represent high-frequency components often found in physical fields. The application domain is learning PDE solutions, with the objective of obtaining compressed functional representations of physical fields characteristic of a given phenomenon.

The authors propose a new technique, Global Fourier Modulation (GFM), which directly parameterizes INR weights using shared cosine bases for a given PDE, along with PDE-sample-specific modulation coefficients. GFM is designed to modulate frequency information in the network weights directly, enabling finer control than previous methods. The idea is further instantiated for learning neural operators between two function spaces, based on a shared latent representation, allowing the method to learn paired representation spaces. Experimental validation is conducted on a series of reconstruction and prediction problems.

The reviewers highlight the novelty of the proposed reparameterization, describing it as a solid and original technical contribution that achieves improvements over baselines. In their initial reviews, they noted some weaknesses, mainly concerning the experimental validation. During the rebuttal, the authors addressed most concerns and added new experiments to respond to the reviewers’ comments. The overall consensus is clearly in favor of acceptance.
Decision: Accept (poster)