Title: PDEfuncta: Spectrally-Aware Neural Representation for PDE Solution Modeling
Venue: NeurIPS 2025
Accepted: yes
Keywords: SciML

meta-learning

data compression
Abstract: Scientific machine learning often involves representing complex solution fields that exhibit high-frequency features such as sharp transitions, fine-scale oscillations, and localized structures. While implicit neural representations (INRs) have shown promise for continuous function modeling, capturing such high-frequency behavior remains a challenge—especially when modeling multiple solution fields with a shared network. Prior work addressing spectral bias in INRs has primarily focused on single-instance settings, limiting scalability and generalization. In this work, we propose Global Fourier Modulation (GFM), a novel modulation technique that injects high-frequency information at each layer of the INR through Fourier-based reparameterization. This enables compact and accurate representation of multiple solution fields using low-dimensional latent vectors. Building upon GFM, we introduce PDEfuncta, a meta-learning framework designed to learn multi-modal solution fields and support generalization to new tasks. Through empirical studies on diverse scientific problems, we demonstrate that our method not only improves representational quality but also shows potential for forward and inverse inference tasks without the need for retraining.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper deals with the high frequency feature of solutions of some PDEs, e.g., Helmholtz equations. The proposed approach is based on two components: 1. The Functa framework [1] of modulating implicit neural representations (INR); 2. Fourier reparameterization [2]. The latter technique is known for capturing high frequency features with theoretic guarantee.\n\nIn experiments, the authors designed two sets of study: 1. Compression and reconstruction; 2. Bidirectional inference. The proposed method outperforms baselines in both ways.\n\n[1] Emilien Dupont, Hyunjik Kim, SM Eslami, Danilo Rezende, and Dan Rosenbaum. From\ndata to functa: Your data point is a function and you can treat it like one. arXiv preprint\narXiv:2201.12204, 2022.\n\n[2] Kexuan Shi, Xingyu Zhou, and Shuhang Gu. Improved implicit neural representation with\nfourier reparameterized training. In Proceedings of the IEEE/CVF Conference on Computer\nVision and Pattern Recognition, pages 25985–25994, 2024.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n\nQuality: The paper demonstrates GFM's superior accuracy, improving PSNR and reducing MSE on 5 types of problems, e.g., convection, Helmoltz1&2, Navier-Stokes, KS (Table 1), and shows competitive performance against neural operator baselines on Euler-NACA and pipeline datasets (Table 4).\n\nClarity: The paper is logically organized with clear problem statements and effectively uses figures (e.g., Figure 2 for GFM pipeline) to illustrate concepts and results.\n\nSignificance: This work features bidirectional inference. It tackles the significant problem of spectral bias in INRs and introduces the PDEfuncta framework, enabling novel multi-modal and bidirectional functional inference between paired function spaces.\n\nOriginality: The Global Fourier Modulation (GFM) is a novel contribution extending Fourier reparameterization to modulated INRs, and its integration into the PDEfuncta framework for bidirectional inference is a novel architectural design.\n\n**Weakness**\n\n1. The main concern is about scalability. The proposed method belongs to INR category which is good at capturing details and high frequency feature. However, the architecture of INR mostly relies on MLP, and in the scenario of PDE, how scalable is such method? This paper does not include related analysis.\n2. In section 4.2, the optimization variables of inner loop and outer loop are not mentioned, causing a little confusion. The variable of inner loop should be $z$, and other variables will be optimized in outer loop according to Algo. 1 in Appendix. A simple explanation in Section 4.2 could be more convenient to readers.\n\nQUESTIONS:\nN.A.\n\nLIMITATIONS:\nN.A.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose Global Fourier Modulation (GFM) to fix spectral bias in modulated INRs. They reparameterize network weights using fixed cosine bases (randomized in phase) + sample specific modulation coefficients.\nThey build upon the Functa idea, to represent paired function spaces (shapes, temporal fields,...) with shared latent vectors for bidirectional inference.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- Novel idea to enable bidirectional inference between paired function spaces via shared latents (meta-learning framework)\n- Creative extension of the cosine reparametrization to modulate INRs\n- Methods shows consistent improvements on their selected datasets\n- Shows potential for unified forward/inverse operator learning\n\n### Weaknesses\n- Only a theoretical explanation for mitigation of spectral bias (no frequency domain analysis or frequency related metrics for empirical validation of claims).\n- More extensive comparison to other spectral bias mitigation methods, to show the direct comparision on spectral metrics (SIREN, WIRE,...).\n- Only limited scope of experiments shown, while claims are high (like solves spectral bias, the overall comparison to neural operators (NO) or physics informed neural networks (PINN) is weak from an experimental standpoint).\n- Typing/Spelling mistakes in text/figures; Table and figure descriptions are weak/need more context\n- Hard to read/follow/understand (missing connected overall story). Focusing on one topic (GFM or PDEFuncta) could have benefited the storyline.\n\nThe paper has interesting ideas, but validation of the spectral improvements are missing (spectral analysis), as well as the validation of beating most existing state of the art models. Experimental setups and results are mostly banished to the appendix. Also missing an extensive comparison to other spectral methods (SIREN,...).\nThe empirical improvements might exist trough the proposed method, but we do not know if they come from better spectral properties or just a better reparameterization in general.\n\nQUESTIONS:\n+ How does GFM compare to other methods in terms of frequency metrics (powerspectrum, nmse,...)? If you bin low/high frequencies, how does it compare? Can you show empirical evidence for spectral bias mitigation?\n+ Did you do a fair hyperparameter search for the other methods/architectures? In Table 1 the MSE of the KS-equation is over 7 for Shift; this seems unusually high for normalized data.\n+ What is your main point you want your reader to take away?\n    - GFM is good to mitigate spectral bias? \n    - Exploration of shared latent modulation vector behaviour\n    - Bidirectional inference possibilities\nA clear focus on one topic helps the overall reading-flow. It was hard to follow in a first read. Currently, it is necessary to read into prior work to understand the methods presented.\n+ Most of the used PDEs are toy datasets (Convection, Helmholtz), in regards to how much higher frequencies influence the PDE. Given that we know very little about the Reynoldsnumber of the datasets, it is hard to assume how much of a challenge the given PDEs are. Can you evaluate your method on dataset like Kolmogorov-Flow or different setups of the KS equation with set higher Reynoldsnumbers (so know to expect turbulences in the data)?\n- Given the claim that your method can compete with NO and PINNs, there should also be more extensive experimental setups, that should be in the main paper (not appendix). There is a single table in the main paper (Table 4), which also indicated that the presented method is beatable by other methods. Can you show a more extensive anlysis of the PDEFuncta part of the paper? Compare on complex dataset the capabilities and to modern baselines.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

comments: SUMMARY:
The authors introduce Global Fourier Modulation (GFM) which addresses the challenge of spectral bias in implicit neural representations (INRs) when modeling complex solution fields of partial differential equations (PDEs). They further propose PDEfuncta, a meta-learning framework that learns bidirectional mappings between paired function spaces.  Experiments demonstrate substantial improvements in reconstruction accuracy and generalization to unseen parameters and resolutions without retraining.

STRENGTHS AND WEAKNESSES:
### Strengths:
* Bidirectional Operator Learning: PDEfuncta supports both forward and inverse mappings in a unified latent space, matching or exceeding neural operator baselines which gives it a clear advantage over PINN and Neural Operators
* GFM seems to provide a huge gain in terms of PSNR as compared to Shift, Scale or FiLM
### Weaknesses:
* The mitigation of spectral bias has not been shown empirically (e.g. with the help of frequency power spectrum plots)

QUESTIONS:
* Neural Operators have been successfully applied to spatiotemporal evolution applications. The applications in the paper seem to be all steady state examples. Can PDEFuncta be extended to spatiotemporal evolutions?
* Can the authors provide frequency power spectrum plots of the GFM reconstructions and report the fourier error metrics[1] for the evaluated methods?

[1] Appendix B in "PDEBench: An Extensive Benchmark for Scientific Machine Learning", Takamoto et al., NeurIPS 22

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to combine FiLM-like conditioning with Fourier-based weight construction to achieve latent representations that are transferable across dataset samples, which are subsequently integrated in a Deep-O-Net like model to predict PDE solutions, called implicit neural representation (INR). For each data sample, a latent vector is constructed, which is used to produce scaling and bias factors to modulate networks weights. Based on that, the authors further extend to use the same latent weights to decode both the input and output of a PDE mapping. Experiments are conducted on diverse 1-step mapping PDE problems.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The claims that GFM creates better high-frequency reconstructions seem to be supported by experiments.\n\n- The proposed method outperforms baseline models, including Shift, Scale, and FiLM.\n\nWeekness:\n\n- Based on my understanding, while GFM exploits the structure of Fourier bases, which filters over space and time for both global and local patters, the baseline methods (Scale, Shift, FiLM) do not. For example, from the bottom row of Figure 6 it seems that the baseline models barely work for the demonstrated sample. As multi-resolution processing is known to be critical to model PDEs [1], a stronger baseline model would also make use of some global-local processing, such as convolutions at multiple scales.\n\n- The latent interpolation experiment in 5.2.1 appears to be rather easy as the tested beta values are quite dense within the training set betas.\n\n[1] Towards Multi-spatiotemporal-scale Generalized PDE Modeling, https://arxiv.org/abs/2209.15616\n\nQUESTIONS:\n- From Algorithm 1 in Appendix I, it seems that no extra model is used to compute the latent vector z (such as an auto-encoder), is this correct? Is this the standard way for training INRs?\n\n- Would the proposed method be extensible to PDEs without periodic boundaries?\n\nTypo:\n\n- Line 171: the first $\\alpha^{k.W}$ should use comma between $k$ and $W$.\n\nLIMITATIONS:\n- Limitation is discussed in Appendix B.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}

comments: SUMMARY:
In this work, the authors propose a novel modulation technique for Implicit Neural Representations (INR), called Global Fourier Modulation (GFM) that injects high-frequency information at each layer of the INR through Fourier-based reparameterization. This enables compact and accurate representation of multiple solution fields using low-dimensional latent vectors. Moreover, the authors adopt GFM to define PDEfuncta, a meta-learning framework designed to learn multi-modal PDE solution fields and support generalization to new tasks. The authors demonstrate that the method not only improves representational quality but also shows potential for forward and inverse inference
tasks without the need for retraining.

STRENGTHS AND WEAKNESSES:
PDEfuncta shows the highest PSNR and lowest MSE across all datasets for 1D benchmarks. The major weakness relies on the auto-encoding of the input data to identify the latent space. This could represents a bottleneck depending on the chosen technique. While optimal performance may require careful hyperparameter tuning to select the appropriate frequency ratio, this flexibility enables GFM
framework to be adapted to a wide range of scientific problems and data characteristics.

QUESTIONS:
Why did the authors focused on a discrete set of parameters to test PDEfuncta?
How can one identify the correct latent space? What is the suggested size of it and how does it affect PDEfuncta?

LIMITATIONS:
The manuscript lacks benchmarks with broadband solution to be approximated. The experiments presented in the manuscript are relatively low frequency and it would be interesting to perform a stress-test on broader-band exercises
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-07