OrthoSolver: A Neural Proper Orthogonal Decomposition Solver For PDEs
Authors: Ying Pang
Jingyuan Wang
Jiahao Ji
Fanhao Mu
Venue: ICLR 2026 Poster (2026)
Keywords: Partial differential equations
Neural operator
Information-theoretic
Proper orthogonal decomposition
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Proper Orthogonal Decomposition (POD) is a cornerstone reduced-order modeling technique for accelerating the solution of partial differential equations (PDEs) by extracting energy-optimal orthogonal bases. However, POD's inherent linear assumption limits its expressive power for complex nonlinear dynamics, and its snapshot-based fixed bases generalize poorly to unseen scenarios. Meanwhile, emerging deep learning solvers have explored integrating decomposition architectures, yet their purely data-driven nature lacks essential physical priors and leads to modal collapse, where decomposed modes lose discriminative power.
To address these challenges, we revisit POD from an information-theoretic perspective. We theoretically establish that POD's classical energy-maximization criterion is, in essence, a principle of maximizing mutual information. Guided by this insight, we propose OrthoSolver, a neural POD framework that generalizes this core information-theoretic principle to the nonlinear domain. OrthoSolver iteratively and adaptively extracts a set of compact and expressive nonlinear basis modes by directly maximizing their mutual information with the data field. Furthermore, an orthogonality regularization is imposed to preserve the diversity of the learned modes and effectively mitigate mode collapse. Extensive experiments on seven PDE benchmarks demonstrate that OrthoSolver consistently outperforms state-of-the-art deep learning baselines.

Published as a conference paper at ICLR 2026
ORTHOSOLVER: A NEURAL PROPER ORTHOGONAL DECOMPOSITION SOLVER FOR PDES
Ying Pang 134, JJinyuun ; Ji! Fanhao Mu
SKLCCSE, School of Computer Science and Engineering, Beihang University, Beijing, China
School of Economics and Management, Beihang University, Beijing. China
MOE Engineering Research Center of Advanced Computer Application Technology,
Beihang University, Beijing, China
*National Superior College for Engineers, Beihang University, Beijing, China {pangy, jywang, SanhaccuJ/bbua, edu. jia.iiiahaaaeee jiahaoji03e aaahaojiil@gmail.commm com
ABSTRACT
Proper Orthogonal Decomposition (POD) is a cornerstone reduced-order model- ing technique for accelerating the solution of partial differential equations (PDEs) by extracting energy orthogonal bases. However, POD's 's inherent linear assumption limits expressive power for complex nonlinear dynamics, and its snapshot-based fixed bases generalize poorly to unseen scenarios. Mcanwhile, emerging deep learning solvers have explored integrating decomposition architec- tures, yet their purely data-driven nature lacks essential physical priors and leads to modal collapse, where decomposed modes lose discriminative power. To ad- dress these challenges, we revisit POD from an information theooreti perspective. We theoretically establish that POD's classical energy moergy-maimmization criterion is, in essence, principle of maximizing mutual information. Guided by this insight, we propose OrthoSolver, a neural proper orthogonal decomposition framework that generalizes this core information-theortti principle the nonlinear domain. OrthoSolver iteratively and adaptively extracts a set of compact and expressive nonlinear basis modes by directly maximizing their mutual information with the data field. Furthermore, an orthogonality regularization is imposed to preserve the diversity of the learned modes and effectively mitigate mode collapse. Extensive experiments on seven PDE benchmarks demonstrate that OrthoSolver consistently outperforms state-of-the-arr deep learning baselines.
INTRODUCTION
Partial Differential Equations (PDEs) constitute the fundamental language describing physical laws across numerous scientific and engineering disciplines (Wazwaz) 2002). However, high-fidelity merical simulations of complex, real-world systems are often computationally prohibitive (Rozza) al.] 2022). To address this issue, decomposition has emerged ing problem-solving by breaking down complex problems into series of simpler, more tractable sub-tasks. This paradigm central to the evolution of PDE-solving methodologies. Traditional numerical solvers, for instance, leverage Model Order Reduction (MOR) techniques, such as Proper Orthogonal Decomposition (POD), for acceleration (Carere et al., (02T). In parallel, data-driven models, spearheaded by deep learning, are evolving from monolithic architectures toward decom- positional, multi-scale, and factorized structures better model complex physical processes (Bhat-) tacharya et al. 2021)
powerful paradigm, simplify-
Within the domain of traditional MOR, Proper Orthogonal Decomposition (POD) is a cornerstone technique (Bright al. 2013). Its core principle involves projecting high-dimensional dynami- cal system onto a low-dimensional subspace spanned by a set of energy-optimal orthogonal basis functions, thereby achieving significant computational acceleration. Despite its widespread adop- tion, predicated on its mathematical optimality under linear assumptions, the efficacy of POD is
"Corresponding author

---

Published as a conference paper at ICLR 2026
fundamentally constrained (Demo al. 2023 First, it exhibits limited generalization, basis functions generated for specific operating conditions often fail to extend to new scenarios. Second, POD's strict linear assumption precludes it from capturing the complex dynamics of highly non- linear systems. For example, when addressing multi-phy phitipphyscss problems, POD typically decomposes each variable independently assumes linear correlation between them, thereby failing to capture the underlying nonlinear physical couplings (Lario et al. 2022).
Although deep learning-based PDE solvers have advanced rapidly, they face distinct bottlenecks. Monolithic operator architectures, such as the Fourier Neural Operator (FNO) (Li et al. 202T) and DeepONet (Lu al., 202I). have limited modeling capacity when confronted with complex scenar- ios (Wu et 2023). In response, subsequent work has explored more sophisticated architectures; U-Net-like (Ronneherger et al., 2015) models including LSS (Wu et al. 2023), U-NO (Rahman et al., 2022). and U-FNO (Wen et al. 2022 employ multi-scale structures, whereas models like Transolver (Wu et al. 2024), Factformer (Li et al., 2023a), and F-FNO (Tran al., 2023) utilize factorization and data slicing to simplify the problem. While these decompositional strategies like Transolver provide a powerful means of simplifying complexity, these approaches often lack ef- fective mechanisms to enforee independence among the decomposed components, leading to mode collapse in complex scenarios a phenomenon wherein the decomposed modes become indistin- guishable, thus losing their differential representation power (Luo et al. (025). Consequently, critical challenge has emerged: designing a unified framework that combines efficient decomposi- tion with robust nonlinear modeling while simultaneously preventing mode collapse.
The crux of this challenge stems from a fundamental dichotomy: while traditional decomposition methods like POD rest upon solid mathematical foundations, their reliance on variance-based met- rics introduces significant errors when applied to nonlinear physical systems. Conversely, the de- composition strategies employed in existing data-driven models often lack a sound theoretical un- derpinning. To address this gap, this work revisits POD from an information -teematton--thorttii perspective, theoretically establishing that its core principle is equivalent to the maxinization of Mutual Infor- mation (MI) under FinearGGaussiar assumption (Chechik et al., 2003 Burges et al. (010). This theoretical link not only clucidates the foundations of POD but also underscores the limitations of cariance-based approaches in nonlinear regimes. Consequently, MI, as universal measure of sta- tistical dependence unconstrained by linearity, provides a more principled framework for capturing the intricate correlations characteristic of nonlinear systems (Globerson & Tishby 2003).
This insight motivates the proposed framework, OrthoSolver: a deep proper orthogonal decomposi- tion framework guided by the principle of maximizing mutual information. OrthoSolver comprises two key modules: a orthogonal basis extraction module predicated on the maximum mutual in- formation principle and dynamics evolution module inspired by POD). The former leverages an information-theon thhorretic objective to extract salient modes from the nonlinear dynamics, while the lat- ter performs efficient evolution within the resulting low-dimensional space. To mitigate mode col- lapse, an orthogonality regularization is introduced to enforce the independence and representational efficacy of the decomposed modes. The contributions of this paper are as follows:
Theoretical Contribution: We reveal and formally establish a deep theoretical connection between POD and mutual information maximization, proving that POD's energy-optimal orthogonal basis decomposition is a special case of maximum mutual information under the linear Gaussian assumption.
Methodological Contribution: We propose OrthoSolver, an end-to-end, information- theoretic-guider deep learning framework for orthogonal decomposition that extends the physical principles of POD to the nonlinear domain.
Extensive experiments on seven benchmark datasets, spanning a diverse range of physi- cal phenomena across both 1D and 2D PDEs, demonstrate that OrthoSolver significantly outperforms existing state-of-the-ane art methods.
2 RELATED WORK
Neural Operators for PDEs. Learning solution operators for entire families of PDEs is a central theme in scientific machine learning. Dominant approaches include DeepONet (Lu et al. 2021) and the Fourier Neural Operator (FNO) (Li et al. 2021), which parameterizes global convolutions
2

---

Published as a conference paper at ICLR 2026
efficiently using the FFT. The success of FNO has inspired range of extensions and hybrid archi- tectures. To better capture features at different resolutions, several works have combined FNO with the hierarchical U-Net (Ronneberger et al. (213)) architecture, leading to models like U-FNO (Wen) et a 2022) and U-NO (Rahman et al. 2022). Other extensions have adapted the core idea to handle complex geometries (Geo-FNO (Li al. (2023b) or leverage alternative transforms like wavelets (MWT (Gupta et al., (021)). Concurrently, Transformers (Vaswani et al. 2017). a cornerstone of deep learning, have been successfully adapted for solving PDEs, Models like FactFormer (Li al. 2023a) leverage low-rank structures to boost efficiency, while others like OFormer (Li et al.) 2022) 2023) address the quadratic complexity of attention by incorporating lin- ear Transformer variants. However, these monolithic operator learning approaches often struggle to generalize complex physical scenarios, facing challenges with intricate boundary conditions and capturing the full spectrum of physical dynamics (Wu et al.] (023). Our work instead adopts a decompositional paradigm, aiming for greater interpretability and robustness by breaking down complex fields into simpler, fundamental components.
and GNOT (Hao et al.
Decomposition -based Models for PDEs. An alternative paradigm focuses on decomposing the so- lution field. One line of work employs purely data-driven decompositions; for instance, LSM (Wul et al.] 2023) maps solutions to a latent spectral basis, while Transolver (Wu et al.. (2024) decomposes the input into learnable slices. A key challenge for these methods is the lack of physical grounding. which can lead to mode collapse, where learned basis functions become redundant (Luo et al. 2025)
Another line of work leverages traditional physics-based decomposition, such as Proper Orthogo- nal Decomposition (POD), and then uses a neural network to evolve the mode coefficients (c.g., POD-DeepONa (Lu al. 2022)). This hybrid approach, however, is fundamentally constrained by the linear decomposition error inherent in POD when applied to nonlinear systems. While some methods (Lario et al., (022) attempt to learn this residual error with machine learning, they do not address the core limitation of the initial linear decomposition. Notably, several theoretical works have explored the connection between Proper Orthogonal Decomposition (POD) and mutual infor- mation from information-theoreti perspective (Chechik et al.) 2003 Globerson & Tishby [2003] Burges et al.] 2010). However, these studies remain confined to proving theoretical equivalences and do not attempt to leverage this insight to generalize POD. In contrast, our work builds directly this theoretical foundation and presents the first framework that extends POD to the nonlinear regime. OrthoSolver achieves this by learning a nonlinear decomposition guided by physically principled objective, thereby simultancously circumventing mode collapse and the linear approximation error inherent in classical POD.
Information Theory in Deep Learning. The use of Mutual Information (MI) as an objective has proven be a powerful tool for learning structured and disentangled representations in deep learn- ing. Foundational concepts such as the InfoMax principle (Veličković et al. 2018) and the Informa- tion Bottleneck principle (Tishyy et al., 2000) have been widely applied in representation learning to distill salient features. While direct MI computation is intractable, this challenge has spurred the development of a rich literature on neural MI estimators that provide tractable bounds, includ- ing MINE (Belghazi et al. 2018). InfoNCE (Oord et al. 2018). and CLUB (Cheng et al.) 2020). among others. OrthoSolver innovatively bridges the gap between this line of research and scientific computing. To the best of our knowledge, this is the first framework to leverage these modern MI estimators to guide the decomposition of PDE solutions.
PRELIMINARIES
3.1 PROBLEM SETUP: OPERATOR LEARNING
In classical machine learning, the goal is to learn a mapping between finiteddimensional Euclidean spaces, f: IR - Rdout, using a dataset of input-output pairs P = yx), While effective for many tasks, this vector-to-vector paradigm is not naturally suited for problems governed by PDEs, where the goal is to learn a mapping between functions.
Consider a parametric PDE of the form (z) f(x), where Ca is differential operator parameterized by a function a(z) (encoding boundary conditions, coefficients, or source terms). For each az)), there exists a solution u(x). Learning across this PDE family amounts approximating the solution operator that maps input functions a to output functions u. This motivates the operator- learning paradigm, which seeks to learn mappings between infinite-dimensiona function spaces.
3

---

Published as a conference paper at ICLR 2026
Operator Learning (Anandkumar et al.
2020)
be defined as:
F x + 3/-
(I)
where X = {ar x : S + ] and y {y y: k S2 [2 } denote input and output function spaces over domain C. R2 dz and dy are channel dimensions, and denotes the parameters.
3.2 PROPER ORTHOGONAL. DECOMPOSITION (POD)
Proper Orthogonal Decomposition (POD) (Rozza al. 2022) is classical model reduction tech- nique that formulates an explicit variance-man maxiimiaamaiiiiiiiionn problem (see Appendix [CI for deriva- tion). Its goal is to identify low-dimensional subspace that captures much data variability ("en- ergy") as possible from the snapshots {u:} Treating each snapshot as a realization of a random variable E IR"" POD formally constructs the orthonormal basis (Øx 1 by iteratively solving the following sequential optimization problem:
arg Var((,,
1+ a (6-
where the objective explicitly seeks directions that maximize the projected variance subject to thogonality constraints.
Practically, this optimal basis is obtained via a linear matrix factorization, typically Singular Value Decomposition (SVD) of the snapshot matrix U [uI...., ux| Any snapshot can then be approx- imated as linear combination of these basis modes:
U4 R3
where as = (m;, .be) is the projection coefficient. By maximizing variance, POD finds the optimal linear subspace. However, its reliance linear factorization limits its capacity to capture complex nonlinear dynamics, motivating data-driven generalizations.
3.3 BEYOND LINEAR CORRELATION: MUTUAL INFORMATION
While variance maximization, used in POD, is sensitive linear correlations, it does not capture more complex statistical structures. In contrast, Mutual Information (MI) from information theory provides general measure of dependency (Globerson & Tishby| 2003). Defined as / (X;Y) ( (X))). Y. MI quantifies the reduction in uncertainty about one variable given knowledge of another. Crucially, its formulation allows it to capture arbitrary nonlinear relationships, making more comprehensive tool for dependency analysis than linear correlation.
4 METHODOLOGY: THE ORTHOSOLVER FRAMEWORK
4.1 FROM VARIANCE TO MUTUAL INFORMATION: PRINCIPLED GENERALIZATION
As Section 3.2 mentioned, the fundamental limitation of Proper Orthogonal Decomposition (POD) its reliance on maximizing projected variance as equation 2 principle optimal for linear systems but ill-suited for complex, nonlinear dynamics. To overcome this, we re-contextualize POD from an information--heoreiic perspective, positing that its variance-baan -baed objective is a constrained spe- cial case of more general principle maximizing mutual information (MT). Existing theoretical works have formally proven that maximizing variance is equivalent to maximizing MI under linear Gaussian assumptions (Chechik al., 2003 Globerson & Tishby 2003 Burges al. 2010):
Theorem 1. Assume that the data snapshots u follow a multivariate Gaussian distribution and that the projection a = (u, is linear operation. Then, maximizing the projection variance Var(a) is equivalent to maximizing the mutual information I [1u a) between the original data and its projection coefficient.
Proof. Since u is Gaussian, its linear projection a is also a univariate Gaussian random variable. The differential entropy of zero-mean Gaussian variable is given by
((a) = log(2ne Var(a)).
(4)
2
4

---

Published as a conference paper at ICLR 2026
Basis-Coeff Block
Input Information
B.E.
9
X1
Sol. Op.
Dato
(g(
X
Prediction
X2
82
y
Basis-Coeff Block
Sol. Op.
Information
N2
42(%)
Xs+
Flow
Y
Pxc
Ground Truth
Basis-Coeff Block
Sol. Op.
x
1 DK
Xx41)
(AL)
(An d2 du)
Reconstruction Loss
Decomposition
Arrccon
0.000
forreoo |G G IlB
Orthogonality Loss
Final Joint Loss:
Lymi Lortho + Lrrcom Epred
Figure I:
The input function u(u is first decomposed into global basis functions and coefficients, which are then evolved in the latent space by the solver and recombined to produce (p²)..
Overall architecture of OrthoSolver.
Because the logarithm is a monotonic function, maximizing the variance Var(a) is equivalent to maximizing the entropy (a):
arg max Var(a) = arg max H(a).
(5)
The mutual information between u and is defined as I(u; a) = H (aa) Halu). Since a is deterministic function of u, its conditional entropy vanishes, i.e., H(a uu)lu 0. Therefore, I(u;a) H (a). Combining these results yields:
arg max Var(a)
> arg max H(a) = arg max I(u;
(6)
This proves that under linear Gaussian conditions, the variance-maximizatiin objective of POD equivalent to anttaal-ifformaiio--aiiiiiiiiiii
objective.
Existing theoretical studies have proven Theorem that POD's variance maximization is equiva- lent mutual information maximization under linear Gaussian assumptions. However, these works remain largely theoretical and have not leveraged this insight to practically generalize POD. We observe that Theorem 1 reveals key insight: POD's use of variance identify dominant bases is essentially a specific instance of using mutual information. However, the variance maximization criterion is fundamentally limited to capturing second-order moments, making ill-suited for char- acterizing the complex, high-order dependency structures in highly nonlinear PDEs. By contrast, mutual information can effectively capture complex nonlinearities. Building on Theorem 1, we gen- eralize the linear decomposition paradigm of POD from the restricted variance metric to the general mutual information metric. This leads to the development OrthoSolver, which adaptively extracts the most informative bases in nonlinear spaces, thereby extending the philosophy of POD to nonlinear decomposition.
4.2 ORTHOSOLVER: MODEL ARCHITECTURE
We generalize the classical POD framework an operator-learinn setting by leveraging coommation-theraee principles. Unlike traditional POD, which relies on a fixed linear basis ob- tained from snapshot matrices, OrthoSolver adaptively learns a nonlinear basis that maximizes mu- tual information with the data, enabling more expressive and data-efficient representations.
5

---

Published as a conference paper at ICLR 2026
Neural POD Operator Learning. In contrast to conventional operator learning methods that seek to learn a direct mapping F X' + + (Anandkumar et al. 2020), the proposed framework decomposes
this process into a composition of three distinct operators:
F D Sq o Eg
where denotes operator composition. The Basis Decomposition Operator &s X (#, R²)) maps an input function æ(H) to a set of K global basis functions ( and corresponding co- efficients {ax(n)) The Solver Operator Se evolves these coefficients to parameter ditions μr. Finally, the Synthesis Operator 'D reconstructs the solution by linear superposition: (u')) >m (
Adaptive Basis Learning. Guided by the principle established in Section 4.1 our decomposition operator Ea operationalizes the generalization of POD. Instead of maximizing variance, its core objective is to learn a set of basis functions
{ that maximize the mutual information (MI):
arg max (Proj(u, { ()))):
{
where Proj(.) represents the projection of the data onto the learned basis. This anformation-theorelr objective drives the learning of compact and expressive basis capable of capturing complex non- linear dependencies. To implement this, we design Eg residual-based sequential process, where each step extracts the single most informative basis-coefficient pair (#x; as (14)) from the current data field.
Data Flow. Figure illustrates the data flow: (i) decomposition onto global basis, (ii) coefficient cvolution a low-dimensional latent space, and (iii) synthesis into the final solution. This modular factorization provides interpretability and allows efficient generalization to new parameter regimes.
4.3
BASIS DECOMPOSITION MODULE VIA MUTUAL. INFORMATION MAXIMIZATION
Our basis decomposition module implements the rnformatiio-thhoorete principle outlined in Scction It mimics the sequential, residual-basee process of POD but replaces the linear, variance-driven objective with a non-linear, information-driven one. The module iteratively extracts basis-coefficient pairs to greedily maximize the information captured from the data field.
4.1
Formally, the input function u(fx) is defined as the initial residual Xi. At each step k. this module produce basis-coefficient pair (Ixx, ax) by solving
max I I(Xx; (k));
$2
where 7(:,)) denotes the mutual information between the residual and the extracted coefficient. Intuitively, this ensures that 4 represents the most informative mode contained in X The Basis BasisExtractor is realized as Factorized Fourier Neural Operators (F-FNO) (Tran et al. 2023 selected for its efficacy in operator learning tasks within function spaces, while the ractoo a Multi-Layer Perceptron (MLP):
4 k FNO (Xe; 0mo,k))
(10)
Q₂ = MLP(X, mm...))
(11)
The residual is then updated as
X X 0g k:
(12)
and the process is repeated for K steps.
For practical optimization, direct maximization of 7(X,, (x) is computationally challenging. We reformulate the objective as minimizing the mutual information between the current representation and the residual after extracting ag., i.c.,
min 7(X,, X, ag T.)) min 7(X,, X
(13)
where X denotes the residual representation. As demonstrated in Appendix C.2 maximizing the information captured by the mode is equivalent to minimizing the information carried over in the residual. The final MI-based loss is formulated as the average over all steps:
K
1
Lyi =
I(X X
(14)
K
1=

---

Published as a conference paper at ICLR 2026
MI Estimation. Direct computation of mutual information is generally intractable in deep learn- ing, as it requires access to the true joint and marginal distributions. To address this, we adopt variational approach that optimizes tractable upper bound of the MI. Specifically,
employ the Contrastive Log-ratio Upper Bound (CLUB) (Cheng et al.) 2020), which provides a tight and efficiently trainable surrogate. For given pair (Xx; ox), CLUB estimates the MI by learning variational distribution ((4x(Xs) to approximate the true posterior p(ax (xx). In practice, this con- ditional distribution can be parameterized by a neural network, such as multi-layer perceptron (MLP). Then the upper bound can be caculated by:
TCLUB (XL 0x) (xx)). [lg ((a) (xx(X)) (p(ax) [log q(ax [x)]
(15)
This estimator can be efficiently optimized using samples from the training batch. This tractable estimate is then substituted into Equation equation 14 to facilitate end-to-end training.
Reconstruction Constraint. To ensure the learned basis functions and coefficients can accurately reconstruct the original input data, reconstruction constraint is imposed. This loss penalizes the discrepancy between the original function and its reconstruction from the full set of extracted modes:
K
Lrecon
ag (u)
(16)
Basis Orthogonality Constraint. As noted in (Luo et al. 2025 Doimo et al. 2022). in complex dataset scenarios, deep learning models employing decomposition strategies such (Wu et al. 2024) may suffer from mode collapse, a phenomenon where optimizers tend to converge towards the redundant features rather than decoupling truly independent components. Mathematically. this phenomenon manifests as high similarity between the learned basis vectors (i.e., Φ; R Φ5), leading approximate linear dependence among the column vectors of the basis matrix G. This results in a decrease in the effective rank of G (i.e., rank G) < capacity of the subspace.
K), thereby limiting the representational
To theoretically strictly avoid this degeneracy and promote basis diversity, we introduce an orthogo- nality constraint. By regularizing the Gram matrix of the basis functions to approximate the identity matrix (i.e., GT G R I), we theoretically ensure the linear independence of the basis vectors, thereby maintaining the full-rank property of the decomposition (Gan = K). The loss function is de- fined using the Frobenius norm as follows:
Cotbo ||G
(17)
where G = | 22. Øx] is the matrix consisting of flattened basis vectors columns, and I is the identity matrix.
4.4 DYNAMICS EVOLUTION AND SOLUTION SYNTHESIS
Once the decomposition module has extracted the global basis functions {Φt} and their corre- sponding coefficients {ax (H)).. }. the Solver Operator Sn predicts the system's 's evolution in the low- dimensional latent space.
For each mode k a dedicated E-FNO-based So ut onOpe evolves the coefficient as (f4) new parameter state μ². The operator takes the coefficient history and the static basis function as input to predict the new coefficient ây (p')):
(x(+z') FNOx, (Concat (a)(H), (x))
(18)
Once the set of predicted coefficients {åx (4') has been inferred for all modes, the Synthesis Operator 'D assembles the final digh-dimensional solution. This is parameter-free free linear combi- nation of the global basis functions weighted by their newly predicted coefficients:
K
Y Σ 0z
()()')
(19)
To train the solver networks, we define a prediction loss that measures the discrepancy between the synthesized solution and the ground truth. We the Relative L.2 error, which is common for evaluating physics-based learning problems:
(Y()) - YH4')l2
Speed=
(20)
(Y4))l2
7

---

Published as a conference paper at ICLR 2026
4.5 MODEL TRAINING
The OrthoSolver framework is trained end-to-end by minimizing a composite objective function that incorporates the four distinct loss terms derived in the previous sections. The overall loss, Limat, is dynamically weighted sum of these components, designed to balance the multifaceted goals of our model.
Total Loss Function. The total loss is composed of four terms, each targeting a specific aspect of the learning process. The Mutual Information loss (CMi) from Eq. equation[4 14iviees the extraction of informative basis functions. The Reconstruction loss (Creco) from Eq. equation [6] ensures the decomposition is faithful to the original data. The Orthogonality loss (Corth) from Eq. equation T7 encourages diversity among the basis functions. Finally, the Prediction loss (Ppred) from Eq. equa- tion 20 trains the solver to accurately evolve the latent coefficients. The combined objective is:
Livial AMLLMI + Frecon + Aunhooontho Apred
(21)
where À are the weights for each loss component. To balance these multi-task objectives, Dynamic Weight Averaging (DWA) is employed (Liu et al. 2019).
5 EXPERIMENTS
To comprehensively evaluate the performance and robustness of our proposed OrthoSolver frame- work, we conduct extensive experiments on a diverse suite of benchmark datasets from the field of fluid dynamics. These datasets span both ID and 2D problems and cover range of physical phe- nomena and complexities. Reproducibility details like code and datasets can be obtained in section Reproducibility Statement] A comprehensive analysis of model efficiency, including parameters, training times, and memory consumption, is detailed in the Appendix D.6
5.1 DATASETS AND BASELINES
Our evaluation is performed on seven benchmark datasets from PDEBench (Takamoto al. 2022 detailed in Appendix [.I] They include canonical problems such as Burgers' equation and Ad- vection equation, as well as more challenging simulations like time-dependent Navier-Stokes flow, allowing us to test the model's ability to handle varying levels of non-linearity and dimensionality.
We benchmark OrthoSolver against a comprehensive suite of ten state-of-the-art methods, rep- resenting the primary families of neural operator learning. These baselines include: Fourier- based models: the foundational FNO (Li al.) 2021) and its variants like F-FNO (Tran et al. 2023) and Wavelet-based model MWT (Gupta et al., (021)). Transformer-based architectures: which leverage attention mechanisms, including GNOT (Hao al., (023), Factformer (Li et al.] 2023a), UPT (Alkin et al., 2024), Erwin (Zhdanov et al. 2025). Multi-scale and hybrid architec- tures: which employ hicrarchical structures, such as the classic U-Net (Ronneberger et al. 2015).
and its operator-learning extensions U-FNO (Wen et al.) 2022) and U-NO (Rahman et al.
2022
Decomposition- models: utilize the decomposition idea, represented by the LSM (Wu et al.
2023) and Transolver (Wu et al. 2024) and Transolver++
(Luo et al. 2025).
5.2
IMPLEMENTATION DETAILS
The model was implemented in PyTorch and trained on a single NVIDIA 3090 GPU. For all OrthoSolver models, we set the number of decomposed modes from K [1, 6). The BasisExt ract or and rameter for Dynamic Weight Averaging (DWA) was set Adam optimizer with an initial learning rate of le - 3. The training duration was 500 epochs for 1D datasets and 200 epochs for 2D datasets. The Relative 1.2 error (Eq. 20 serves as the primary evaluation metric.
both utilize 1 layer F-FNO. The temperature pa-
1.0. Models were trained using the
5.3
MAIN RESULTS
Table I presents the primary quantitative comparison of OrthoSolver against a comprehensive suite of baseline methods. The results unequivocally demonstrate that our proposed framework achieves
8

---

Published as a conference paper at ICLR 2026
Table 1: Overall comparison of Relative 1.2 across the seven benchmark datasets. Best and
second- best results are in bold and underlined, respectively.
1D Datasets
2D Datasets
Model
Advection
Burgers
NS
DiftSorp
DiffReac
NS
DiffReac
FNO (Li et al.
2021
0.0051
0.0166
0.0168
0.0014
0.0038
0.0168
0.0884
F-FNO Tran al.
2023
0.0038
0.0920
0.0399
0.001911
0.0429
0.0091
0.0584
MWT Cupta al.
2021
0,5823
0.5403
0.1702
0.027137
0.0132
0.0861
0.6015
GNOT (Hao ct al. 20231
0,9999
0.9999
0.4801
0.1634
0.0830
0.9017
0.9961
Factformer (I. ct
202.3a
0.0076
0.0849
0.0971
0.0050
0.0062
0.0305
0. 1040
Erwin (Zhdanov al.
2025
0.0054
0.0923
0.0507
0.001703
0.0046
0.0155
0.0189
UPT Alkin
2024)
0.0085
0.2352
0.0861
0.002361
0.0053
0.0245
0.1573
U-Ner (Konneberger
2015
0.0247
0.0570
0.0936
0.0013
0.0016
0.0341
0.1261
U-FNO (Wnn el al. 2022)
0.0060
0.0192
0.0221
0.002224
0.0023
0.0130
0.0313
U-NO (Rahman et al.
2022
0.0240
0.0932
0.3626
1.344320
0.9792
0.0449
0.1261
LSM (Wi at.
2025
0.0271
0,4188
0.3025
0,001445
0.0011
0.0370
0.0817
Transolver [u
2024
0.0036
0.0973
0.0335
0.001380
0.0012
0.0282
0.1662
Transolver++ [Luo al. 2025
0.0077
0.2892
0.1137
0.001678
0.0026
0.0197
0.1363
OrthoSolver(Ours))
0.0033
0.0150
0.0157
0.001372
0.0008
0.0055
0.0172
new state-of-the-art performance across all seven benchmark datasets, often by a significant mar- gin. The superiority of our method is particularly pronounced in the complex, multi-dimensional 2D scenarios. For instance, on the 2D Navier-Stokes and 2D Diffusion-Reaction benchmarks, Or- thoSolver reduces the prediction error by over 39% and 45% respectively. compared to the next-best performing methods. Furthermore, OrthoSolver consistently secures the top rank on all five ID datasets, showcasing robustness and versatility across diverse physical systems. This consistent, state-of-the-aa performance validates the core principles of our framework: by replacing the linear assumptions of classical decomposition with a non-linear, information-thcorrticc objective, Ortho- Solver effectively identifies more compact and expressive basis, leading to superior accuracy.
5.4 ABLATION AND ANALYSIS
Table 2: Ablation studies on loss components and parameter sensitivity analysis of the number of modes (K). We report the Relative L. error across all benchmarks. The full model uses K 4.
Ablation Design
Ady
Burgers
1D-NS
DiffSorp
1D-Reac
2D-NS
2D-Reac
MI Obj (Cau)
0.0045
0.0216
0.0334
0.001530
0.0013
0.0109
0.0262
w/o
Recon (Cason)
0.0046
0.0181
0.0229
0.001474
0.0011
0.0079
0.0233
Ortho (Cante)
0.0053
0.0186
0.0494
0,001413
0.0011
0.0159
0.0238
K=1
0.0117
0.0638
0.0932
0.003000
0.0040
0.0335
0.0237
K=2
0,0065
0,0321
0,0319
0,001608
0.0022
0.0087
0.0241
Modes
K=3
0.0037
0.0178
0.0208
0,001527
0.0018
0.0076
0.0247
K=5
0.0046
0.0162
0.0269
0.0013
0.0009
0.0069
0.0229
K=6
0.0050
0.0235
0.0305
0.001554
0.0012
0.0170
0.0197
OrthoSolver (K=4)
0.0033
0.0150
0.0157
0.001372
0.0008
0.0055
0.0172
Ablation results are summarized in Table Removing any of the three auxiliary loss constraints leads to significant degradation in performance: eliminating the orthogonality constraint (Conto) causes an average drop of 35.43%, removing the Mutual Information objective (LM)) results 34.71% % drop, and removing the reconstruction constraint (Crecm) leads to a 23.71% % decline. These results demonstrate that every module extended from Theorem plays significant role our nonlinear decomposition framework.
Experimental Results on Mode Collapse. In our baseline experiments, we observed that while Transolver [Wu et al., 2024) ranks second on the relatively simple Advection and DiftSorp datasets, its performance drops significantly on the NS and Burgers equations, which involve more variables

---

Published as a conference paper at ICLR 2026
and higher complexity. To further quantify the linear correlations between modes across differ- ent datasets, we calculated their mode correlation coefficients. Higher coefficients indicate stronger linear correlations, implying greater redundancy among modes. The average correlation coefficients of Advection and Diff datasets are 0.3796 and 0.4729, respectively. In contrast, these values are significantly higher the NS and Burgers datasets, reaching 0.7470 and 0.8 0.8104, respectively. This clearly indicates that in complex datasets, high similarity between modes leads to mode collapse, ultimately resulting in performance degradation.
Table 3: Comparison of correlation coefficients.
Method
Adv
Burgers
ID-NS
DiftSorp
ID-Reac
2D-NS
2D-Reac
w/o Ortho. Constraint
0.6487
0.8071
0.7962
0.7731
0.8760
0.7598
0.8217
OrthoSolver
0.0702
0.0626
0.0894
0.0437
0.0738
0,0533
0.0480
To further demonstrate the effectiveness of our model in addressing mode collapse, we analyzed the change in average correlation coefficients before and after adding orthogonal regularization in Table 3 Experiments show that introducing orthogonality constraints reduces the average linear correlation between bases from 0.7832 to 0.0631, indicating effective suppression of mode collapse.
Sensitivity Analysis of Parameter K. Our study on the number of modes K in Table 2 reveals that performance improves across different datasets as K increases from to However, further increasing the number of modes to K = 6 leads to decline in performance. This suggests that the initial modes extracted by our MI-maximization principle capture the most significant physical information, while subsequent modes contain diminishingly useful information for the prediction task and may even introduce noise. This result proves the effectiveness of our approach in identifying compact yet highly informative basis.
Table 4: Mutual information between initial state Xu and mode functions Ok.
Dataset
MI do)
MI(Xo, 61)
MI Φ2)
MI 0s)
Adv
3.6432
0.2524
0.0242
0,0173
Burgers
0.1693
0.0767
0.0115
0.0050
ID-NS
2.5685
0,7470
0,3063
0.0803
DiffSorp
0.1093
0.0687
0.0429
0.0015
Dc-Rca
0.6522
0,4821
0.4352
0,0821
2D-NS
0.1434
0.0783
0.0249
0.0027
2D-Reac 2.3915
1.1094
0.3091
0.0926
Analysis of Mode Interpretability. To further verify whether our extracted modes align with the principle of capturing the most important components, calculated the mutual information be- tween different modes and the original data. The results are presented in Table Across different datasets, we observe a trend similar to POD: the mutual information between the extracted modes and the original variables gradually decreases as K increases. This proves that our decomposition mechanism, based on mutual information maximization, extracts features of the basis space with the maximum information content at each step.
6 CONCLUSION
This work resolves the dual challenges of linearity in classical POD and mode collapse in deep learning solvers through a novel information-theoreti perspective. We theoretically establish that POD's energy-mane maximizaaiiiiiionnn principle is, in essence, a form of mutual information (MI) maximiza- tion. Building on this fundamental insight, we introduce OrthoSolver, a framework that generalizes this principle to non-linear systems. By iteratively extracting basis functions that maximize MI with the data field while enforcing orthogonality, OrthoSolver learns compact and highly infor- mative basis, effectively mitigating mode collapse. Extensive experiments confirm that method consistently outperforms state-of-the- baselines. This information--hherrtiic reframing not only addresses longstanding challenges but also opens new avenues for developing more principled and physics-aware deep learning models for scientific computing.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
Prof. Jingyuan Wang's ' work was partially supported by the National Natural Science Foundation of China (No. 72242101, 72222022, 72171013), and the Special Fund for Health Development Research of Beijing (2024-2G-0012)), This work is supported by State Key Laboratory of Complex & Critical Software Environment (SKLCCSE).
ETHICS STATEMENT
The authors have read and complied with the ICLR Code of Ethics. This research does not involve human subjects. personally identifiable information, or sensitive data. The datasets used are publicly available benchmarks, and the proposed method is intended for general research purposes. We foresee no direct potential for harm or negative societal impacts from this work.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our results, we provide a comprehensive set of resources.
Source Code. Our full implementation available at URL The repository includes detailed README mn file with instructions for setting up the environment, downloading data, and running training and evaluation scripts. All dependencies are listed the requi rements txt file.
Datasets. Our experiments are conducted on publicly available datasets from PDEBench Takamotto al. 2022) which public available. More details about datasets can be obtained
D.1
Implementation Details. All hyperparameters, model architectures, and experimental settings are detailed D.4] This includes learning rates, and optimizer configurations and model configurations for each experiment.
Theoretical Details. The proof of our theoretical claims are provided in Theorem
C.2
Computing Environment. All experiments were conducted on a server with an NVIDIA RTX 3090
GPU, using Py Torch version 2.3.0, CUDA 11.8, and Python 3.10.
REFERENCES
Benedikt Alkin, Andreas Fürst, Simon Schmid, Lukas Gruber, Markus Holzleitner, and Johannes Brandstetter. Universal physics transformers: A framework for efficiently scaling neural opera- tors. Advances in Neural Information Processing Systems, 37:25152-255 2024.
Anima Anandkumar, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Nikola Kovachki, Zongyi Li, Burigede Liu, and Andrew Stuart. Neural operator: Graph kernel network for partial differ- ential equations. In ICLR 2020 workshop on integration of deep neural models and differential equations, 2020.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In International conference on machine learning, pp. 531-540. PMLR, 2018.
Kaushik Bhattacharya, Bamdad Hosseini, Nikola Kovachki, and Andrew M Stuart. Model reduc- tion and neural networks for parametric pdes. The SMAI journal of computational mathematics, 7:121-157, 2021.
Ido Bright, Guang Lin, and J Nathan Kutz. Compressive sensing based machine learning strategy for characterizing the flow around cylinder with limited pressure measurements. Physics of Fluids, 25(12), 2013.
Christopher JC Burges al. Dimension reduction: A guided tour. Foundations and Trends@ in Machine Learning, 2(4) 2010.

---

Published as a conference paper at ICLR 2026
Giuseppe Carere, Maria Strazzullo, Francesco Ballarin, Gianluigi Rozza, and Rob Stevenson. A weighted pod-reduction approach for parametrized pde-constrained optimal control problems with random inputs and applications to environmental sciences. Computers & Mathematics with Applications, 102:261-276, 2021.
Gal Chechik, Amir Globerson, Naftali Tishby, and Yair Weiss. Information bottleneck for gaussian variables. Advances Neural Information Processing Systems, 16, 2003.
Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, and Lawrence Carin. Club: A contrastive log-ratio upper bound of mutual information. In International conference on machine learning, pp. 1779-1788. PMLR, 2020.
Nicola Demo, Marco Tezzele, and Gianluigi Rozza. A deeponet multi-fidelity approach for residual learning in reduced order modeling. Advanced Modeling and Simulation in Engineering Sciences, 10(1):12, 2023.
Diego Doimo, Aldo Glielmo, Sebastian Goldt, and Alessandro Laio. Redundant representations help generalization in wide neural networks. Advances in Neural Information Processing Systems, 35: 19659-19672, 2022.
Amir Globerson and Naftali Tishby. Sufficient dimensionality reduction. Journal of Machine Learn- ing Research,
2003.
Gaurav Gupta, Xiongye Xiao, and Paul Bogdan. Multiwavelet--ased operator learning for differen- tial equations. Advances in neural information processing systems, 34:24048-24062, 2021.
Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. Gnot: A general neural operator transformer for operator learning. International Conference on Machine Learning, pp. (2556-12569) PMLR, 2023.
Andrea Lario, Romit Maulik, Oliver T Schmidt, Gianluigi Rozza, and Gianmarco Mengaldo. Neural-network learning of spod latent dynamics. Journal of Computational Physics, 468: 111475, 2022.
Zijie Li, Kazem Meidani, and Amir Barati Farimani. Transformer for partial differential equations' operator learning, arXiv preprint arXiv:2205. 230,,621,3.1,,,,,, 2022.
Zijie Li, Dule Shu, and Amir Barati Farimani. Scalable transformer for PDE surrogate modeling. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023a.
Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhat- tacharya, Andrew M. Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021.
Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar, Fourier neural opera- tor with learned deformations for pdes general geometries. J. Mach. Learn. Res., 24:388:1- 388:26, 2023b.
Shikun Liu, Edward Johns, and Andrew Davison. End-to-end multi-task learning with attention. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1871-1880, 2019.
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218-22 229, 2021.
Lu Lu, Xuhui Meng, Shengze Cai, Zhiping Mao, Somdatta Goswami, Zhongqiang Zhang, and George Em Karniadakis. A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Computer Methods in Applied Mechanics and Engineer- ing, 393:1 14778, 2022.
12

---

Published as a conference paper at ICLR 2026
Huakun Luo, Haixu Wu, Hang Zhou, Lanxiang Xing. Yichen Di, Jianmin Wang, and Mingsheng Long. Transolver++: An accurate neural solver for pdes on million-scale geometries. arXiv preprint 20X1::25002.02144, 2025.
Aaron den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding, arXiv preprint arXiv: 2018.
Md Ashiqur Rahman, Zachary E. Ross, and Kamyar Avizzadenesheli. U-NO: u-shaped neural operators. CoRR, abs/2204. 11127, 2022. doi: 10:48550/ARXII.2204.111
Olaf Ronneberger. Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Nassir Navab, Joachim Hornegee, William M. Wells III, and Ale- jandro F. Frangi (eds.), Medical Image Computing and Computer-Assisted Intervention MIC- CAI 2015 18th International Conference Munich, Germany, October 5 9, 2015, Proceedings, Part III, volume 9351 of Lecture Notes in Computer Science, pp. 234-241. Springer, 2015. doi:
Gianluigi Rozza, Giovanni Stabile, and Francesco Ballarin. Advanced reduced order methods and applications in computational fluid dynamics. SIAM, 2022.
Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Daniel MacKinlay, Francesco Alesiani, Dirk Pflügger, and Mathias Niepert. Pdebench: An extensive benchmark for scientific machine learning. Advances in Neural Information Processing Systems, 15:1596-1611, 2022.
Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/0004057, 2000.
Alasdair Tran, Alexander Patrick Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- fion processing systems, 30, 2017.
Petar Veličković, William Fedus, William Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. arXiv preprint 10Xiv:180911344, 2018.
Abdul-Majid Wazwaz. Partial differential equations. CRC Press, 2002.
Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M Benson. U- fno-an an enhanced fourier neural operator-basee based deep-lcarning model for multiphase flow. Ad- vances in Water Resources, 163:104180. 2022.
Haixu Wu, Tengge Hu, Huakun Luo, Jianmin Wang, and Mingsheng Long- Solving high- dimensional pdes with latent spectral models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning. ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 37417-37438. PMLR, 2023.
Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: A fast transformer solver for pdes general geometries. In Forty-first International Conference Machine Learning. ICML 2024, Vienna, Austria, July 2 21-27, 2024, 2024.
Maksim Zhdanov, Max Welling, and Jan- Willleelem van de Meent. Erwin: A tree-based hierarchical transformer for scage-scale physical systems. arXiv preprint 17X11 2025.
13

---

Published as a conference paper at ICLR 2026
A STATEMENT ON THE USE OF LARGE LANGUAGE MODELS (LLMS)
During the preparation of this work, the use of LLMs was confined to the following areas: language polishing and code debugging. For the manuscript, LLMs were employed to improve grammar, fine sentence structure, and enhance the overall clarity of the text. For the software implementation, LLMs served as debugging aid, assisting in the identification of potential errors in code snippets and suggesting structural improvements.
B
NOTATION
Here we summarize the key notations used throughout the paper.
Table 5: The notation in this paper
Symbol
Meaning
Input function space
Oulput function space single data snapshot; a high-dimensional discrete field
u
Parameters of the PDE (e.g., boundary conditions, coefficients)
The predicted function or data snapshot
X
Residual data field at the k-th decomposition step The k-th basis function (a high-dimensional field) The scalar coefficient for the k-th basis function
OR
The predicted coefficient for the k-th basis function The Basis Decomposition Operator (Encoder)
Eg
Sa
The Solver Operator that evolves coefficients
D
The Synthesis Operator that reconstructs the solution
C THEORETICAL FOUNDATIONS
C.I THEORETICAL DERIVATION OF PROPER ORTHOGONAL DECOMPOSITION
Proper Orthogonal Decomposition (POD) aims extract an orthonormal basis that captures as much data variability ("energy") as possible from set of centered snapshots {uy)}ı= where uy E R and -ソ=
M U Treating each snapshot realization of random variable u, POD seeks
sequence of orthonormal basis vectors { that maximize the variance of the projected data in a hierarchical manner:
Φ% arg max
Var( d)).
(22)
|+
(11(01....01111
Since the data is centered, the variance of the projection can be expressed as
M
Var (u, Φ))
Σ [uJ +)
1 M
ug
1 = の co.
(23)
M
where C NxN the sample covariance matrix of the data.
Maximizing o CΦ under the unit-norm constraint leads to the following eigenvalue problem:
Co Ad.
(24)
where Φ is an eigenvector of C and À is its associated eigenvalue. The POD modes are thus given by the eigenvectors of C, sorted in descending order of X, so that Φi captures the maximum variance, Φ the second largest variance, and so on.
Any snapshot can then be approximated as a linear combination of the first modes:
ask = (u,, (2)-
(25)
14

---

Published as a conference paper at ICLR 2026
Thus, POD provides optimal linear subspace (in the sense of maximum variance) for representing the data.
C.2 PROOFS OF INVERSION OF MI OBJECTIVE
In our framework, we aim sequentially extract the most informative basis-coefficient pair (P): (g)) from residual field Xg. The ideal objective is to maximize the Mutual Information (MI) that the extracted coefficient ag shares with the field Xg. as this ensures the extracted mode maximally informative. This objective is written as:
max / (Xx,, as)
However, direct optimization of this term can be challenging. Instead, we use surrogate objective: minimizing the MI between the current field Xg and the subsequent residual field Xx+11 We now prove the equivalence these objectives.
The total information content of the field X₁ is its entropy, (Xx).. The decomposition step splits X into the extracted component (represented by a&) and the residual Xx+1- Since X can be perfectly reconstructed from ag. 4s. and Xk+1 (where X241 Xx ab the conditional entropy H (Xxx)ee (ax: XB+1) ) is zero.
The MI between Nş and the pair (ak, X Xk+1) is thus:
I [(Ki ok, X2+1) H ((x)) - H ((aaaan, Xx+1) = H (Xx)
Using the chain rule for mutual information, we can expand this term:
((Ki du Xx+1) ((xx:a)) + I(Xxi Xx+1
Combining these two equations gives:
((xx) I(Xx;as) + I(Xx; Xx+1las)
Since H(Xx) is a constant for given data distribution, maximizing the term I(Xa; is mathe- matically equivalent minimizing the term T(Xx; Nx+1/ag).
Now, we analyze the term that our model minimizes in practice, LMI I(Xe, Xx+1). By defini- tion, / [(x,, XK+1) (X++1) - (X++1 Since X is a deterministic function of XR. the conditional entropy H (Xx+1 [Xx) is zero. Therefore, minimizing the MI between the input and the residual equivalent to minimizing the entropy of the residual itself:
min (Xx,, Xx+1) = min (Xx+1)
The objective min (Xx+11) (making the residual as random/unstructured practical and effective surrogate for the ideal objective min I(X;; Xx41 Intuitively, by ensuring the residual field X contains minimal information (low entropy), we enforce that the maximal amount of salient, structured information from X has been captured the extracted coefficient ag. This justifies the inversion: maximizing the information captured by the mode is achieved by minimizing the information that remains in the residual.
possible) serves
Thus, establish the equivalence:
max I(X 0R) min I(X,, Xk+1)
C.3 MATHEMATICAL ANALYSIS OF MODE COLLAPSE AND ORTHOGONALITY SOLUTION
In this section, we provide a rigorous mathematical explanation of the Mode Collapse phenomenon observed in decomposition based deep learning models and explicitly prove how our proposed or- thogonality constraint theoretically resolves this issue.
C.3.1 THE MATHEMATICAL ESSENCE OF MODE COLLAPSE
Let the learned basis functions be represented by the matrix G = [1) $2...., E R where K is the number of modes and is the feature dimension.
15

---

Published as a conference paper at ICLR 2026
Definition (Mode Collapse). Mode collapse in the context of subspace decomposition is charac- terized by the redundancy of learned features, where a subset of basis vectors converges to highly similar directions. Mathematically, this implies that for distinct indices i o j. Φ; c Ø for some scalar c.
Rank Deficiency. This redundancy leads linear dependence among the column vectors of G. Consequently, the effective rank of the basis matrix decreases:
rank G) < K
(26)
When the rank is deficient, the subspace spanned by G, denoted as span( ( {==)), has dimension strictly less than K. This indicates that the model has wasted computational capacity on redundant features and failed capture the full spectrum of physical dynamics, leading to suboptimal recon- struction and prediction performance.
C.3.2 THEORETICAL GUARANTEE OF THE ORTHOGONAALITY CONSTRAINT
To prevent the rank deficiency described above, introduce the Basis Orthogonality Constraint. We formally prove that minimizing this objective ensures the basis matrix remains full-rank, thereby preventing mode collapse.
Proposition. Minimizing the orthogonality loss Lonho = | I| promotes linear in- dependence among basis vectors { ensuring that G maintains full column rank, i.e., rank(G) K.
Proof. Consider the global minimum of the optimization problem where Lunho = O.
1. The condition Lotha implies that the Gram matrix of the basis functions equals the identity matrix:
G -G=LL Ix
(27)
2. By definition of the identity matrix, for any distinct pair of columns i,
t Φ
i j), and || =
(28)
This indicates that the set of vectors ( Φx is orthonormal.
3. Linear Independence: An orthonormal set of non-zero vectors is linearly independent. Suppose there exist scalars Ci....y CK such that E GΦ: 0. Taking the inner product with any Φj:
Σ Eapi ΦÚ
Σ ()) 1
Cj =
(29)
Since this holds for all j. the vectors are linearly independent.
4. Full Rank Property: Since the K columns of G are linearly independent, the matrix G has full column rank:
rank(G) K
(30)
D EXPERIMENT DETAILS
D.1 BENCHMARKS
Our evaluation is performed seven benchmark datasets from PDEBench (Takamoto et al. 2022 comprehensive suite for scientific machine learning. The selected problems cover a range of canoni- cal and challenging simulations to test our model's ability to handle varying levels of dimensionality and non- loneaiiea
1D Advection The 1D Advection equation models the transport of quantity without deformation or diffusion. is linear, first-order hyperbolic PDE fundamental to fluid dynamics. The governing cquation is:
()) + (t, =
where u(t,x) is the transported quantity and is the constant advection speed.
16

---

Published as a conference paper at ICLR 2026
ID Burgers' Equation The Burgers' equation is non-linear PDE that models fundamental pro- cesses fluid dynamics, including shock formation and wave breaking. It incorporates both non- linear advection and diffusion terms:
z)) + (z v/T ((t
where y is the diffusion coefficient.
ID and 2D Compressible Navier-Stokes (NS) The compressible Navier-Stokes (NS) equations are set of coupled non-linear PDEs that describe the motion viscous, compressible fluids. They are foundational in aerodynamies and gas dynamics, modeling complex phenomena like shock waves.
Oep + V (pv) = pld.v + Vv) = = Vp + + 7/3)VVVV .v)
0c(E + +
[p+++ po v
where is the mass density, v is the fluid velocity, is the gas pressure, is an internal energy described by the equation of state, o' is the viscous stress tensor, and 1/ and are shear and bulk viscosity, respectively. This equation can describe more complex phenomena, such as shock wave formation and propagation. For simplicity, we denote them NS in our tables.
1D Muffusion-Sorption (DiffSorp) This equation models diffusion process that is slowed down by a sorption mechanism, where the retardation factor depends non-li non-linearly on the variable itself. is highly applicable to real-world problems like contaminant transport in groundwater. The govern- ing equation is:
0,u(t, x) D/ D/R(u) O2 u(t,
where the diffusion coefficient and R(u) is the non-linear retardation factor.
ID and 2D -eactionn (DiffReac) This system models the interaction between diffusion processes and local reactions.
ID Cnitusion-Reeaction This equation combines standard diffusion process with non- linear source term that can drive rapid, exponential dynamics. The equation is: 0,u(t, xr)
u(t,uu(t pa(1 u) 0.
This is more complex extension involving two non-linearly cou- pled variables, an activator and an inhibitor, which can produce complex patterns. The sys- tem is modeled by the Fitzhugh-Naguma equations and is applicable to biological pattern formation.
2D Diffusion- Reattion::
Table Details of the seven benchmark datasets selected from PDEBench. NS refers to the com- pressible Navier-Stokes equations.
Dataset
Dimensions
Resolution (Space x Time)
Variables
Samples
1D Advection
ID
1024 x 200
10,000
1D Burgers
1024 200
10,000
1D NS
ID
1024 x 100
3 10,000
ID DiffSorp
ID
1024 100
10,000
ID DiffReac
ID
1024 x 200
I
10,000
2D NS
2D
64 x 21
4
1,000
2D DiffReac
2D
642 100
2
1,000
D.2 METRICS
The primary evaluation metric used across experiments the Relative L2 error, also referred to as the normalized RMSE (nRMSE) in the PDEBench paper. It provides a scale-independent measure of the prediction error. The metric is defined as:
Upred
ueruella
Crel
(31)
17

---

Published as a conference paper at ICLR 2026
where Upred is the predicted solution, Utrue is the ground truth solution, and || ll denotes the n..norm. The Relative L2 error we used is not averaged over time but calculated over the entire spatiotemporal domain.
D.3 DYNAMIC WEIGHT AVERAGING (DWA)
As discussed in Model Training, we employ the Dynamic Weight Averaging (DWA) strategy (Lu al. 2019) balance the contributions of multiple loss components and avoid manual hyperpa- rameter tuning. The total objective function is composed of four terms:
Lotal AMI + Arecon Creon + Xorth Cortho + Aprod
(32)
The DWA algorithm dynamically adjusts the weights A based on the rate of change each loss. For the k-th loss component at training epoch t, the relative loss change rate wx(t) is calculated as:
Lx 1) 1)
(1)(
(33)
L*
where (x-(- and (t- represent the average loss values for task k: in the previous two epochs. The weight (t) is then updated using a softmax normalization with a temperature parameter T:
exp(wx(t)TT)
(a(t) Σj exp(1x
(34)
where j.j E {MI, recon, ortho, pred). This mechanism automatically assigns higher weights to tasks with slower convergence rates to balance the training process.
D.4 IMPLEMENTATION DETAILS
We benchmark OrthoSolver against comprehensive suite of 13 state-of aeffthhe-rtt methods, represent- ing the primary families of neural operator learning. These baselines include:
Spectral-based models:
FNO (Li et al. 2021): : A pioneering method that learns the integral kernel of an op- erator in Fourier space, performing efficient global convolution via the Fast Fourier Transform.
F-FNO (Tan et al. 2023): A parametee--efficient FNO variant that factorizes the multi-dimensional spectral convolution into sequence of one-dimensional transforms enable deeper architectures.
MWT Guupta et al., 2021): An operator learning model that uses multiwavelet trans- forms to create sparse, multi-resolution representation of the integral kernel, ex- celling at capturing localized features.
Transformer-based architectures:
GNOT (Hao et al., 2023): A flexible Transformer architecture designed to han- dle irregular meshes and multiple heterogeneous input functions via a novel linear- complexity attention mechanism.
Factformer (Li al. 2023a): A scalable Transformer for structured grids that factor- izes the high-dimensional attention kernel into product of one-dimensional integrals along each spatial axis.
Erwin (Zhdanov et al. 2025): A hierarchical Transformer that leverages ball tree partitioning to achieve limear-complexity attention for large-scale irregular physical systems, capturing both local detail and global interactions through progressive coars- ening and cross-ball rotation mechanisms.
UPT (Alkin et al.) 2024)): A universal neural operator framework that compresses arbitrary Eulerian or Lagrangian inputs into fixed-size latent space, enabling effi- cient latent-space rollouts and flexible spatio-temporal querying across diverse physi- cal simulation paradigms.
Multi-scale and hybrid architectures:
18

---

implemented Published as a conference paper at ICLR 2026
U-Net (Ronneberger al. 2013): A classic symmetric encoder-decoder architecture that fuses multi-scale features using skip connections to enable precise localization of details.
U-FNO (Wen et al. 2022): A hybrid model that embeds Fourier Neural Operator blocks within U-Net's hierarchical framework to capture both global dynamics and local features.
U-NO (Rahman et al. 2022): A general, memory-efficient meta-architecture that adapts the U-Net's s multi- mestlrresolution structure to accommodate any type of neural op- crator block.
Decomposition- models:
LSM (Wu er al., 2023): A method based on Learned Spectral Methods, which decom- poses the solution into a series of learned spectral functions.
Transolver (Wu et al. 2024 An efficient model that slices the high- sigh-dimensional
spatial domain into lower power-dimesionaa subspaces and applies transformers therein.
Transolver++ (Luo et al., 2025): An accurate and highly parallel neural PDE solver that decomposes million-scale mesh data into adaptive "eidetic physical states" via local-aware slicing and Gumbel reparameterization, achieving linear scalability and state-of-the-are performance on industrial-scut geometries.
To ensure a rigorous and fair comparison, all experiments are conducted within unified framework, sharing the same data loader, training pipeline, and evaluation metrics.
Training Protocol All models are trained using the AdamW optimizer with an initial learning rate of 10-3, We employ a cosine annealing learning rate scheduler to gradually decrease the learning rate throughout the training process. The number of training epochs is set to 500 for 1D datasets and 200 for 2D datasets to account for increased computational cost.
Hyperparameter Settings Baseline Models: All bascline models, including FNO, U-FNO, U- NO, LSM, Transformer, Factformer, GNOT, MWT, F-FNO, and U-Net, are implemented with a unified configuration. Specifically, the hidden dimension (n idden) is set 64, the number of attention heads (n heaadsss is 8, and the number of layers (n_layers) 8. For Transolver, its unique number of slices (slice (elice_nu)) is set 64. For Erwin, use small erwin_conf configgiig.ooo which is mainly used in Experiments in the code repo. All hyperparameter settings were aligned with the original papers closely possible. Furthermore, all experiments within the
framework to ensure consistency.
Our proposed model, OrthoSolver, is implemented with network depth (n.layez ) of 2 and modes fnum.block. All other training strategies and hyperparameters are kept identi- cal to the baselines to ensure a fair comparison, highlighting the parameter efficiency and represen- tational power of our architecture.
Our Model:
D.4.1 COMPUTATIONAL. ENVIRONMENT
All experiments were conducted on single NVIDIA RTX 3090 GPU with 24GB of VRAM. Our implementation based on the Py PyTorrch framework.
D.5
MAIN RESULTS
The comprehensive experimental results, along with the quantified percentage improvements of our model relative to current sfate-of-the-arr (SOTA) baselines, are summarized in Table 7 This comparison clearly highlights the consistent performance gains achieved by our approach across all benchmark datasets.
D.6 MODEL EFFICIENCY
To comprehensively evaluate the computational efficiency of our proposed OrthoSolver framework, Table provides detailed summary of all models in terms parameter count, as well as memory and time consumption during both training and testing phases.
19

---

Published as a conference paper at ICLR 2026
Table 7: Overall comparison of Relative 1.2 across the seven benchmark datasets. Best and
second- best results are in bold and underlined, respectively.
ID Datasets
2D Datasets
Model
Advection
Burgers
NS
DiftSorp
DiffReac
NS
DiffReac
FNO Li et al.
2021
0.0051
0.0166
0.0168
0,0014
0.0038
0.0168
0.0884
F-FNOT Tran
2023
0,0038
0.0920
0.0399
0.001911
0.0429
0.0091
0.0584
MWT Cupta et al.
2021
0.5823
0.5403
0.1702
0.027137
0,0132
0.0861
0.6015
GNOT (Hao ct al. 20233
0.9999
0,9999
0,4801
0,163441
0.0830
0.9017
0.9961
Factformer [L. ct al.
2023a
0.0076
0.0849
0.0971
0.0050
0.0062
0.0305
0.1040
Erwin (Zhdanov et 2025
0.0054
0.0923
0.0507
0.001703
0.0046
0.0155
0.0189
UPT Alkin et al. 2024
0.0085
0.2352
0.0861
0.002361
0.0053
0.0245
0.1573
U-Net (Ronneberger et al.
2015
0.0247
0.0570
0.0936
0.0013 98
0,0016
0.0341
0.1261
U-FNO (Wen et al. 2022
0.0060
0.0192
0.0221
0.002224
0.0023
0.0130
0.0313
U-NO (Rahman el
2022
0,0240
0.0932
0.3626
1.344320
0.9792
0.0449
0.1261
LSM
(Wu al.
2025
0.0271
0.4188
0.3025
0,001445
0.0011
0.0370
0.0817
Transolver Wu
2024
0.0036
0.0973
0.0335
3.001380
0.0012
0.0282
0.1662
Transolver++ [Luo et al. 2025)
0.0077
0.2892
0,1137
0.001678
0.0026
0.0197
0.1363
Orth)))lvee(Ours)
0.0033
0.0150
0.0157
0.001372
0.0008
0.0055
0.0172
Relative Promotion
8.33%
9.64%
6.55%
0.58%
27.27%
39.56%
8.99%
Table & A comprehensive summary of performance and efficiency metrics for all evaluated models,
including dataset-specific ranks.
Training
Testing
Model
Params
Rel-L22 Rank
Mem (MB)
Time (s)
Mem (MB)
Time (s)
FNO 4753412
430.65
720.83
124.65
102.41
(4, 2, 2, 5, 5)
FLFNO
1591044
374.17
1315.18
41.52
140.66
(3, 6, 5,
2,
MWT
27437
118.68
14996.90
23.26
1099.46
(10. 10, 8, 10, 10)
GNOT
1236572
4942.57
4714.25
38.93
445.55
11, 10, 10,
Factformer
451972
1261.89
3036.21
29.75
267.77
(6, 7,,5,,,8,7,,66
U Net
17320388
672.12
1000.88
220.10
141.43
68.4.6.3,4.7,77
ULFNO
39394188
1327.12
2167.64
532.97
270.92
(5.3.3.7.5.3.2)
U-NO
50798104
2393.08
2393.08
987.84
214.15
(7.7. 11.
LSM
19201348
1077.44
2186.17
301.88
221.02
4,2,,,
Transolver
779204
2305.22
2315.93
41.21
240.91
(2,8,4,
OrthoSolver 1113748
1639.52
3727.69
47.77
224.67
(1, I,,
The analysis reveals that OrthoSolver achieves a highly competitive efficiency profile while deliv- ering state-of-the-art predictive accuracy (ranked first on all datasets). Firstly, regarding model size. OrthoSolver, with I.IM parameters, is lightweight model. It is substantially efficient than large-scale architectures such as U-Net (17.3M), LSM (19.2M), and U-FNO (39.4M), making it easier to store and deploy.
parameter-
The advantages of OrthoSolver are particularly pronounced during the inference (testing) phase. Its testing memory footprint (47.77 MB) is remarkably low, significantly outperforming models like FNO, U-Net, and LSM. This indicates strong potential for deployment in resource-constrained environments. Concurrently, its testing time (224.67 s) moderate and practical, ensuring swift predictions and surpassing several baselines including Factformer, U-FNO, and GNOT.
The primary computational trade-off for OrthoSolver lies in the training phase. Its training time and memory consumption are in the mid-to-high range, exceeding those of simpler models like the standard FNO. We posit that this increased training cost is reasonable price for its superior performance, likely attributable to the framework's 's more complex optimization process required to learn the decomposition and reconstruction of an orthogonal basis for the physical fields.
In summary, OrthoSolver strikes an effective balance between state-of-the accuracy and com- putational efficiency. While training is more resource-intensive, yields a highly compact and
20

---

Published as a conference paper at ICLR 2026
efficient model for inference, making it compelling framework for practical applications where deployment performance critical.
D.7 ABLATION AND ANALYSIS
Table 9: Ablation studies loss components and modes. We report the Relative 1.2 emror and the percentage deterioration relative to the full model.
Ablation Design
Adv
Burgers
1D-NS
DiftSorp
1D-Reac
2D-NS
2D-Reac
MI Obj (Lma)
0.0045
0.0216
0.0334
0.001530
0.0013
0,0109
0,0262
w/o
deterionation
27%
31%
53%
10%
38%
50%
34%
Recon (Crecon)
0.0046
0.0181
0,0229
0.001474
0.0011
0,0079
0.0233
28%
17%
7%
27%
30%
26%
Ortho (Corha)
0.0053
0.0186
0.0494
0.0014
0.0011
0.0159
0.0238
38%
19%.
68%
3%
27%
65%
28%
K=1
0.0117
0.0638
0.0932
0.0030u0
0.0040
0,0335
0.0237
K=2
0.0065
0,0321
0.0319
0.0016us
0.0022
0.0087
0,0241
Modes
K=3
0.0037
0.0178
0.0208
0.001527
0.0018
0.0076
0.0247
K=5
0.0046
0.0162
0.0269
0.001398
0.0009
0.0069
0.0229
K=6
0,0050
0.0235
0.0205
0.001504
0.0012
0.0170
0.0197
OrthoSolver (K=4)
0.0033
0.0150
0.0157
0.001372
0.0008
0.0055
0.0172
Table 10: Mutual information between initial state Xo and mode functions Φk.
Dataset
MI(Xo, do)
MI(Xo, Φ1)
MI(Xo, 62)
MI(Xo, da)
Adv
3.6432
0,2524
0.0242
0.0173
Burgers
0.1693
0,0767
0.0115
0,0050
ID-NS
2.5685
0.7470
0.3063
0,0803
DiffSorp
0.1093
0.0687
0.0429
0.0015
ID-Reac
0.6522
0.4821
0.4352
0.0821
2D-NS
0.1434
0.0783
0.0249
0.0027
2D-Reac
2.3915
1.1094
0.3091
0.0926
Table 11: Comparison of correlation coefficients.
Method
Adv
Burgers
ID-NS
DiftSorp
ID-Reac
2D-NS
2D-Reac
w/o Ortho. Constraint
0.6487
0.8071
0.7962
0.7731
0.8760
0.7598
0.8217
OrthoSolvvr
0.0702
0.0626
0.0894
0.0437
0.0738
0.0533
0,0480
21


Reviews:
[{"review_id": "hDvDmkcsTb", "reviewer": "Reviewer_GFif", "summary": "The authors introduce a paradigm shift for POD away from optimal energy-based modes to maximizing mutual information among modes. The work also introduces a novel learning architecture for mode decomposition/training in neural PDEs. Strong empirical performance is reported across several PDE benchmarks.", "questions": "**Q1. Theoretical novelty.** Under linear-Gaussian models, MI maximization over orthonormal projections selects the KLT principal subspace, i.e., POD (see [1]). What is new in the theoretical results beyond restating the equivalence.\n\n**Q2. Ablation study.** I find the ablation results appear to show almost monotonically decreasing MSE for Advection and 1D-NS as the order increases. Here $K=4$ appears to be an outlier. Please consider reporting a full ablation study across the modes, including at least $K=3$ and $K=5$.\n\n**Q3. Mode collapse.** The paper makes claims about resolving mode collapse, but there is no supporting empirical evidence of when mode collapse occurs or that the method presented effectively overcomes mode collapse. I recommend the authors either reduce the claims about mode collapse or provide a supporting study.\n\n[1] Burges, Christopher JC. \"Dimension reduction: A guided tour.\" _Foundations and Trends® in Machine Learning_ 2.4 (2010): 275-365.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- The paper tackles relevant challenges in neural PDEs including principled mode selection and training.\n- The work bridges statistical objectives (MI) and reduced-order modeling, which is of broad interest.\n- The paper presents a novel and well motivated architecture for performing mode decomposition and training in neural PDEs.\n- This architecture achieves SOTA results across several benchmark PDEs.", "weaknesses": "- The theoretical novelty appears limited under the standard Gaussian assumptions. In particular, for Gaussian data MI-optimal linear projections coincide with KLT/PCA, i.e., POD. This is well known (e.g., [1]) in information-theoretic sources.\n- The authors make claims about overcoming mode collapse are made without supporting empirical evidence or sufficient study.\n\n[1] Burges, Christopher JC. \"Dimension reduction: A guided tour.\" Foundations and Trends® in Machine Learning 2.4 (2010): 275-365.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xYtBk86D1m", "reviewer": "Reviewer_wLXC", "summary": "The paper builds a connection between POD and mutual information maximization under linear-Gaussian assumptions. Based on this, the authors proposed a data-driven neural proper orthogonal decomposition solver for PDEs. The mutual information is encoded into the total loss function, which will increase the training difficulty. The authors conduct many numerical experiments and many comparisons with other methods to demonstrate the performance of the proposed method.", "questions": "Refer to Weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Providing a large amount of the empirical results.", "weaknesses": "1. The equivalence between variance and mutual information only holds under the strong linear Gaussian assumption, which is rarely satisfied in nonlinear partial differential equations. This article does not explain why this connection is still meaningful or useful in the highly nonlinear cases it aims to address.\n2. Too many penalty terms have to be introduced into the loss function. Is it very hard to optimize?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8TyCeTcd0n", "reviewer": "Reviewer_e7gJ", "summary": "The paper introduces OrthoSolver, a Neural Operator architecture that combines efficient decomposition with robust nonlinear modeling by exploiting an information-theoretic perspective on dimensionality reduction. The method introduces two key innovations to the neural operator community: an information-theoretic nonlinear approach for identifying reduced modes (enhance expressivity), and an orthogonality regularization term that prevents redundant feature extraction (reduce mode collapse). The approach is both elegant and theoretically sound. The authors validate their method on five different 1D and two different 2D PDEs, and further investigate the key design choices of their framework through an in-depth ablation study.", "questions": "1. Are the modes interpretable? For example, in classical POD, the most energetic modes are those that explain the greatest variance. Do we observe a similar behavior here? It would be helpful to plot the modes in order of energy level to assess their interpretability.\n2. In Table 2, why does the model without the reconstruction constraint seem to perform almost as well as the full model? I would have expected this constraint to be very important, as it measures the discrepancy between the original function and its reconstruction from the full set of extracted modes.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper addresses the problem of combining efficient mode decomposition (to extract relevant features) with robust nonlinear modeling, while simultaneously preventing mode collapse. This problem is relevant as it clearly captures the dichotomy between traditional Reduced Order Model (ROM) techniques, which have solid mathematical foundations but suffer from linearity limitations, and Neural Operator–based data-driven modeling, which offers high nonlinearity but often lacks theoretical soundness. Among the main strengths of the paper, I highlight the following:\n\n1. An original application of mutual information optimization to guide PDE mode decomposition, which could potentially impact other fields such as ROM and manifold learning.  \n2. A clear and well-structured review of the Neural Operator literature.  \n3. The introduction of a Basis Orthogonality Constraint to promote diversity among the learned basis functions.  \n4. A comprehensive experimental section that includes both 1D and 2D PDEs.  \n\nThe derivation presented in Appendix C.2 appears sound, and the experimental details are sufficiently complete to allow for reproduction of the results.", "weaknesses": "My primary concern is robustness and comparison with state-of-the-art models. Given these clarifications in an author response, I would be willing to increase the score. \n\n1. A more in-depth analysis and explanation of the model’s performance (Table 1) are needed. In particular:  \n   - Important models [1, 2, 3] are missing and should be included as benchmark baselines. The field is evolving rapidly and it is important to compare against the best models.\n   - The evaluation metric for Table 1 is not clearly explained — is the L2 error averaged over time?  \n\n2. How were the hyperparameters for both your model and the baselines chosen? In the **Hyperparameter Settings** section, this information is not specified. Are the hyperparameters the same as those used in the PDEBench paper’s baseline results (see Tables 2–9 in [4])? Are the results better than the ones reported by PDEBench (in order to claim SOTA).\n\n3. Can you provide variance results across multiple network initialization seeds for the different problems? This is important to ensure that the results are not cherry-picked for a specific initialization and that the methodology is robust.\n\n\n***References***\n\n[1] Zhdanov, Maksim, Max Welling, and Jan-Willem van de Meent. \"Erwin: A tree-based hierarchical transformer for large-scale physical systems.\" ICML, 2025.\n\n[2] Alkin, B., F  ̈urst, A., Schmid, S., Gruber, L., Holzleitner, M., and Brandstetter, J. Universal physics transformers: A framework for efficiently scaling neural operators. NeurIPS, 2024.\n\n[3] Luo, H., Wu, H., Zhou, H., Xing, L., Di, Y., Wang, J., and Long, M. Transolver++: An accurate neural solver for pdes on million-scale geometries. CoRR, 2025.\n\n[4] Takamoto, Makoto, et al. \"Pdebench: An extensive benchmark for scientific machine learning.\" Advances in Neural Information Processing Systems 35 (2022): 1596-1611.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)