Disentangled Representation Learning for Parametric Partial Differential Equations
Authors: Ning Liu
Lu Zhang
Tian Gao
Yue Yu
Venue: ICLR 2026 Poster (2026)
Keywords: Neural Operators; Physics Discovery; Inverse Problems
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Neural operators (NOs) excel at learning mappings between function spaces, serving as efficient forward solution approximators for PDE-governed systems. However, as black-box solvers, they offer limited insight into the underlying physical mechanism, due to the lack of interpretable representations of the physical parameters that drive the system. To tackle this challenge, we propose a new paradigm for learning disentangled representations from NO parameters, thereby effectively solving an inverse problem. Specifically, we introduce DisentangO, a novel hyper-neural operator architecture designed to unveil and disentangle latent physical factors of variation embedded within the black-box neural operator parameters. At the core of DisentangO is a multi-task NO architecture that distills the varying parameters of the governing PDE through a task-wise adaptive layer, alongside a variational autoencoder that disentangles these variations into identifiable latent factors. By learning these disentangled representations, DisentangO not only enhances physical interpretability but also enables more robust generalization across diverse systems. Empirical evaluations across supervised, semi-supervised, and unsupervised learning contexts show that DisentangO effectively extracts meaningful and interpretable latent features, bridging the gap between predictive performance and physical understanding in neural operator frameworks. Our code and data accompanying this paper are available at \url{https://github.com/ningliu-iga/DisentangO}.

Published as a conference paper at ICLR 2026
DISENTANGLED REPRESENTATION LEARNING FOR PARAMETRIC PARTIAL DIFFERENTIAL EQUATIONS
Ning Liu'& Lu Zhang* Department of Mathematies Lehigh University
Tian Gao IBM Research
Yue Yu
Department of Mathematics
Lehigh University
ABSTRACT
Neural operators (NOs) excel at learning mappings between function spaces, serv- ing as efficient forward solution approximators for PDE-governed systems. How- ever, as black-box solvers, they offer limited insight into the underlying physi- cal mechanism, due to the lack of interpretable representations of the physical parameters that drive the system. To tackle this challenge, we propose new paradigm for learning disentangled representations from NO parameters, thereby effectively solving an inverse problem. Specifically, we introduce DisentangO, a novel hyper-neural operator architecture designed to unveil and disentangle la- tent physical factors of variation embedded within the black-box neural opera- tor parameters. At the core of DisentangO is a multi-task NO architecture that distills the varying parameters of the governing PDE through task-wise adap- tive layer, alongside a variational autoencoder that disentangles these variations into identifiable latent factors. By learning these disentangled representations, DisentangO not only enhances physical interpretability but also enables more robust generalization across diverse systems. Empirical evaluations across su- pervised, semi coni-superiseed, and unsupervised learning contexts show that Disen- tangO effectively extracts meaningful and interpretable latent features, bridging the gap between predictive performance and physical understanding in neural op- erator frameworks. Our code and data accompanying this paper are available at https://githinn com ni iin liu http://innni---aaiieeeeeeee iga/Disentango
INTRODUCTION
Interpretability in machine learning (ML) refers to the ability understand and explain how models make decisions (Rudin et al. 2022, Molnar 2020). As ML systems grow complex, especially with the use of deep learning and ensemble methods (Sagi & Rokach 2018), the reasoning behind their predictions can become opaque. Interpretability addresses this challenge by making the models more transparent, enabling users to trust the outcomes, detect biases, and identify potential flaws. It is critical factor in applying Al responsibly, especially in fields where accountability and fairness essential (Cooper et al,) 2022). In physics, where models endeavor to capture the governing laws and physical principles, understanding how model arrives its predictions is vital for verifying that aligns with known scientific theories. This transparency is key for advancing scientific discovery, validating results, and enhancing trust in models for complex physical systems.
Discovering interpretable representations of physical parameters in learning physical systems is challenging due to the intricate nature of real-world phenomena and the often implicit relationships between variables. In physics, quantities like force. energy, and velocity are governed by well- established laws, and extracting them in way that aligns with physical intuition requires models that go beyond mere pattern recognition. Traditional ML models may fit the data but fail to provide a physically interpretable way. To address this, recent developments include integrating physical constraints into the learning process (Raissi et al.] 2019). such as embedding conservation laws (Liu jet al. 2023; 2024a), symmetries (Mattheakis et al.) | 2019). or invariances (Cohen & Welling 2016) directly into model architectures. Additionally, methods like symbolic regression (Biggio et al.
"Equal contribution
1 "Corrsponniigg author, yuy21 aeeeiighhee

---

Published as a conference paper at ICLR 2026
2021) and sparse modeling (Carroll et al. 2009) aim to discover simple, interpretable expressions that capture the underlying dynamics. However, balancing model expressivity with interpretability remains significant challenge, as overly complex models may obscure the true physical relation- ships, while oversimplified models risk losing critical details of the system's 's behavior.
We introduce DisentangO, novel variational hyper-neural operator architecture to disentangle physical factors of variation from black-box neural operator parameters for solving parametric PDBs. Neural operators (NOs) (Li et al., 2020a c learn mappings between infinite-dimensional function spaces in the form of integral operators, making them powerful tools for discovering con- tinuum physical laws by manifesting the mappings between spatial and/or spatiotemporal data; see You et al. 2022a):3 Liu et al. (2024a b [ Ong et al. (2022); Cao (202T); et al. (2019) (221); Goswami et al. y Gupta et al. (2021) and references therein. However, most NOs serve as efficient forward surrogates for the underlying physical system under supervised learning setting. As result, they act as Mack-box universal approximators for a single physical system governed by a fixed set of PDE parameters, and lack interpretability with respect to the underlying physical laws. In contrast, the key innovation of DisentangO lies the use of a hypernetwork architecture that distills the varying physical parameters of the governing PDE from multiple physical systems through an unsupervised learning setting. The distilled variables are further disentangled into distinct physical factors enhance physical understanding and promote robust generalization. Consequently, Dis- entangO effectively extracts meaningful and interpretable physical features, thereby simultaneously solving both the forward and inverse problems. Our key contributions are:
We bridge the divide between predictive accuracy and physical interpretability. and intro- duce a new paradigm that simultaneously performs physies modeling (i.e., as forward PDE solver) and governing physical mechanism discovery (i.e., as an inverse PDE solver).
We propose novel variational hyper-neural operator architecture, which coin Disen- tangO. DisentangO extracts the key physical factors of variation from black-box neural operator parameters of multiple physical systems. These factors are then disentangled into distinct latent factors that enhance physical interpretation and generalization across differ- ent physical systems.
We provide theoretical analysis on the component-wise identifiability of the true generative factors in physics modeling: by learning from multiple physical systems, the variability of hidden physical states in these systems promotes identifiability.
We explore the practical utility of disentanglement and perform experiments across broad range of settings including supervised, semi-supervised, and unsupervised learning. Re- sults show that DisentangO effectively extracts meaningful physical features.
2 BACKGROUND AND RELATED WORK
Neural operators. Learning complex physical systems from data is essential in many scientific applications (Carleo et al., 2019 Liu et al. 2024c Karniadakis et al. [002 Zhang et al., 2018 Cai
let al. 2022 Pfau et al. 2020 He al. [201 Jafarzadeh et al.) 2024). When governing laws unknown, models must be both resolution-invariia for consistent performance across discretiza-
tions and interpretable for domain experts. Neural operators (NOs) achieve the former by learning mappings between
function spaces (Li et al. 2020a b You et al. 2022a Ong ct al. 2022 Cao |021| Lu al. 2019 202 Goswami et al. 2022 Gupta et 2021). enabling ac- curate and consistent predictions of continuum physical surrogates. However, NOs lack interpretable representations of physical states, limiting their ability to reveal underlying physical mechanisms.
Hypernetworks. Hypernetworks (Ha et al., 2016 Chauhan et al. 2024) are a class of neural net- work (NN) architectures that use one NN generate weights for another NN, both trained in an end-to-end manner. This allows soft weight sharing across tasks, benefiting transfer learning and dynamic information sharing (Chauhan et al., 2023). Hypernetworks can also be employed as a ver- satile technique in existing NN architectures. For instance, in Nguyen et al.; Oh & Peng (2022), hypernetwork is employed to generate parameters for a VAE model and enable multi-task learning. Similarly, Lee et al. (2023) integrates the hypernetwork architecture with neural operators. However, none of the existing work discusses the capability of hypernetworks in hidden physics discovery.
Forward and inverse PDE learning. Existing NOs serve efficient surrogates for forward PDE solving but often act as black-bos approximators, lacking interpretability. In contrast, deep learning
methods for inverse PDE solving (Fan & Ying 2023 Molinaro et al., 2023; Jiang al. 2022
2

---

Published as a conference paper at ICLR 2026
1r (x)
d
(x))
14(00
O
3
4
9
Bq
Iterative Fourier layers
Interpretable physics discovery
00
decoder
Loss
data loss
KT. loss
(semi-)juuperee jaspervsperrinnn
reconstruction loss
Multi-tusk leammnng
First decoder
Socond decoder
Figure 1: Overview of the DisentangO architecture, Each task correspond to different (hidden) PDE parameter b. For illustration, the same input function f," is shown for multiple tasks to highlight that different parameter fields 6" can produce different output functions u" under identical input f: in practice, f may vary across tasks. The -asc-specific lifting parameters 0% are encoded and reconstructed through VAE, and the reconstructed parameters 0% are fed into the iterative Fourier layers to form the task-specific neural operator G". Loss components are overlaid to indicate where each term the objective Lioes is computed.
2023) aim to reconstruct PDE parameters from solution data but face challenges due to ill-posedness. To address this, many NOs incorporate prior information via governing PDEs Yangg ct al. 2021
Chen et al.
Li al. 2021), regularizers (Dittmer et al. 22200 Obmann et al. [2020] Ding et al.] 2022 Chen et al. 2023), structured operators (Lai et al. 2019 Yilmaz (2001). However, these methods
assume prior knowledge of the model form, which is often unrealistic. A recent approach Yuu et al,,
2024) employs attention construct data-dependent kernel for inverse mapping but does not disentangle the learned kernel or extract interpretable parameters. To our knowledge, DisentangO the first to tackle both forward (physics prediction) and inverse (physics discovery) PDE learning while simultaneously identifying distinct physical parameters from the learned NO.
Disentangled representation learning. Disentangled representation learning separates data into distinct, interpretable factors, each capturing an independent variation. It has critical implications for transfer learning, generative modeling, and AI fairness, enabling models to generalize by leveraging isolated features. Key advances have been driven by models like 3-VAE (Higgins et al.) 2017), PactorVAB (Kim & Mnih, 2018), and InfoGAN (Chen et al.) (016). which promote disentanglemeet through latent regularization and mutual information constraints. While early work focused on supervised or semi-supervised methods (Ridgeway & Mozer 2018 Shu el al., [2019] Mathieu al. 2019). recent efforts target unsupervised learning (Duan al.) 2019). though challenges remain (ocatello et al.) 2019), with ongoing efforts to improve metrics, robustness. and applicability to complex data. Most research has centered computer vision and robotics, where latent factors have hww.n-intemppeeeeeeeeeeeeeeeeeee iitep-nitterpeetable visual meanings, while its exploration in physical system learning remains limited (Lingsch al., 2024 Tong al. 2024 Fotiadis et al. 2023). Moreover, existing work disentangles from data, whereas our work is the first to disentangle from black-box NN parameters.
3 DISENTANGO
We consider a series of complex systems with different hidden physical parameters:
[u] (m) f()),
Ω
(3.1)
Here, Ω C R is the domain of interest, f(x) is function representing the loading on Ω u(xx)) is the corresponding solution of this system. KG represents the unknown governing law, e-g. balance laws, determined by the (possibly unknown and high-dimensional) parameter field b. For instance, in a material modeling problem, K' often stands for the constitutive law and b can be vector (b RR:) representing the homogenized material parameter field vector-valued function (b L (2; Rd)) representing the heterogeneous material properties. Both scenarios are considered in empirical experiments in Section
Many physical modeling tasks can be formulated as either a forward or an inverse PDE-solving problem. In a forward problem setting, aims find the PDE solution when given PDE informa-
3

---

Published as a conference paper at ICLR 2026
tion, including coefficient functions, boundary conditions, initial conditions, and loading sources.
That means, given the governing operators Ic, the parameter (field) b. and loading field f(x) in equation 3.1 the goal is to solve for the corresponding solution field u u(x). through classical PDE
solvers (Brenner & Scott 2007) or data-driven approaches (Lu et al. 2019 [i et al. 2020c). As a result, a forward map is constructed:
g: (b, f) u.
(3.2)
Here, b and f are input vectors/functions, and u is the output function.
Conversely, solving inverse PDE problem involves reconstructing the underlying full or partial PDE information from PDE solutions, where one seeks to construct an inverse map:
(3.3) Unfortunately, solving an inverse problem is typically more challenging due to the ill-posed nature of the PDE model. In general, a small number of function pairs (u. f) from a single system does not suffice in inferring the underlying parameter field b, making the inverse problem generally non- identifiable (Molinaro et al. 2023).
H. : (u, f) b.
Herein, we propose novel neural architecture alleviate the curse of ill- ill-poseddees without access to the exact governing partial differential equation (PDE). The key ingredients are: 1) the construc- tion of multi-task NO architecture, which solves both the forward (physics prediction) problems simultaneously from multiple PDE systems with different hidden parameters; 2) a generative model to disentangle the key features from NO parameters that contain critical information of b, as the inverse (physics discovery) problem solver.
3.1 VARIATIONAL HYPER- NEURAL OPERATOR AS A MULTI-TASK SOLVER
Notation and data model assumptions. We denote by f(x) and uxx) the input and output func- tions of the NO, respectively. Let p(-) denote a probability density function, EE.) the expectation, and ||-I] the (2-norm. We assume (noisy) observations of both u and f are available on common physical domain ?2 where E U C (Q:: Rdx) and f F L Rd)). Here, 24 and F are the Banach spaces of solution fields and loading fields, respectively. Formally, we consider training datasets D", 17 1, ..... S, each corresponding to the same PDE equation 3. but with different (hidden) physical parameter b" E B, with B the parameter space. In our multi-task learning model, each dataset corresponds one task. We assume the hidden parameter is generated according to:
b~ P. 2 ~ p(zlb)
(3.4)
C IRd is the latent embedding of b. Each dataset contains measurements from Train function pairs (x), (x)) Although method accommodates datasets with varying numbers of function pairs, we use the same number of function pairs in this work for sim- plicity and denote n"rain Merain- While actual observations are gathered on discrete sensor set x = [z,),,= and inevitably contain observational noise of the solution, we adopt the standard assumption that this noise is additive and i.i.d. Letting [f bj denote the true forward operator of equation 3.1 we write the data model associated with multi-task learning as:
where E EEZC
D [f(x)) # Perain D = U (
(3.5)
with
f f~ = f.
(x;). w = um( + Gnii = 0' + Gjājo G ~ N((0, x
(3.6)
The forward modeling objective is to learn a surrogate solution operator g[: 0') for gl, where O denotes the task-specific NO parameters for the forward surrogate operator in the 17-th task, associated with 6". Note that all tasks share the same NO architecture, G, and the surrogate operator for each task depends on the physical system 17 through the task task-specific parameter 07.
For inverse modeling, let Ht denote the (unknown) inverse operator that maps function pairs (u, f) to the underlying PDE parameter data of (u. f) may not contain sufficient information to identify all features of b. For instance, in a Dirichlet boundary condition problem, x²) = for all E in, making dz not learnable on a.. A more realistic goal is therefore to recover the underlying mechanism of b in the space of identifiability, i.e., z. Thus, our second objective is to construct an inverse map by estimating embedding tn of the hidden parameter 6"d from 8":
practice, identifying Gtis often impossible, as the available
H(00 O1)) 2"
(3.7)
4

---

Published as a conference paper at ICLR 2026
where Oµ trainable parameters of H. and 2" E Rd denotes the learned latent variables that can be transformed to the ground-truth latent variable x" via an invertible function h. The goal of disentanglement is discover &, together with the solution operator g.
The first objective learns the forward operator G[:; in a supervised fashion, in the form of cunncion-to-functionn mappings, for all (hidden) parameters b" in the range of interest; while the second objective learns the vector-to-vector mapping from d to z", which is unsupervised due the hidden physics nature. We propose employ a variational autoencoder (VAE) (Kingma & Welling 2013) as the representation learning approach for the second objective, and a meta-learned NO architecture as universal solution operator for the first objectivd"] The key is to pair these archi- tectures as hyper-neure neural operator, which simultaneously solves both forward and inverse problems. Although the proposed strategy generic and thus applicable to other multi-tast neural operator ar- chitectures, to provide universal architecture of & for different tasks, we adopt meta-learning strategy following the meta-learned neural operator (MetaNO) (Zhang et al.) (023). MetaNO developed based on the implicit Fourier neural operator (IFNO), PDE solution operator with rel- atively small number of trainable parameters You et al. 2022b). In MetaNO, task-wise adaptation is applied only the trainable parameters in the first layer of the NO, whereas all other parameters are shared across tasks. For an L-layer MetaNO, we write:
([f:0")(>) G[f: f,, 0Q] (2)) 20 (Jo,)² Pop. [x))
(3.8)
where P. Q are saallow-layer MLPs that map low-dimensional vector to a high-dimensionas vector and vice versa, parameterized by 0% and fq. respectively. Each intermediate layer, J. is constructed as mimetic of fixed-point iteration step and parameterized by 0,, Supported by the universal ap- proximator analysis (Zhang et al. 2023). different PDEs share common iterative (0.r) and projection (02) parameters, with all the information about parameter b encoded in the lask-wise lifting parame- ters f%. Hence, to provide an inverse map from the task-wise NO parameter ( to the key fcatures of the PDE parameters, z", one construct H. as the mapping from 0% to zn using neural network architecture. In this work, we employ a standard MIP:
() == MLP(0%)
(3.9)
since all other NO parameters are invariant to the change of b. This construction substantially reduces the degrees of freedom in 8", making the invertibility assumption in the next section feasible. To simplify notation, forward and inverse operator architecture is provided in Figure
0 denote 0% in the subsequent discussion. An overview of the
We now define the learning objective. The overall objective is maximize the log data likelihood:
Eax p(0, D)) = max [[ [llog 0) + p(0 + logg(f)))
Note that p(f) remains constant over different NO parameter 0, and the assumption in eq. 3.6 yields log(p(0 f)) = log()(0)). Additionally, the assumption in eq. 3.4 guarantees that is generated from the latent space over z, hence log(p(0)) log J- p(0) = pp jdz. The overall objective then becomes:
max ((oog(p(( D))) = max [(log(p(u). 0))) + E log p(01z)p(z)dz))
(3.10)
However, the second term in this formulation is generally intractable. We variational posterior q(0)2) 2) to approximate the actual posterior p(0)z) and maximize the evidence lower bound (ELBO):
1 Σ S
LELBO
Ey(z on) log p(0) DK (1<=)))))))))
n=1
with DKL(-H denoting the KL divergence between two distributions. Putting everything together, we obtain the loss functional:
s
Lioss = 5
Σ 1=
B(lo((p(u)f. (')))) By(==1oo) log p(0*(=") - DxL (<(x"100))l<<(")))
(3.11)
'Note that tasks differ in the PDE coefficients (the physical parameters 6²) and correspondingly the NO parameter 6", not in the NO architecture its input-output structure.
5

---

Published as a conference paper at ICLR 2026
The above formulation naturally lends itself to hierarchical variational autoencoder (HVAE) archi- tecture (Vahdat & Kautz 2020). The encoder aims to obtain the posterior qu. (x" (*) and provide the inverse map H.:
M (2"
(3.12)
Note that although represents a deterministic inverse mapping from the NO parameters to the latent variables, in equation [.12 we approximate this mapping using a prohabilistic encoder to account for uncertainty and enable variational inference. This follows standard practice in the VAE literature, where deterministic relationships are modeled probabilistically for tractable inference.
Then, the first decoder processes the estimated latent variable & and reconstructs the corresponding NO parameter 0:
0(2).
(3.13)
Lastly, the second decoder reconstructs the forward map G, by further taking the estimated and the loading function f and estimating the output function u.
(3.14)
3.2 DISENTANGLING THE UNDERLYING MECHANNSMM
DisentangO aims to identify and disentangle the components of the latent representation z, which serves as an inverse PDE solver. However, to capture the true physical mechanism, a natural ques- tion arises: is it really possible to identify the latent variables of interest (i.e., 2) with only observa- tional data { ? marginal data distribution in all domains, we can indeed achieve this identifiability for the generat- ing process proposed equation 3.4 and equation 3.6 Before the formal theorem, we first state our assumptions:
We now show that, by learning the model (P2. 9. H. 9) that matches the true
Assumption 1. (Density Smoothness and Positivity) The probability density functions for and z, which we denote pa and Pz. are both smooth and positive.
Assumption 2. (Invertibilityy) The task-wise parameter & can be generated by through an invertible and smooth function - Moreover, for each given f. we denote g g.f: (xx) as the operator mapping from Rede U. 9y is also one-to-one with respect to Assumption 3. (Conditional Independence) Conditioned b, each component of is independent of each other: log logP2)((2(b) Ee= log Pa.(b(2;) (b).
Assumption 4. (Linear independence) There exists 2d_ + values b, such that the 2d, vectors w(z b²) w(2 bo) with j = I,... 2d are linearly independent. Here,
w(z,b') = (2,, 6¹)
Oqs, (za,.b) b²) Oza.
0 (21. (21.b) b²)
Iaqa (Zd,, 023,
(3.15)
Ozi
0=7
with (z,z, = :ll==)ppa:a Po:b>
First, we show that the latent variable can be identified up to an invertible component-wist trans- formation: for the true latent variable z E IRd,, there exists an invertible function h : Rd: -> Rdd,
such that 2 = h(z).
Theorem 1. We follow the data-generating process in equation 3.4 and equation 3.6 and Assump-
tions 2 Then. by learning (ps. 9. H. 9) to achieve:
Pūf Pulf
(3.16)
where u and á are generated from the true process and the estimated model, respectively, z is identifiable up to invertible function h.
Proof: Please see Appendix[A)
Additionally, with additional assumptions on conditional independence and datum variability, we can further obtain the following theoretical results on component-wist identifiability: for each true component 2i, there exists a corresponding estimated component ε, and an invertible function h; + R. such that 2, h2))
Theorem 2. We follow the setta-genrating process in equation 3.4 and equation 3.6 and Assump-
tions 1 4 Then, by learning (pz, ÿ. H. 9) to achieve equation 3.16 component-wise identifiable.

---

Published as a conference paper at ICLR 2026
Proof: Please see Appendix
Discussion assumptions. Intuitively, Assumptions 12 required to guarantee that there exists a smooth and injective mapping between the ground-truth latent embedding 2 and the learned em- bedding 2. Assumption indicates sufficient variability across physical systems. This is common assumption in the nonlinear ICA literature for domain adaptation (Hyvarinen et al. 2019) Khe- makhem al. 20200] Kong al. 2023). Further discussions and validation the assumptions provided in Appendix A To our best knowledge, it is the first time the component component-wise wise identifiabil- neural operator learning.
ity is discussed in the context
3.3 A GENERIC ALGORITHM FOR VARIOUS SUPERVISION CASES
Although DisentangO is primarily designed for the challenging scenario of learning without super- vision on b nor prior knowledge on the PDE model form in equation 3.1 its methodology is generic and readily can be extended handle the scenarios with partial full measurements of b, which are common in classical inverse PDE benchmark problems. In this section, we discuss the practical utility of DisentangO under three scenarios:
(SC1: Supervised) The value of b" is given.
(SC2: Semi-supervised) The value 6 not given, but a label c(6") (e.g.. a classification of b²) is given.
(SC3: Unsupervised) No value or label is given for b" for each task.
pseudo Algorithm D is provided the Appendix.
To obtain the posterior in equation 3.12 we assume that each latent variable satisfies a Gaussian distribution distinct means and diagonal covariance, i.e., Ga,,, E, (210) == (O), (0), then estimate its mean and covariance using MLP. As result, the KL-divergence term in the ELBO admits closed form:
ds
SC2/SC3: DEL (a(=10) |I p(z))
Σ
(Ex)) (mx)? - log((()))) -
(3.17)
2 1=1
In the supervised setting, we take (=(b
b", and then the KL divergence term writes:
1 dx
SC1: DEL (a(210) ll p(z))
Σ 2 :=1
(E.) + (m= b)? 2 2log((E2)) 1).
(3.18)
For the first decoder, the likelihood p()" is factorized Gaussian with mean μa and covariance Eg. computed from another MLP. By taking as input Monte Carlo samples once for each x, the reconstruction accuracy tenm can be approximated as:
de
((0); 10)0-((up (1o))) 2(Ex)?
de
Eg(=10) logp(012))
(log(Eo); +c)
where de is the dimension of and is a constant.
For the second decoder, we parameterize it as multi-task NO in equation 3.8 and model the discrepancy between &1f""; (xx)) and the ground truth u(xx) as an additive independent unbiased Gaussian random noise c, with
$18 0-] (mxx) (arx) + Gn.j.k Gnj. [00 w²),
where is the standard deviation of the additive noise as defined in the data model. In practice, the observational noise is unknown, and we treat to as a tunable hyperparameter. Then, with uniform spatial discretization of size Ar in domain Ω constant terms can be written as:
Rd, the log likelihood after eliminating the
Berain
(()7; u." -.3(arx)) 12(000"
(3.19)
2 2
2d...
In our empirical tests in Section for the unsupervised and semi-supervised scenarios, choose q as a standard Gaussian distribution following the independence assumption Theorem Addi- tionally, to avoid overparameterization, take E, I with dg being tunable hyperparameter.

---

Published as a conference paper at ICLR 2026
In this case, the total objective be further simplified as:
Loss
Ba
Σ
@1
u²] 12(())
| 1p8l-11 n8l1 3 3kellu21?
for the unsupervised scenario, and
8
Lososs
Ba ==1
01 - u²] (2(2))
+ -1000 - -v8l + 3KL 11 - b²1
S
7=1
for the fully supervised scenario. Here, Ba := w ad.. and 3K == of are treated as tunable hyperparameters. In the semi-supervised scenario, we incorporate partial supervision by equipping the above loss with a constraint following Locatello et al. (2020):
o
Lloss S
Σ 7-1
Ba Σ
[053; 6m
u3] L2(())
+ -00-1
18 + 3KL 11 + Bets ( c(("))
with Bats tunable parameter and La(=", c(3)) the corresponding loss term based provided information (e.g.. Le is the cross-entropy loss when taking e as the classification label of (3").
Note that 3KL is closely connected to the adjustable hyperparameter in 3-VAE (Higgins et al. 2017). Taking a larger 3x encourages disentanglement (Locatello et al. 2019), while also adds additional constraint BBuggess et al. 2018). On the other hand, the data reconstruction term forces the latent factors to contribute to the "reconstruction" of the more complicated solution field in a global perspective, and hence increasing Sg is anticipated to alleviate information loss. In our empirical study, we demonstrate the interplay between these two tunable hyperparameters.
the implicit capacity of the latent bottleneck resulting in information loss
4 EXPERIMENTS
We assess DisentangO across vari- ous physics modeling and discovery datasets, denoting model with latent dimension n as DisentangO-n. Our evaluation focuses several key pects. Firstly, we demonstrate the capability of DisentangO in forward PDE learning, and compare its perfor- mance with a total of 14 relevant base- lines. In particular, we select 8 NO baselines. i.c., FNO [Li et al. 2020c), UFNO (Wen et al.] 2022). NIO (Moli-
Table 1: Test errors and the number of trainable parameters in experiment 1. Bold number highlights the best method.
Models
#param
per-epoch
Test errors
time (s)
data
(SCT)
DisentangO
697k
12.2
1.65%
4.63%
MetaNO
296k
9.8
1.59%
NIO
709k
5.6
15.16%
FNO
698k
9.1
2.45%
14.55%
UFNO
720k
21.2
7.61%
11.23%
WNO
672k
183.5
8.09%
9.95%
PIANO
699k
16.9
7.99%
15.35%
CAMEL
654k
5.53
112.28%
SMDP
67 lk
5.12
17.76%
naro et al. 2023 WNO (Tripura
InVeEnt
707k
5.16%
Chakraborty 2022). PIANO (Zhang
FUSE
706k
2.2
4.99%
ctt al. 2024). MetaNO (Zhang et al.
FUSE-f
707k
11.4
16.33%
6.19%
2023 FUSE (Lingsch et al. 2024) and its extension, as well as six non- NO baselines, i.e., CAMEL (Blanke &
VAE
698k
2.8
49.97%
16.34%
VAE
664k
81.11%
16.27%
8 VAE
698k
2.8
51.16%
16.47%
Lelarge 2023). VAEt (Tong et al.
et al.] (023), two VAE variants Kingma & Welling 2013) and 3-VAE (Higgins et al. 2017). Secondly, showease the merits of
2024). SMDP
DisentangO in inverse modeling for interpretable physics discovery. We perform parametric studies
on the associated disentanglement parameters. Lastly, we provide interpretations of the disentan- gled parameters. Further details and an additional experiment for identifiability demonstration are respectively.
provided in Appendices B and C.4
4.1 SUPERVISED FORWARD AND INVERSE PDE LEARNING
We start by investigating DisentangO's capability in solving both forward and inverse PDE problems in a fully supervised setting. Specifically, we consider the constitutive modeling anisotropic fiber-reinforced hyperelastic materials governed by the Holzapfel-Gasser--0 (HGO) model,
8

---

Published as a conference paper at ICLR 2026
where the data- data- genaratinn process is controlled by sampling the governing material parameter set {E, v, h1, 22. a and the latent factors can be learned consequently in a supervised fashion. In this setting, the model takes as imput the padded traction field and learns to predict the displacement.
As the number of latent factors is fixed for supervised learning in this experiment, we defer our ab- lation study to the second experiment. We report in Table I our experimental results. Note that only FNO, PIANO, FUSE-f, VAE and its variations are able to handle both forward and inverse problems, whereas MetaNO can only solve the forward problem, and NIO, InVAErt, and FUSE can only solve the inverse problem. With this caveat in mind, MetaNO achieves the best performance in forward PDE learning, with Disentang0 performing comparably well and beating the third best model by 32.7% in accuracy. In terms of inverse modeling with full latent supervision (SC1), DisentangO is the only method that can hold the error well below 5%, outperforming the second best method by 7.2% and the second best joint (i.e., simultaneous forward and inverse) solver by 25.2%.
4.2 SEMI-SUPERVISED MECHANICAL MNIST BENCHMARK
We consider semi- supervised learning and apply DisentangO to the Mechanical MNIST (MMNIST) benchmark (Lejeune 2020). MMNIST comprises 70,000 heterogeneous material specimens under- going large deformation, each governed by material model of the Neo-Hookean type with varying modulus converted from the MNIST bitmap images. In our experiment, we take 500 images (with a 420/40/40 split for vaiiiing/alidationneee
and generate 200 loading/response data pairs per sample on a 29 x 29 structured grid, simulating uniaxial extension, shear, equibiaxial extension, and con- fined compression load scenarios. Since only partial knowledge is available for each image (i.e., the corresponding digit), we apply semi sum-rvupee learning to the latent factors to classify the digits.
Table 2: Test enrors and number of trainable parameters in experiment 2. DisentangO is abbreviated as DNO due space limit. Bold number highlights the best method that can handle both forward and inverse settings.
Models
DNO-2
DNO-5
DNO-10 10
DNO-15
VAE
B VAE
MetaNO
PIANO
#param (M)
0.66
0.97
1.49
2.02
2.02
2.02
0.35
2.05
8g
12.82%
9.56%
7.36%
6.29%
16.34%
17.13%
2.68%
13.73%
8a 10
11.51%
9.16%
6.62%
5.95%
8a- 100
11.49%
8.43%
6.65%
5.48%
8a- 1000
11.62%
8.22%
6.50%
5.80%
Ablation study. Firstly, we investigate DisentangO's predictability in forward PDE learning by comparing its performance to MetaNO (i.e., the base meta-learned NO model without the hyper- network structure). As seen in Table - MetaNO achieves forward prediction error of 2.68%, which serves as the optimal bound for DisentangO. As we increase the latent dimension in Disen- tangO from 2 to 15, the prediction error drops from 11.49% to 5.48% and converges to the optimal bound. Next, we study the role of the data loss term in disentanglement by gradually varying 8a from 8a= 1 to 3.= = 1000. In Table 2 we observe a consistent improvement in accuracy with the increase in 3a. where the boost in accuracy becomes marginal or slightly deteriorates beyond Ba 100. We thus choose 3.2 = 100 as the best DisentangO model in this case. Besides offering an increased accuracy in forward modeling, the data loss term also enhances disentanglement as discussed in Section 3.2 This is evidenced by the illustration in Figure 3, where the unsupervised mutual in- formation (MI) score that measures the amount of MI across latent factors consistently decreases in both DisentangO-2 and DisentangO-15 as we increase Ba from Ba = 1 (solid lines) to 33, = 100 (dashed lines). On the contrary, the classification term poses a negative effect on disentanglement, as indicated by the increase in Pets leading to an increase in MI scores. This is reasonable because the classifier linearly combines all the latent factors to make classification. The more accurate the classification, the stronger the correlation across latent factors. We then move on to study the effect of eumissupe learning by comparing the model's performance with and without latent partial supervision. While the models without semi- seni-supervision is slightly more accurate in that the forward prediction accuracy with 3a = 1 reaches 12.60% and 6. 166 in DisentangO-2 and DisentangO-15. respectively, the latent scatterplot in Figure 2 reveals the inability of the DisentangO model without latent semi-supervision to acquire the partial knowledge of the embedded digits from data. In con- trast, although the accuracy of DisentangO with latent semi-supervision slightly deteriorates due to the additional regularization effect from the classification loss term, it is able to correctly recognize the embedded digits and leverage this partial knowledge in disentangling meaningful latent factors.
Comparison with additional baselines. We compare DisentangO with additional basclines, i.c., VAE and B- VAE. We abandon the three NO-related baselines in this and the following experiments

---

Published as a conference paper at ICLR 2026
as they are black-box approximators and do not possess any mechanism to extract meaningful infor- mation without supervision. In this context, even the least accurate DisentangO-2 model (i.e., with 3a = 1) with significantly fewer parameters outperforms the selected baselines in Table 21.5% and 25.2%, respectively, with the best performing Disentang0-15 model beating the baselines by 66.5% and 68.0% in accuracy, respectively. We do not proceed to compare the physics discovery capability between them as the baseline models are considerably inaccurate in forward prediction.
Interpretable physics discovery. To showcase the interpretability of DisentangO, we visualize its latent variables through latent traversal based on a randomly picked loading field as input in DisentangO-2. as displayed in F Appendix [B.3] One can clearly see that the digit changes from "6" to "0" and then "2" from the top left moving down, and from "6" to "I" and then "7" moving to the right. Other digits are visible as well such as "7", "9", "4" and "8" in the right-most column. This corresponds well to the distribution of the latent clustering in Figure 2 More discus- sions on the interpretability of DisentangO-1s can be found in Figures and 8 Appendix B3.]
4.3 UNSUPERVISED HETEROGENEOUS MATERIAL LEARNING
We demonstrate DisentangO in unsupervised learning in the context of learning synthetic tis- sues that exhibit highly organized structures, with collagen fiber arrangements varying spa- tially. In this case, it critical to understand the underlying low-dimensional disentangled prop- erties in the hidden latent space of complex, inferred
Table 3: Test errors and number of trainable param-
eters in experiment 3. DisentangO is abbreviated as DNO due to space limit. Bold number highlights the best method for both forward and inverse settings.
Models
#param (M)
Test error
Ba
Aa- 100
DNO-2
0.63
26.33%
25.18%
high-dimensional microstructure, from experimental mechanical measurements, We generate two datasets, each containing 500 specimens and 100 loading/displacement pairs.
DNO-S
0.94
17.51%
15.75%
DNO-10
1.46
10.76%
10.01 %
DNO-15
1.98
7.11%
7.02%
DNO-30
3.55
5.33%
5.28%
VAE
3.55
61.10%
The first dataset features variations in fiber ori- entation distributions using Gaussian Random Field (GRF) and the second differs fiber angles in two segmented regions, separated by center- line with a randomly rotated orientation.
B-VAE
3.55
57.04%
MetaNO
0.32
2.67%
PIANO
3.42
50.71%
We report in Table 3 our experimental results of DisentangO with different latent dimensions and data loss strength Sd. along with comparisons with the baseline model results. Consistent with the findings in the second experiment, increasing 3a results in boost in accuracy by comparing the rows in the table. The model's predictive performance also converges to the optimal bound of MetaNO as we increase the latent dimension from to 30, where the model's prediction error improves from 25. 18% to 5.28%. This significantly outperforms the best baseline model by 90.7%. On the other hand, the effect of the data loss term on disentanglement further proved in Figure 4 where in- creasing 3r leads to decrease in MI between the latent factors. thus encouraging disentanglement. Lastly, we interpret the mechanism of the learned latent factors in DisentangO-3 via learning map- ping between the learned latent factors and the underlying material microstructure and subsequently performing latent traversal in each dimension. The results are shown in Figure 5 where the three latent factors manifest control on the border rotation between the two segments, the relative fiber orientation between the two segments, and the fiber orientation of the top segment, respectively.
5 CONCLUSION
We present DisentangO for disentangling latent physical factors embedded within black-box NO parameters. Disentang( leverages hypernetwork-type NO architecture that extracts varying pa- rameters of the governing PDE through a task-wise adaptive layer, and further disentangles these variations into distinct latent factors. By learning disentangled representations, DisentangO not only enhances physical interpretability but also enables robust generalization across diverse physical sys- tems under different learning contexts. Limitations: Since the scalability of DisentangO is gov- erned by the scalability of the NO backbone used for forward modeling, which has been extensively discussed in previous works, this study focuses on experiments involving high latent dimensions. Demonstrations on high-dimensional PDEs are beyond the scope of the current work.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
L. Zhang was partially funded by the NSF grant DMS 2436624 and the AFOSR grant FA9550-22-1- 0197. Y. Yu would like to acknowledge the support by the National Institute of Health under award IROIGMI57589.00 Portions of this research were conducted on Lehigh University's Research Computing infrastructure, partially supported by NSF Award 2019035.
REFERENCES
Martin Alnæs, Jan Blechta, Johan Hake, August Johansson, Benjamin Kehlet, Anders Logg, Chris Richardson, Johannes Ring. Marie E Rognes, and Garth N Wells. The fenics project version 1.5. Archive of Numerical Software, 3(100), 2015.
Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, and Giambattista Parascan- dolo. Neural symbolic regression that scales. In International Conference on Machine Learning, pp. 936-945. Pmir, 2021.
Matthieu Blanke and Mare Lelarge. Interpretable meta-learning of physical systems. arXiv preprint arXiv:2312.00477, 2023.
Susanne Brenner and Ridgway Scott. The mathematical theory of finite element methods, volume 15. Springer Science Business Media, 2007.
Christopher P Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Des- jardins, and Alexander Lerchner. Understanding disentangling in beta-vae. arXiv preprint arXiv:1804.03599 2018.
Shengze Cai, Zhiping Mao, Zhicheng Wang, Minglang Yin, and George Em Karniadakis. Physics- informed neural networks (PINNs) for fluid mechanics: A review. Acta Mechanica Sinica, pp- 2022.
2022.
Shuhao Cao. Choose transformer: Fourier or galerkin. Advances in neural information processing systems, 34:24924-24940, 2021.
Giuseppe Carleo, Ignacio Cirac, Kyle Cranmer, Laurent Daudet, Maria Schuld, Naftali Tishby, Leslie Vogt-Maranto, and Lenka Zdeborová. Machine learning and the physical sciences. Re- views of Modern Physics, 91(4):045002, 2019.
Melissa K Carroll, Guillermo A Cecchi, Irina Rish, Rahul Garg, and A Ravishankar Rao. Prediction and interpretation of distributed neural activity with sparse models. Neurolmage, 44(12:122-122,
2009.
Vinod Kumar Chauhan, Jiandong Zhou, Soheila Molaei, Ghadeer Ghosheh, and David A Clifton. Dynamic inter-treatment information sharing for heterogeneous treatment effects estimation. arXiv preprini arXiv:2305.15984 2023.
Vinod Kumar Chauhan, Jiandong Zhou, Ping Lu, Soheila Molaei, and David Clifton. A brief review of hypernetworks in deep learning. Artificial Intelligence Review, 57(9):250, 2024.
Ke Chen, Chummei Wang, and Haizhao Yang. let data talk: data-regularized operator learning theory for inverse problems. arXiv preprint arXiv:2310.0983: 2023.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Info- gan: Interpretable representation learning by information maximizing generative adversarial nets. Advances in neural information processing systems, 29, 2016.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International confer-
ence on machine learning, pp. 2990-2999. PMI.R, 2016.
Feder Cooper, Emanuel Moss, Benjamin Laufer, and Helen Nissenbaum. Accountability in an algorithmic society: relationality, responsibility, and robustness in machine learning. In Proceed- ings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 664-876, 2022.
11

---

Published as a conference paper at ICLR 2026
Wen Ding, Kui Ren, and Lu Zhang. Coupling deep learning with full waveform inversion. arXiv preprint arXiv:2203.0/799 2022.
Sören Dittmer, Tobias Kluth, Peter Maass, and Daniel Otero Baguer. Regularization by architecture: A deep prior approach for inverse problems. Journal of Mathematical Imaging and Vision, 62: 456-470, 2020.
Sunny Duan, Loic Matthey, Andre Saraiva, Nicholas Watters, Christopher P Burgess, Alexander Lerchner, and Irina Higgins. Unsupervised model selection for variational disentangled represen- tation learning. arXiv preprint arXiv: /905. /2614, 14 2019.
Yuwei Fan and Lexing Ying. Solving traveltime tomography with deep learning. Communications in Mathematics and Statisties, 11(1):3-19, 2023.
Stathi Fotiadis, Mario Lino Valencia, Shunlong Hu, Stef Garasto, Chris D Cantwell, and Anil An- thony Bharath. Disentangled generative models for robust prediction of system dynamics, In International Conference on Machine Learning, pp. 10222-10248, PMLR, 2023.
Somdatta Goswami. Aniruddha Bora, Yue Yu, and George Em Karniadakis, Physics-informe neu- ral operators. 2022 arXiv preprint arXiv:2207.05744 055448 2022.
Gaurav Gupta, Xiongye Xiao, and Paul Bogdan. Multiwavelet- operator learning for differen- tial equations. Advances in neural information processing systems, 34:24048-24062, 2021.
David Ha, Andrew Dai, and Quoe V Le. Hypernetworks. arXiv preprint wrXiv:1609.09106 2016.
Qizhi He, Devin W Laurence, Chung-Hao Lee, and Jiun-Shyan Chen. Manifold learning based data-driven modeling for soft biological tissues. Journal of Biomechanics, 117:1 10124, 2021.
Irina Higgins, Loic Matthey, Arka Pal, Christopher P Burgess, Xavier Glorot, Matthew M Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vac: Learning basic visual concepts with constrained variational framework. ICLR (Poster), 3, 2017.
Benjamin Holzschuh, Simona Vegetti, and Nils Thuerey. Solving inverse physics problems with score matching, Advances in Neural Information Processing Systems, 36, 2023.
Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference Artificial Intelligence and Statistics, pp. 859-868. PMLR, 2019.
Siavash Jafarzadeh, Stewart Silling, Ning Liu, Zhongqiang Zhang, and Yue Yu. Peridynamic neural operators: A data-driven nonlocal constitutive model for complex material responses. Computer Methods in Applied Mechanics and Engineering, 425:116914 16914, 2024.
Hanyang Jiang, Yuehaw Khoo, and Haizhao Yang. Reinforced inverse scattering. arXiv preprint arXiv:2206.04/88 2022.
George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physice-informed machine learning. Nature Reviews Physics, 3(6))422-440, 2021.
Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoen- coders and nonlinear ica: A unifying framework. In International conference on artificial intelli- gence and statistics, pp. 2207-2217. PMLR, 2020.
Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In International Conference on Machine Learning, pp. 2649-2658. PMLR, 2018.
Diederik Kingma and Max Welling.
Auto-encoding variational bayes.
arXiv preprint
arXiv:1312.61 14, 2013.
Lingjing Kong, Shaoan Xie, Weiran Yao. Yujia Zheng, Guangyi Chen, Petar Stojanov, Victor Akinwande, and Kun Zhang. Partial identifiability for domain adaptation. arXiv preprint wrXiv:2306.005/1 2023.
12

---

Published as a conference paper at ICLR 2026
Ru- Yu Lai. Qin Li, and Gunther Uhlmann. Inverse problems for the stationary transport equation in the diffusion scaling. SIAM Journal on Applied Mathematics, 79(6):2340223 2358, 2019.
Annika Lang and Jürgen Potthoff. Fast simulation of gaussian random fields. Monte Carlo Methods & Applications, 17(3), 2011a.
Annika Lang and Jürgen Potthoff. Fast simulation of gaussian random fields. Monte Carlo Methods and Applications, 17(3):15--211 214, 2011b. doi: dot:1001511/mmma.2111...m URL https:// ori.org/ 10. 1515 mcma. 2011 .009
Doksoo Lee, Lu Zhang, Yue Yu, and Wei Chen. Deep neural operator enabled concurrent mul- titask design for multifunctional metamaterials under heterogeneous fields. Advanced Optical Materials, pp. 2303087, 2024.
Jac Yong Lee, Sung Woong Cho, and Hyung Hwang. Hyperdeeponet: learning operator with complex target function space using the limited resources via hypemettwrk. arXiv preprint arXiv:2312.15949 2023.
Emma Lejeune. Mechanical mnist: A benchmark dataset for mechanical metamodels. Extreme Mechanics Letters, 36: 36:100659, 2020.
Zongyi Li, Nikola Kovachki, Kamyar Arizzadenesheli, Burigede Liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differ- ential equations. arXiv preprint arXiv:2003.03465. 2020a.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Andrew Stuart, Kaushik Bhat- tacharya, and Anima Anandkumar. Multipole graph neural operator for parametrie partial differ- ential equations. Advances in Neural Information Processing Systems, 33:NeurIPS 2020, 2020b.
Zongyi Li, Nikola Borislavov Kovachki. Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier NeuralOperator for Parametric Partial Differential Equa- tions. In International Conference Learning Representations, 2020c.
Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Arizzadenesheli, and Anima Anandkumar. Physics-informed neural operator for learning partial differential equations. 2021 arXiv preprint vrXiv:2111.0379 2021.
Levi E. Lingsch, Dana Grund, Siddhartha Mishra, and Georgios Kissas. Fuse: Fast unified simulation and estimation for pdes. arXiv preprint arXiv:2405.. 14558, 2024.
Ning Liu, Yue Yu, Huaiqian You, and Neeraj Tatikola. Ino: Invariant neural operators for learning complex physical systems with momentum conservation. In International Conference Artifi- cial Intelligecce and Statistics, pp. 6822-6838. PMLR, 2023.
Ning Liu, Yiming Fan, Xianyi Zeng, Milan Klöwer, Lu Zhang, and Yue Yu. Harnessing the power of neural operators with automatically encoded conservation laws. In Forty-first International Conference on Machine Learning, 2024a.
Ning Liu, Siavash Jafarzadeh, and Yue Yu. Domain agnostic fourier neural operators. Advances in Neural Information Processing Systems, 36, 2024b.
Ning Liu, Xuxiao Li, Manoj R Rajanna, Edward W Reutzel, Brady Sawyer, Prahalada Rao, Jim Lua, Nam Phan, and Yue Yu. Deep neural operator enabled digital twin modeling for additive manufacturing. arXiv preprint arXiv:2405.09572. 2024c
Francesco Locatello, Stefan Bauer, Mario Lucie, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf. and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference on machine learning, pp. 4114-4124. PMLR, 2019.
Francesco Locatello, Michael Tschannen, Stefan Bauer, Gunnar Rätsch, Bernhard Schölkopf, and Oliver Bachem. Disentangling factors of variations using few labels. In Eighth International Conference on Learning Representations. OpenReview. net, 2020.
13

---

Published as a conference paper at ICLR 2026
Lu Lu, Pengzhan Jin, and George Em Karniadakis. Deeponet: Learning nonlinear operators for iden- tifying differential equations based on the universal approximation theorem operators. arXiv preprint ar
2019.
Lu Lu, Pengzhan Jin, Guofei Pang. Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via DeepONet based the universal approximation theorem of operators. Nature Machine Intelligence. (3):221-229, 2021.
Emile Mathieu, Tom Rainforth, Nana Siddharth, and Yee Whye Teh. Disentangling disentanglement in variational autoencoders. In International conference on machine learning, pp. 4402-4412. PMLR, 2019.
Marios Mattheakis, Pavlos Protopapas, David Sondak, Marco Di Giovanni, and Efthimios Kaxiras. Physical symmetries embedded in neural networks. arXiv preprint arXiv:/904.0899 2019.
Roberto Molinaro, Yunan Yang, Björn Engquist, and Siddhartha Mishra. Neural inverse operators for solving pde inverse problems. arXiv preprint arXiv:2301.1116/ 2023.
Christoph Molnar. Interpretable machine learning. 2020.
Phuoc Nguyen, Truyen Tran, Sunil Gupta, Santu Rana, Hicu-Chi Dam, and Svetha Venkatesh. Hy- pervae: Variational hyper-encoding network.
Daniel Obmann, Johannes Schwab, and Markus Haltmeier. Deep synthesis regularization of inverse problems. arXiv preprint urXiv:2002.00/55, 2020.
Geunseob Oh and Huei Peng. Cvae-h: Conditionalizing variational autoencoders via hypernetworks
and trajectory forecasting for autonomous driving. arXiv preprint arXiv:2201.09874, 2022.
Yong Zheng Ong, Zuowei Shen, and Haizhao Yang. IAE-Net: integral autoencoders for discretization-invarian learning. arXiv preprint 0rX1v:2203.05142, 2022.
David Pfau, James S Spencer, Alexander GDG Matthews, and W Matthew C Foulkes. Ab initio solution of the many-electron schrödinger equation with deep neural networks. Physical Review Research, 2(3):033429, 2020.
Maziar Raissi, Paris Perdikaris, and George Karniadakis. Physics-informi neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019.
Karl Ridgeway and Michael C Mozer. Learning deep disentangled embeddings with the f-statistic loss. Advances in neural information processing systems, 31, 2018.
Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong, In- terpretable machine learning: Fundamental principles and 10 grand challenges. Statistic Surveys, 16:1-85, 2022.
Omer Sagi and Lior Rokach. Ensemble learning: A survey. Wiley interdisciplinary reviews: data mining and knowledge discovery, 8(4))e1249 2018.
Rui Shu, Yining Chen, Abhishek Kumar, Stefano Ermon, and Ben Poole. Weakly supervised disen- tanglement with guarantees. arXiv preprint arXiv: 1910.0977, 2019.
Guoxiang Grayson Tong, Carlos A Sing Long, and Daniele E Schiavazzi. Invaert networks: A data- driven framework for model synthesis and identifiability analysis. Computer Methods in Applied Mechanics and Engineering, 423:1 16846, 2024.
Tapas Tripura and Souvik Chakraborty. Wavelet neural operator: a neural operator for parametric partial differential equations. arXiv preprint arXiv:2205.02191 2022.
Arash Vahdat and Jan Kautz. Nvac: A deep hicrarchical variational autoencoder. Advances in neural information processing systems, 33:1 33:19667-19679, 2020.
14

---

Published as a conference paper at ICLR 2026
Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M Benson. U- fno-an an enhanced fourier neural operator-base based deep-learning model for multiphase flow. Ad- vances in Water Resources, 163:104180, 2022.
Liu Yang, Xuhui Meng, and George Em Karniadakis. B-pinns: Bayesian physics-informed neu- ral networks for forward and inverse pde problems with noisy data. Journal of Computational Physics, 425:109913, 2021.
Özdogan Yilmaz. Seismic data analysis, volume 1. Society of exploration geophysicists Tulsa, 2001.
Minglang Yin, Enrui Zhang, Yue Yu, and George Em Karniadakis. Interfacing finite elements with deep neural operators for fast multiscale modeling of mechanics problems. Computer Methods in Applied Mechanics and Engineering. in press, pp. 115027, 2022.
Huaiqian You, Yue Yu, Marta D D'Elia, Tian Gao, and Stewart Silling, Nonlocal kernel network (NKN); A stable and resolution-imdeeendent deep neural network. Journal of Computational Physics, pp. arXiv preprint Stt1::2001.0221111 2022a.
Huaiqian You, Quinn Zhang, Colton Ross, Chung-Hao Lee, and Yue Yu. Learning deep implicit fourier neural operators (IFNOs) with applications heterogeneous material modeling. Computer Methods in Applied Mechanics and Engineering, 398:115296 2022b. doi: https://doi.org/ org///00.
1016/j.cmaa (t16/j.cma.2022... 15296.
Yue Yu, Ning Liu, Fei Lu, Tian Gao, Siavash Jafarzadeh, and Stewart Silling. Nonlocal attention operator: Materializing hidden knowledge towards interpretable physics discovery. arXiv preprint arXiv:2408. 07307, 2024.
Linfeng Zhang, Jiequn Han, Han Wang, Roberto Car, and E Weinan. Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics. Physical Review Letters, 120(14):143001,, 2018
Lu Zhang, Huaiqian You, Tian Gao, Mo Yu, Chung-Hat Lee, and Yue Yu. Metano: How to trans- fer your knowledge learning hidden physics. Computer Methody in Applied Mechanics and Engineering, 417:1 16280, 2023.
Rui Zhang, Qi Meng, and Zhi-Ming Ma. Deciphering and integrating invariants for neural operator learning with various physical mechanisms. National Science Review, 11(4):nwad36, 2024.
15

---

Published as a conference paper at ICLR 2026
3.6
004
2
0.2
9.2
0.02
0.04
-6 -2
0.8
-0.4
-02
0.0
0.4
-0.05
-0.22
0.00
0.25
0.500
0,075
D100
Figure 2:
MMNIST scatterplot with DisentangO-2 and 3a
1:
left:
(3tt 1,
100, data 18.81)), middle: (Bkr 10, Bats
10, data 16.94%), right: fully unsupervised
DisentangO-2 without classification loss (5ks 100, data error 12.65%).
A IDENTIFIABILITY ANALYSIS
A.1 1
PROOF OF THE MAIN THEOREMS
We first provide the proof of Theorem
Proof: With equation 3.16 we have:
Poj = Po(f:0]s > (o)(0) - -
c Pals - P
Note that the parameter 0 varies with the change of b. Per the data generating process in equation 3.6 the distribution of b is invariant to f. Therefore, the distribution of 0 also invariant to f:
Pa Pols 2 ( " Vf F
Denoting r := Gj 9 it is the transformation between the true 0 and the estimated one, and it is invertible and invariant with respect to f.
We proceed to derive the relation between and &: since 0 = r((),, with the invertibility assumption θ 71 (2) and 0 - (2). we obtain:
H((0) = 7 o r()) = 7 Moroo (2).
Denoting h : Horo -- is the transformation between the true latent variable and the estimated one, and it is invertible because r, H. and H. are all invertible.
We now show the proof of Theorem
Proof: With the independence relation assumption, we have
(=)b (z)))(z)), (2)6(21) = Il P2.1((2))
Denoting क. := log P2, l.. it yields:
log logP2)((2(0)
Σ (1(2z,))) log logga)(((b)) = Σ ().
With the change of variables we have
|Ja- + Σ
Σa( gi(zi b) log j
Σ ai(2,, b),
Pzb Pb(2)bb PœIB
where |Jn .| stands for the absolute value of the Jacobian matrix determinant of h -1 Differentiating the above equation twice with respect to and ēg. h * q. yields
@Lqc(zi, b) Oz; 0z: agi(z,, b) 822;
8a log Jn 0.
Σ
+
+
(A.I)
Oz? Dā, 0āg 8z, d2
02
To show the identifiability, rewrite the Jacobian Jn. as:
0z
Jn
02
The invertibility results shown in Theorem I Indiicates that is full rank. Next, we will use the linear independence assumption to show that there exists one and only non-zero component in each Oz
row 02
16

---

Published as a conference paper at ICLR 2026
Taking b = b b..... .b in equation A.1
and subtracting them from each other, we have
de 8 q:2 b')
8% (2, b²)
Oz, 0z, (dgi(as,b) b')
(q, b²)
82
Σ
0,
0zf
0Ey dz,
dz
8z.
where j 1, 1,.... 2dz. With the linear independence condition for w, this is 2d x 2dz linear system, and therefore the only solution
Oz dz:
0²zz
0.
0,
0&g 2ag
05
for 1,.., ,dz. The first part implies that, for the i-th row of the Jacobian matrix Jn have 0z,
+ 0 for at most one element E {1,'.. ;d)), hence z is identifiable up to permutation and component-wise invertible transformation.
0&
A.2 FURTHER DISCUSSION ON THE ASSUMPTIONS
Herein, we provide additional discussion the validity and emprical validation for Assumptions
As seen in the proof above, Assumptions 1 (Density Smoothness and Positivity) and 2 (Invertibil- ity) are required guarantee that there exists smooth and injective mapping h H Ā-11 from the ground-truth latent embedding z to the learned embedding 2. Furthermore, the smoothness assumption further makes it feasible to take derivatives of x with respect to 2, which supports the permutation-wet wise identifiability proof for Theorem 2. Here, we note that the smoothness assumption may possibly be relaxed to C Assumptions 3 (Conditional Independence) and 4 (Linear Indepen- dence) are needed to show that the Jacobian of h has one and only one non-zero component for each column. Without these assumptions, it is possible that the data from different lack variability.
While the first three assumptions are common in many VAE architectures, the last assumption is plausible for many real-world data distributions. For instance, when the prior on the latent vari- ables p( (b) is conditionally factorial, where each element z; has univariate exponential family distribution given conditioning variable b:
b = II Q1(2)) (b) exp
Σ (a)
where Q is the base measure, (b) is the normalizing constant, Tújj the sufficient statisties, and Ào, are the corresponding parameters depending on This exponential family has universal approximation capabilities. Additionally, we note that this distribution is conditionally independent with
k Σ (cs
q: lz.((.(zz)) log Z.(b))
and the linear independence indicates that the matrix formed by
(xx 6²) - w2z, b²) = (T'()(b") - A(b")) T>(b(") - x(b")) , -= 1,
has full rank 2d
In the fully supervised case, the conditional independence and linear independence assumptions automatically guaranteed by picking proper distributions p(z(b) when designing the VAE architec- ture. In the cemi-supervised and unsupervised cases, one can also validate these assumptions when the true values of is given on some tasks, by inferring an empirical distribution of p(x(b) from these tasks. To investigate such a capability, the additional synthetic experiment Appendix C.4 we consider an unsupervised setting, estimate p( p(z(b) from the trained model, and check the linear independence condition by calculating the vector in equation 3.15 for each task b") and forming an S x 2d, matrix from all tasks. When the rank of this matrix is 2d it means that can select 2d +1 tasks from them with sufficient variability, such that the linear independence condition is satisfied. As demonstration, in Appendix C.4 we validate this assumption synthetic dataset and show that the identifiability can be largely achieved with Gaussian distributions of distinct means and variances.
17

---

Published as a conference paper at ICLR 2026
0,0 200,00
-0.10
4 (100100
0.55
(1100
(19011))
11,300
Ry
2. 0..00
0.3
1. 40.00
- (1001000)
2,9 01.00
2, (190.199)
1.5
0.4
0.35
0.25
0.5
0.2
100 0.3
20
40
80
Figure 3: MMNIST unsupervised against Ba with DisentangO-2 (left) and DisentangO-1 15 (right). By comparing HJ (solid lines) with 3a 100 (dashed lines), increasing Sa forces the latent factors to maximize contained information and turn decreases MI, thus encouraging disentanglement.
0.54
-10
MI
0.44
A
Figure Unsupervised MI score against Ber DisentangO-2 in heterogeneous material learning.
Diu
2
Din 1
Din2
Figure 5: Latent traversal of DisentangO-3 in unsupervised heterogencous material learning, where the three latent dimensions control the border rotation between the two segments (top), the relative fiher orientation between the segments (middle), and the fiber orientation of the top segment (bottom), respectively. Legend indicates fiber orientation ranging from to n.
B ADDITIONAL EXPERIMENTAL DETAILS, RESULTS AND DISCUSSION
We provide additional details in training and baseline models, as well as more results as a supplement of Section 4
TRAINING DETAILS
In all experiments, adopt the Adam optimizer for optimization and n-layer DisentangO model. where n = 8 in the first experiment and n = 16 in the second and third experiments due to increased complexity. For fair comparison across different models, tune the hyperparameters,
18

---

Published as a conference paper at ICLR 2026
including the learning rates, the decay rates, and the regularization parameters, minimize the validation loss. Experiments are conducted on a single NVIDIA Tesla A100 GPU with 40 GB memory. A pseudo algorithm of all three scenarios is summarized in Algorithm
Algorithm A pseudo algorithm of DisentangO.
Denote data reconstruction loss Ldata, task-wise NO parameter reconstruction loss Lyecon. KL loss L and sqmii-sppervision loss Laemi as:
1
Litaa
- E(log(p(u)(f,0')) Lrecon
- -Eggzz logp(0"|22))
S
S 1=
171
8
1 LKL -5 17
Σ
DeL ("==")0"))||(("")} Leemi = Lo(z",
2: SC1: Supervised/SC3: Unsupervised
3: The total loss is comprised of the data loss, the NO parameter reconstruction loss, and the KI. loss: Lioes = BaLdata + Liecon + -KLLKKL
4: SC2: Semi-supervised
5: The total loss is comprised of the data loss, the NO parameter reconstruction loss, the KL loss, and remi-supervisedd loss: Lioss BaLdata + Lieeon + BKLLKL + Seie
B.2 SUPERVISED FORWARD AND INVERSE PDE LEARNING
The parameter of each baseline is given in the following, where the parameter choice of each model selected by tuning the number of layers and the width (channel dimension), keeping the total number of parameters on the same magnitude.
MetaNO: We use 8-layer IFNO model with the lifting layer the adaptive layer. We keep the total number of parameters in MetaNO the same as the number of parameters used in forward PDE learning in DisentangO.
NIO: We closely follow the setup in Molinaro et al. (023), where two neural operators (DeepONet and FNO) are stacked together realize the operator-to-function intuition. The first operator maps multiple solution functions to set of representations (which can be seen as an analog of eigenfunctions), and the second operator infers the underlying parameter field from the mixed representations. As NIO requires the solution field as input, it cannot be used as a forward solver. Hence, NIO only solves the inverse PDE problem, and it can only be applied to the fully supervised setting. Specifically, as the encoder for the branch net and a fully connected neural network with two hidden layers of 256 neurons the trunk net, with the number of basis functions to 50. For the FNO part, we use one Fourier layer with width 32 and modes 8, as suggested in Molinaro et al. (023).
use four convolution blocks
mapping, we consider the inverse optimization procedure following Lee al. (2024), and develop two-phase pro- cess to solve the forward and inverse problems sequentially. In the first phase, we construct the forward mapping from the loading field and the ground-truth material parameter to the corresponding solution u as: G b, OPNO (z) u(x)). This can be seen as an analog of the forward solution operator G in our setting. Then, with the trained FNO as a surrogate for the forward solution operator, we fix its NN parameters PNNO, and use it to- gether with gradient- inverse solver. Specifically, given set of loading/solution data pairs {(fi, u.) J.), we start from a random guess of the underlying material parameters (typically chosen as the aver- age of all available instances of material parameters for fast convergence), and minimize the difference between the predicted displacement field from FNO and the ground-truth one:
FNO: Since FNO is originally designed as
optimization to solve for the optimal material parameters as an
6" = argmin, - OFNO
As the FNO parameters are fixed, we can back propagate this loss and optimize the input material parameters in an iterative fashion. We adopt 4-layer FNO with width 26 and
19

---

Published as a conference paper at ICLR 2026
modes 8. For the forward model, in addition to the loading field and the coordinates as input, we also concatenate the ground-truth material properties to form the final input. For the inverse model, employ an iterative gradient-based optimization to solve for the optimal material parameters for physics discovery. For fair comparison, in terms of the averaged per-epoch runtime, we report the sum of both the forward and inverse solvers. The averaged per-epoch runtimes for the forward solver and the inverse solver are 6.2 seconds and 2.9 seconds, respectively, accounting for the total per-epoch runtime of 9.1 seconds in [
UPNO: The 2D U-FNO model extends the Fourier Neural Operator (FNO) architecture by incorporating U-Net-style skip connections. The network consists of six spectral convolu- tion blocks, each combining a global Fourier operator and a local 1 x convolution. In the later three layers, additional skip-enhanced feature extraction is provided by U-Net blocks. The input consists of spatial features fzz y), concatenated with coordinate embeddings (z, y) and 5 material parameters. The input is lifted higher-dimensional latent space via linear layer of size [8, 20). Each of the six layers performs the following composite operation:
1/+1 ( + W3(0y) + (1;()))),
where F3 is a spectral convolution layer (2D Fourier transform, mode truncation, linear transformation, and inverse transform), W, is a x convolution, and Uj is a U-Net block (included only in layers 4 to 6). After all six layers, the output is projected through two fully connected layers of sizes [20, 128] and [128, 2] to produce the final 2-channel output.
WNO: The Wavelet Neural Operator (WNO) consists of 4 stacked wavelet kernel integral layers. The input is first lifted to a high-dimensional representation using a fully con- nected layer of size [8, 27]. Each of the 4 layers combines a learned local operator W (implemented as 1 x convolution) with non-local wavelet-based integral operator K (via continuous wavelet transform). The architecture follows: The input fz. y) is first augmented by concatenating it with the positional grid coordinates (1r, y) and additional 5 material parameters. This augmented input then lifted into higher-dimensional space via a fully connected layer that maps the input to a hidden width of wi witth. The core the architecture consists of 4 wavelet-based layers, the output is projected back to the desired output space using two linear layers of sizes [27, 128] and [128, 41 41 2]. Mish activations are applied after each intermediate layer except the last projection layer.
CAMEL: The architecture consists of two networks, V-Net and C-Net
G(f(z); 0, w) = c((f(x); 8) + w The score network Vg is a 5-layer MLP with width 320 and Tanh activations, with layer sizes (d, 320, 320, 320, 320, r] The coefficient network co a 3-layer MLP with width 128 and Tanh activations, with layer sizes (d, 128, 128, 1),, where d is the size of the flatten f(z), and 5
T ToTo(f(a (f);
(B.1)
SMDP: Since we do not have an explicit physical operator to evaluate (2, we use only the score network, implemented as a 6-layer MLP with layer sizes [128, 512, 256, 128, 64, 32, dz], where de denotes the dimension of the latent space.
PIANO: We use the ground-true -ruth material properties the physical-invariant invariant embedding. The loading field is first processed by a lifting layer, while the material parameters are passed through attention module implemented 32. The outputs of the lifting layer and the attention module are then concatenated and fed into a convolutional layer with width 26, followed by a 3-layer IFNO with width 26 and modes 8. For both the forward and inverse solution procedures, we adopt the same settings as in the FNO baseline.
3-layer MI.P with hidden dimension
We directly take the VA implementation from Lingsch al. 2024 and de- fine the encoder, the VAE encoder and the decoder as 4-yyyer MLPs with hidden dimension 96 and silu activation function.
VAErE::
FUSE: For the forward model, we take three Fourier layers in addition to the first band- limited lifting layer that increases the dimension of the parameters and performs an in- verse Fourier transform. On the other hand, the inverse model maps the functional input to the parameter space by employing a concatenation of two Fourier layers and a band- limited forward Fourier transform that generates fixed-size latent representation of the
20

---

Published as a conference paper at ICLR 2026
input function, followed by flow flo-mmtcchin posterior estimation (FMPE) that maps to the final parameter output. All Fourier layers have latent width of 32 and 8 modes, while the FMPE flow has layers of width 360.
FUSE-f: Since the original FUSE model assumes a constant loading field, it cannot handle situations where the imput loading field changes that create multiple instances of PDE system. We therefore create FUSE variation (denoted as "FUSE-f") that takes a concate- nation of both the displacement field and the loading field as input to the inverse model. For the forward model, we concatenate the loading field to the output of the first band- limited lifting layer and subsequently use a one-layer MLP to map it back to the original dimension. All other settings are the same the original FUSE baseline.
VAE: We use a 2-layer MLP of size [1681, 136, 30] as the encoder, and another 2-layer MLP of size [30. 136, 1681] as the decoder, with the size of the bottleneck layer being 30.
Conv We use convolutional layer with 136 kernels of size 3 x 3 with stride of 2 pixels and a fully connected layer of size [59976, 5] as the encoder, and a fully connected layer of size of [5, 59976] and a transposed convolutional layer with 2 kernels of size 3 x 3 as the decoder.
3-VAE: The parameter choice of the 3-VAE baseline is the same as the VAE baseline, except that we tune the /3 hyperparameter.
Model-agnostic architecture. The proposed DisentangO architecture is model-agnostic and can incorporate any neural operator as the forward solver, including FNO, UFNO, and WNO. This flexi- bility stems from DisentangO's 's design, which only requires designating the lifting layer as the task- wise adaptation layer and adding the VAE structure for latent disentanglement (i.e., the encoder and the first decoder in Figure []. DisentangO's overall performance scales directly with the underlying neural operator's capabilities. As long as the chosen NO achieves comparable forward prediction accuracy to IFNO, DisentangO will maintain similar performance, since (I) the upper bound of forward prediction accuracy depends on the base neural operator, and (2) the inverse prediction accuracy depends on how well the latent factors are encoded the bisee lifting layer param- eters. We select IFNO for three key advantages: (1) parameter efficiency: the layer-independent parameter setting significantly reduces trainable parameters compared to alternatives, (2) theoretical guarantees: IFNO is provably universal approximator PDE solution operators, and (3) natural seamless compatibility with MetaNO's s lifting layer approach for task-wise adaptation. If the IFNO part is replaced with other neural operators such the considered baselines of FNO, UFNO, and WNO as in Table [ the forward prediction accuracy will decrease (cf. the data test errors in Table D. meaning that the upper bound of DisentangO's forward prediction accuracy will decrease. Additionally, the inverse prediction accuracy (i.e., the test error in Table []] will likely drop as well, because the degraded forward prediction accuracy typically indicates degraded encod- ing of latent information in the lifting layer, which will negatively impact the inverse modeling even the VAE part can reconstruct the lifting layer parameters perfectly. As IFNO performs the best in forward prediction and requires the least amount of trainable parameters, we choose IFNO as the forward prediction backbone in DisentangO.
integration:
To demonstrate DisentangO's compatibility with alternative neural operators, we replace IFNO with UFNO in the DisentangO architecture. The performance comparison is presented in Table 4
Table 4: Test errors and the number of trainable parameters in experiment 1. Bold number highlights the best method.
Models
#param
per-epoch time (s)
Test errors
data
2 (SCT)
DisentangO
697k
12.2
1.65%
4.63% %
MetaNO
296k
9.8
1.59%
MetaUFNO
304k
11.4
3.55%
DisentangO-UFF UFFNO
688k
14.2
6.61%
9.85%
UFNO
720k
21.2
7.61% %
11.23%
21

---

Published as a conference paper at ICLR 2026
Figure MMNIST latent traversal based on a randomly picked loading field as DisentangO-2 input.
B.3 MECHANICAL MNIST BENCHMARK
We demonstrate the latent traversal based randomly picked loading field DisentangO-2 input. One can clearly see that the digit changes from "6" to "O" and then "2" from the top left moving down, and from "6" to "1" and then "7" moving to the right. Other digits are visible well such as "7", "9", "4" and "8" in the right-most column. This corresponds well to the distribution of the latent clustering in Figure 2 We also provide two exemplary MMNIST latent interpretations of the learned DisentangO-1e 15 in Figures 7 and 8
B.4 LATENT VISUALIZATION AND PHYSICAL INTERPRETATION
As the hidden parameter field 1 of the PDE generally not accessible, especially in the semi- supervised or unsupervised settings, one cannot directly reconstruct 8 from the learned latent vari- ables z. Through disentangled representation learning, the latent variables z are anticipated to con- tain critical information of 6", but there does not exist direct and explicit mapping from to 6", As a result, one cannot directly reconstruct 5 from 2. However, there are several tricks one can play with to obtain meaningful interpretations from the learned 2. On one hand, one feed randomly selected input into DisentangO and manually define a desired z in the latent space and perform a for- ward pass using the trained DisentangO. b can then be visualized from the output, as demonstrated
22

---

Published as a conference paper at ICLR 2026
04-
0.4
9.4
Dinersion 30
0.2-
0.2
0.2
Bimension
-02-
-22
-04-
-0.4
-9.4
0.5
0.5
9.25
0.75
0,0
0.4
Dimendion 9
Dimension
Dimendion2
04-
0.4
0.4
0.2-
0.2
Bimendio
-02
-0.2
-04-
-0.4
-0.4
0.4
-0.4
Dimerrion 3
Dimonsion
Dimonsson
0.4
0.4
0.4
Dinersioo 30
0.2
0.2
0.2
-022
-0.2
-.4
0.4
-0.4
-0.6
-0.4
-0.2
0.4
-059
0,00
0.25
Diirendion
Dimension
Dimension
04.
0.4
0.4
Dironsion 20
0.2-
0.2
0.2-
Dimernton
-0.2-
-94-
-0.4
-2.4
0.4
02
Dimonsion 11
Dimonsionn..
0.4-
02
Dineriinn 10
02-
-02
-0.2
-0.4
-24
D'mansion1s
Dimers on
Figure 7: Exemplary MMNIST latent interpretation DisentangO-15. dimension 10 encodes the information for digit '8', evidenced by the fact that all y < -0.4 regions on the scatterplots contain only digit '8'.
Figure O in the MMNIST experiment. On the other hand, one can also train a simple MLP and construct mapping from to b", provided that 6" is available. With this mapping hand, one can then traverse and visualize what each dimension of z controls. This is demonstrated Figure in the third experiment.
B.5 FURTHER DISCUSSION
Probabilistic latent space. We use a variational (probabilistic) encoder rather than a deterministic map for two reasons:
23

---

Published as a conference paper at ICLR 2026
Olmrrrzionn
0.5 Dimension 9
0.50
-025
9.25
0.75
0.0
0.4
Dimension
Dimondion2
Dmersion2
Dimenabo13
0.4
62
04
6.s
Dimersion 3
Dimonsion
Dimonsion
Dmandonn12
Dmmenin 12
-0.6 -0.4
02
-0.4
0.4
0,00 0.25
Dimrendion
Dimension
Dimension
Omassan 12
Dimeriion
0.4
0.2
54
0.2
Dimoriiinn 9
Dimendinn 10
Dimondionnn
Dimesoon12
Dimunsion 12
Dimansion1
Dimerts on
Figure 8: Exemplary MMNIST latent interpretation DisentangO-IS: dimension 12 encodes the information for digit '3', evidenced by the fact that all y < -0.45 regions on the scatterplots contain only digit '3',
Regularization and stability of the inverse map. The inverse operator in ill-posed PDE settings is highly sensitive: small perturbations in the solution u can correspond to large changes in the inferred parameters. The VAE's s KL term imposes distributional prior the latent variables, preventing the encoder from collapsing to arbitrarily sharp or unstable mappings. This yields smoother, more stable inverse that generalizes better under noise and limited data.
Identifiability and disentanglement. The identifiability results we reference (e.g., up to component-wiss invertible transformations) rely the latent distribution having simple, factorized prior. The VAE provides exactly this structure: a normalized, independent latent prior that constrains the representation and supports the theoretical guarantees.
24

---

Published as a conference paper at ICLR 2026
Scalability, Scaling interpretable neural operators to complex, high- sigh-dimensional PDEs is critical for broader impact. Our work is structured to first establish the foundational capabilities of DisentangO in controlled settings where disentanglement and representation quality can be rigorously verified. Notably, our work tackles scenarios with partially or fully unknown physics: the second eepedddeeemmee demonstrates DisentangO's ability to disentangle and identify latent codes when only partial knowl- edge is available, and the third experiment aims to mimic a real-world fully unsupervised scenario where the physical parameters are entirely unknown, yet DisentangO successfully extracts inter- pretable latent factors. DisentangO is designed to be inherently scalable for several key reasons: (1) Modular architecture: DisentangO inherits the computational efficiency of the underlying neural operator backbone (IFNO in our implementation), which supports batched, GPU-accelerated infer- ence in high dimensions. (2) Constant latent dimension: the latent code dimensionality remains low regardless of spatial resolution or system complexity, enabling practical inverse inference even for complex systems. (3) Linear scaling: the latent inference network scales linearly with the number of PDE instances, independent of spatial dimensionality. (4) Plug-and-plat design: DisentangO can incorporate any state-of-the-ar neural operator as its backbone, inheriting its scalability properties. In essence, the scalability of DisentangO is governed by the scalability of the neural operator used for forward modeling. As long as the NO component performs well and the training data spans sufficient variability in physical system parameters, DisentangO remains applicable.
experiment
Encoding b in lifting layers. Our choice to encode the parameters b in the lifting layers 0p is jus- tified through three complementary perspectives. (1) Theoretical expressivity: as demonstrated in MetaNO (Zhang et al. 2023), varying only the lifting layer across tasks is sufficient to universally approximate a wide class of parametric operators. DisentangO inherits this universal approximation property while maintaining a meta-learning framework where the core iterative Fourier layers serve as a shared meta- meta-operator across tasks. (2) Representation identifiability: restricting adaptation to the lifting layer significantly enhances the identifiability of latent representations, a critical require- ment for disentangled inverse learning. When adaptation is allowed in deeper layers, the mapping between latent codes and operators becomes more diffuse and less interpretable. (3) Empirical validation: our experiments across diverse PDE types confirm that this design choice enables Dis- entangO to: (a) accurately model task-specific operators, (b) recover interpretable physical factors, and (c) generalize to unseen parameter combinations and interpolation tasks.
Interpretability beyond parameter reconstruction. Our approach provides interpretability the level of identifiable physical factors, which is often more meaningful than raw parameter recon- struction. Many physical parameters are inherently non-identifiable from observational data alone (e.g., material properties in regions with zero stress). Our method identifies the factors that actu- ally govern system behaviors. Moreover, our experiments demonstrate clear interpretable patterns: (1) MMNIST (Figure 6): latent traversal reveals smooth transitions between digit patterns, show- ing the method captures meaningful morphological variations, (2) Heterogeneous materials (Fig- ure 5): each latent dimension controls specific physical aspects: border rotation, relative fiber ori- entation, and segment-wiss fiber orientation, (3) Semi-supervised results (Figure 2): latent factors successfully cluster different digit classes without explicit supervision. Our theoretical results on component-wise identifiability (Theorems [ and guarantee that the learned factors correspond to true generative factors up to invertible transformations. This provides theoretical grounding for interpretability claims. In terms of practical utility, even without exact b reconstruction, the disentangled factors can enable system classification and similarity assessment, design space explo- ration through latent traversal, transfer learning to new parameter regimes, and anomaly detection in physical systems.
Quantitative interpretability metric. The evaluation of interpretability depends critically on the availability of ground truth, which varies across our three practical scenarios, i.e., supervised, semi- supervised, and unsupervised. In the fully supervised setting corresponding to experiment 1, When true generative factors are available, we use quantitative metrics. The interpretability metric the latent supervision test error (cf. Table []. which directly measures how well the learned latent variables z recover the true physical parameters b. In the semi-supervised setting corresponding to experiment 2, the goal of interpreting physics is to discover the underlying microstructure gov- erning the deformation. Under this setting, since the true generative factors are not available, one cannot use any closed-form metric to evaluate interpretability. However, since the ground-truth mi- crostructure generation is controlled by MNIST digits, successful disentanglement should reveal this digit information. We thus follow the standard evaluation methods in disentangled representation
25

---

Published as a conference paper at ICLR 2026
learning and investigate if the learned latent variables indeed contain the digit information via latent traversal, and Figure confirms that our learned latent variables capture the underlying digit struc- ture, demonstrating physical interpretability through microstructure discovery. In the third setting of unsupervised learning, without ground truth labels, we evaluate interpretability through latent demonstrates that our method discovers meaningful physical factors: border rotation between segments, relative fiber orientation between segments, and fiber orientation of individual segments physically meaningful properties for understanding the microstructure mechanism.
traversal analysis. Figure [
Physical insights across different PDE types. Our method is designed for parametric PDE fam- seeenaroosios where underlying physics follows the same governing equations with varying pa- rameters. This represents a very common and scientifically important class of problems in computa- tional physics and engineering. While our current framework focuses on parametric variation within PDE families, the core architectural principles can extend to different PDE types. For example, in multi-physics systems, our approach could handle different but related physics (e.g., heat conduction vs. diffusion-reaction) by learning shared latent factors capturing common physical principles while using task-speciff decoders. Extending heterogeneous PDE types represents exciting research frontier that would require modular architectures for different equation types, shared representation learning across physics domains, and physies-informed constraints for cross-domain transfer.
Accuracy-nterpreeaaliity trade-off. As discussed, MetaNO represents the forward prediction upper-bound for Disentango. The accuracy gap between the is primarily due insufficient latent dimensions relative to data complexity. Taking experiment for instance, the data generation process may involve more than 30 true generative factors due to numerical solver noise and system complexity. Table cleaarl shows the error decreases from 25.18% to 5.28% as we increase latent dimensions from 2 to 30 trend that would continue with higher dimensions. We stopped at 30 dimensions because: (1) 5.28% error is reasonably accurate for practical use, and (2) limited com- putational resource. In practice, users should gradually increase latent dimensions until convergence to MetaNO's performance, ensuring the model capacity matches the true generative factors. Disen- tangO prioritizes joint modeling with interpretability over pure forward accuracy. For applications requiring only forward prediction, specialized NOs remain more accurate. However, for scientific discovery where understanding mechanisms essential, the slightly degraded accuracy with full interpretability represents a favorable trade-off.
C DATA GENERATION
C.I EXPERIMENT 1 SUPERVISED FORWARD AND INVERSE PDE LEARNING
we consider the constitutive modeling of anisotropic reirfrreifor hyperelastic materials gov- erned by the Holzapfe--Gasee--Ogae (HGO) model, whose strain energy density function can be written as:
E
E lnJ
(I - 2) 2(1+ + v]
v) h1 (cxx()(a))) ts((( 2) 6(1 2n)
E
J
2 J).
(C.1)
2k2
where (-) indicates the Macaulay bracket, a, Ki and ka are the fiber angle, modulus and expo- nential coefficient, respectively, E denotes the Young's 's modulus of the matrix, V is Poisson's 's ra-
tio, and S(a) describes the fiber strain of the two fiber groups, S(a) [(a) 1-1a-1-1 T, is the tt invariant of the right Cauchy-Great tensor C, In tr(C) and I, n Cn)(a)).) with In this context, the data generation process is controlled by sampling the material set (E, v, ki, k2, a)) and the latent factors can be learned consequently in a supervised fashion. The physical parameters are sampled from ls 20 0.3), and a 21|=/10, x/2]. To generate the high-fidelity (ground- truth) dataset, we sample 220 material sets, which are split into 200/10/10 for training/validation// respectively. For each material set, we sample 50 different vertical traction conditions (ar) on the top edge from a random field, following the algorithm in Lang & Potthoff (201 10); Yin et al. (2022).
na(a) =
[900 100) _/2 4/00.001, 0.1), E
[[055001, 0.6001],
(y(x) is taken as the restriction of 2D random field, ((x) = F 1/2
on the top
edge. Here, [(x:) is Gaussian white noise random field on R2, 7 (wf w represents
26

---

Published as a conference paper at ICLR 2026
correlation function, and W1, W2 the wave numbers and y directions, respectively. Then, for each sampled traction loading, we solve the displacement field on the entire domain by minimizing the potential energy using the finite element method implemented in FEniCS (Alnas et al.] 22015). Sample data of the obtained dataset illustrated in Figure In this setting, the model takes as input the padded traction field and learns predict the resulting displacement field.
b
b2
33
U1 x
U2 x
U3 x
U
U2y
U3
Figure 9: Illustration of the HGO data, loading displacement pairs of three materials = 1, 2,3. Top: three instances different loadings (2), which corresponds to different ft. Bottom: corresponding displacement solutions u", illustrating the impacts of system (b") variability in solution operators.
C.2 EXPERIMENT MECHANICAL MNIST BENCHMARK
Mechanical MNIST is benchmark dataset of heterogeneous material undergoing large deformation, modeled by the Neo-Hooken Hookean material with a varying modulus converted from the MNIST bitmap images (Lejeune 2020). It contains 70,000 heterogeneous material specimens, and each specimen is governed by the Neo-Hookean material with varying modulus converted from the MNIST hitmap images. We illustrate samples from the MMNIST dataset in Figure 10 including the underlying microstructure, two randomly picked loading fields, and the corresponding displacement fields.
C.3 EXPERIMENT 3 UNSUPERVISED HETEROGENEOG MATERIAL LEARNING
We generate two sets datasets in this case, varying the material microstructure in the following two ways.
Varying fiber orientation distribution. We generate the samples by controlling the predefined parameters of Gaussian Random Field (GRF). With the GRF sharpened by the thresholding, the values to binary field represent two distinct fiber orientations. The binary field is smoothed using windowed convolution. To address boundary conditions, the matrix is padded with replicated edge values, ensuring that the convolution works uniformly across the entire grid. After the smoothing process, the padded sections are removed, and the remaining field used to construct the fiber field. We use two fixed fiber angles 3. 2n for the corresponding binary field. We generate 300 material sets, each with 500 loading/displacemen pairs, and divide these into training, validation, and test sets in 200/50/50 split. Exemplar samples from this dataset illustrated in Figure [T] These samples demonstrate the variability b, which critical for the latent variable identifiability in unsupervised learning settings, proved in Theorem 2
Varying fiber orientation magnitude and segmentation line rotation. Instead controlling the fiber orientation angles the binary field as constant values, sample the orientation distri-
27

---

Published as a conference paper at ICLR 2026
(a) MMNIST material microstnuchure
O 2
3
4
5
6
6
9
(b) Loading #38 and the correpoondigg displacement field
(c) Loading #123 and the coresponding displacement field
Figure 10: Illustration of exemplar MMNIST samples in the semi-supervised scenario. (a): material parameter field corresponding different b" (b): displacement fields (second row) ugs corresponding the same loading field (first row) fas. (c): displacement fields (second row) us corresponding to the loading field (first row) f138-
nase
6o0
-coes
coco
cuse
-0.00
uses
fix
f2x
f3x
fly
fzy
f3y
1.00
0.00
b1
-0.00
b2
coe
-o09
L0B
100
33
-0a1
.000
440
.400
11x x
U2
u3z x
U y
U2 y
U3
Figure 11: Illustration of fiber orientation magnitude and segmentation line rotation. Upper: three different loading instances of f.. Bottom left: three different microstructure instances of b", Bottom right: correspond- ing solution fields u.
bution consisting of two segments with orientations o' and a22 each side, respectively, separated by a line passing through the center. The values of a; and at are independently sampled from a uni- form distribution over [0, 2Tr), and the centerline's rotation sampled from [0, w]. We generate 300 material sets, each with 500 loading/displacemmnt pairs, and divide these into training, validation, and test in 200/50/50 split.
28

---

Published as a conference paper at ICLR 2026
Loading and displacement pairs for microstructure. After generating the specimens with varying fiber orientations, we feed the (x) as the a(z)) to the HGO model, and keep the material property set { v, 11, {2} as constant. For each microstructure, we randomly sample loading and displacement pairs for each microstructure from the previous step.
The loading in this example is taken the body load, f(x). Each instance is generated as the restriction of 2D random field, Φ(x) FF (y (z)) Here, r(x) is Gaussian white noise random field on 122,, 7 (w] + w represents correlation function, w1 and w2 are the wave numbers on and directions, respectively, and F and F denote the Fourier transform and its inverse, respectively. This random field is anticipated to have zero mean and covariance operator C = (--)-22, with A being the Laplacian with periodic boundary conditions on [0. 2] and we then further restrict to S2. For the detailed implementation of Gaussian random field sample generation, we refer interested readers to Lang & Potthoff (2011a). Then, for each sampled loading field fz( and microstructure field (z),, we solve for the displacement field u, (z) on the entire domain.
C.4 EXPERIMENT 4 SYNTHETIC DATASET FOR IDENTIFIABILIT DEMONSTRATION
We demonstrate using experiment 4 that the disentangled latent factors from DisentangO correspond the true generative factors. We generate synthetic data following the process in equation C.2 We work with latent variables z of dimension 2 and sample from ~ ((P, II IR22 where for each microstructure b, we sample Pb ([-4, 4] and σb [1, 10] and
4/00, E RR
M ~ 10, 1] _2 A [1, 21, 31, 411T M E 18x2 b u E RR, W U[0 1] R 4x1 1 ~~ 00 R
= Az Wo a (0 02) +1.
(C.2)
(C.3)
We generate 700 data pairs of (z,0) corresponding to 700 tasks, and for each task, we generate 200 samples of (f, u) pairs
d
3
5
MCC
0.7836
0.8013
0.8237
0.9121
R (ank(w)
0.5673 3
0.6026
0.6473
0.8242
4
4
Table 5: The MCC and H² scores evaluate identifiability the synthetic dataset in experiment 4.
We conduct experiments on the synthetic dataset and split the tasks into d/100/500 for training, validation and test, respectively, with d chosen from the set of (3, 5, 7,9) to investigate the effect of the number of training tasks on the identifiabiliyy. In particular, we measure the component-wise identifiability of the latent variables by computing the Mean Correlation Coefficient (MCC) and the coefficient of determination R² scores on the test dataset, and report the results in Table 5 where observe monotonic growth on both MCC and R²2 with the increase of the number of training tasks d. Figure 12 illustrates the alignment on the test dataset between the true generative factors and the disentangled factors from DisentangO in the case = To verify Assumption 4, we calculate the rank of the matrix w(z.80 b). Specifically, for each 6, with the VAE encoder, we
Oq (q, b')
and (2, b))
calculate the fb and σb. sample z, and calculate the corresponding
Oz,
Oz?
where 1.....d... dz,J = ....d.
29

---

Published as a conference paper at ICLR 2026
True vanable Zo
True
Disentangled variable
Disentangled variable
True variable 25
True variable 2:
Disentangled variable
Disentangled variable
Figure 12: The scatter plot of the true generative variables 2 and the disentangled latent variables å from the synthetic dataset with d = in experiment 4.
30


Reviews:
[{"review_id": "rFxIIImyzs", "reviewer": "Reviewer_3g1H", "summary": "This paper introduces a methodology for learning solution maps of both forward and inverse problems simultaneously with a single neural operator architecture. This is accomplished with a hypernetwork structure. The type of physical problem considered is a family of partial differential equations with forcing and coefficients. The authors use a neural operator to map forcings to solutions for fixed coefficient (supervised). The trained neural operator for this map will depend on the coefficient. So, the authors then train a hypernetwork to map the trained neural operator's parameters (which depend on the coefficient) back to a representation of the coefficient itself (unsupervised). This is the inverse problem part. The proposed method is applied to various benchmark problems in computational mechanics and its accuracy and interpretability are assessed are compared to alternative methods.", "questions": "Questions:\n1. I find Figure 1 hard to understand. Why are all three input fields $ f $ in the figure the same? What is \"multitask\" about that? What is the function that is being input into the Fourier neural operator layers? Why does the figure discuss loss functions? An \"architecture\" should be completely independent of any loss or training approach.\n\t\t\n1. Is it crucial that $ n_{train} $ is fixed to be the same number for all $ S $ datasets?\n\t\t\n1. Line 200: it is not clear yet why the hypernetwork map from parameters to coefficient needs to be unsupervised. Do you assume knowledge of the form of the PDE itself? I thought the physics was supposed to be \"hidden\"? It seems line 330 considers a fully supervised version.\n1. Lines 205--215 are confusing. What is $\\theta$? If it is the trainable network parameters, why do you mention that $\\theta$ is related to Eqns 3.5--3.6, which only involve the data and are independent of any choice of architecture? It is likely that this is a notation issue.\n1. What is the role of VAE here compared to an standard autoencoder or deterministic map? It is not clear why the probabilistic method is chosen over a deterministic one.\n1. Lines 243--245: what are the different tasks? these should be stated clearly in terms of the PDE model.\n1. Sec 4.1, I don't think NIO is applicable to this test problem because it only is defined for empirical-distribution-to-function inverse maps. What is the observed data for the inverse Holzapfel–Gasser–Ogden (HGO) model?\n\nTypos and notation:\n1. The authors severely shrink the space between paragraphs, sections, and figure captions to make the page limit. The result is not visually appealing.\n1. line 146-147: should the noise depend on $ x\\in\\Omega $? E.g., $ \\zeta=\\zeta(x) $ here and elsewhere in the paper. Lines 184 is not well defined because adding scalar Gaussian noise to function values $u(x)$ does not even define a function $ u $. Maybe they mean to define a spatially correlated Gaussian noise process instead?\n1. Eqn 3.7 $\\mathcal{G}$ should depend on $ \\eta $ (or $ z $)?\n1. Does $ E $ mean expectation? This is never defined. $\\mathbb{E}$ is much more common.\n1. Line 206-207: What is $\\mathcal{D}$, is this $\\mathcal{D}_{tr}$ or a different dataset? Also, the authors should say that the max is taken over $\\theta$ instead of being ambiguous.", "limitations": "", "rating": 2, "confidence": 2, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "The paper is original. The literature review is good. I like the main idea of using hyper-networks to solve forward and inverse problems in an end-to-end fashion. Such methods have potential to significantly advance the field of SciML if scalable enough. The theory part seems adequate and the assumptions and their limitations are discussed well. While not mathematically sophisticated, the results seem interesting and important for the problem considered in the paper. \nThe paper considers several numerical experiments and benchmarks against many other related methods. The numerical comparisons seem well done.", "weaknesses": "This paper makes a scientific machine learning (SciML) contribution. As such, it should be accessible to SciML readership. Unfortunately, I do not think this is the case. There is a lot of language and conventions that are unfamiliar to SciML researchers. It seems the paper is mainly written for a core CS/Ml audience. Overall, this paper was very hard to read. I kept going back and re-reading paragraphs. There is a lot of text, but there are often only vague claims of addressing physical interpretability. The paper would benefit from more precise writing and less filler text in Secs 1-3.\nThe quality is low at times. Notation is not defined before being used, causing much confusion. For example, all of a sudden $\\theta^\\eta_P$ is used in line 221, where the meaning of P is not discussed.\n\nSome claims seem to good to be true. For example, lines 168--169 says \"... we propose a novel neural architecture to alleviate the\ncurse of ill-posedness without requiring prior knowledge.\" This seems impossible because the ill-posedness is inherent to the inverse map being learned and cannot be removed by an architecture alone, unless the architecture incorporates prior knowledge (AKA, regularization). While the VAE does regularize the training, it is not clear if it regularizes the ill-posedness inherent in the data and the physical problem.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VIb0w4Lyye", "reviewer": "Reviewer_L3DK", "summary": "This paper proposes DisentangO, a novel variational hyper-neural operator architecture designed to learn disentangled, interpretable representations of physical factors from parametric PDE systems. The core idea is to perform disentanglement not on the observational data (e.g., the solution field $u(x)$), but on the parameters of the neural operator itself. Specifically, it uses a hierarchical variational autoencoder (HVAE) to encode and decode the task-specific lifting-layer parameters ($\\theta_P^\\eta$) of a meta-learned Implicit Fourier Neural Operator (IFNO). The authors claim this framework can simultaneously solve the forward problem (physics prediction) and the inverse problem (physics discovery). They provide theoretical analysis to guarantee component-wise identifiability of the latent factors and demonstrate results across supervised, semi-supervised, and unsupervised learning tasks.", "questions": "Can the authors justify the omission of SOTA neural operators (FNO, FUSE, etc.) as baselines in Experiments 2 and 3? Furthermore, could they provide results for the more direct baseline: a MetaNO for prediction combined with a VAE trained on the data ($u,f$) for discovery?\n\nThe results (e.g., 2.67% error for MetaNO vs. 5.28% for DisentangO in Table 3) show a significant drop in predictive accuracy. Can the authors comment on this trade-off? Is this an inherent cost of the method, and in what practical applications would this be an acceptable exchange?\n\nTable 4 shows a dramatic performance drop when using UFNO, which contradicts the \"model-agnostic\" claim. Can the authors clarify this? Does the method's success (both for prediction and discovery) depend heavily on the specific structure of the IFNO lifting layer?\n\nWhat is the primary motivation for assuming the lifting-layer parameters $\\theta_P^\\eta$ are a better representation for disentanglement than the physical data fields ($u, f$) themselves, especially given that using $\\theta_P^\\eta$ seems to damage the model's forward predictive capabilities?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper's primary strength is its originality. It shifts the focus of disentanglement from the data space ($u, f$) to the model's parameter space ($\\theta_P^\\eta$). This is a creative approach to embedding an inverse solver inside the forward solver's parameterization, using the MetaNO lifting layer as the information bottleneck.\n\nThe inclusion of a theoretical analysis on identifiability (Section 3.2, Theorems 1 and 2) is a major strength. It provides a formal basis for the claim that the latent factors $z$ can, in principle, recover the true generative factors. This is a level of rigor often missing in empirical machine learning papers. The empirical validation of the assumptions (Appendix C.4) further strengthens this.\n\nThe paper's \"physics discovery\" claims are strongly supported by its qualitative visualizations. The latent traversals in Figure 5 (unsupervised) and Figure 6 (semi-supervised) are clear and compelling. They show that the learned latent dimensions correspond directly to meaningful and distinct physical variations (e.g., fiber orientation, digit morphology), providing strong intuitive evidence that the disentanglement was successful.", "weaknesses": "The paper's performance claims are unsubstantiated due to improper baselines in the semi-supervised and unsupervised experiments (Exp 2 & 3). The models are benchmarked against standard VAEs that are not designed for the task and fail (e.g., 61.10% error). To be taken seriously as a neural operator, the method must be benchmarked against other SOTA NOs (like FNO, FUSE, etc.), not just its own backbone.\n\nThe paper's core hypothesis is that disentangling from parameters is the right approach. It never validates this against the simpler, more obvious alternative: (1) training a MetaNO for prediction and (2) training a VAE on the data ($u, f$) for interpretation. Without this comparison, there is no evidence that the proposed complex HVAE architecture is justified.\n\nThe method pays a high price for interpretability, with the forward prediction error nearly doubling in the unsupervised case. This severe trade-off is never discussed. It is unclear for what practical scientific or engineering application this level of accuracy degradation would be acceptable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FQcTRiY7Oc", "reviewer": "Reviewer_QjzN", "summary": "Researchers propose DisentangO, a novel neural operator architecture that extracts interpretable, disentangled physical parameters from a black-box model, bridging the gap between predictive accuracy and physical understanding in PDE solutions. While neural operators are powerful for solving PDEs, their black-box nature limits physical insight. To address this, the authors introduce DisentangO, a hyper-neural operator designed to solve the inverse problem of identifying the underlying physical parameters. It combines a multi-task neural operator with a variational autoencoder to disentangle latent physical factors from the model's own parameters. This approach not only improves interpretability but also enhances generalization, as demonstrated in supervised, semi-supervised, and unsupervised learning scenarios.", "questions": "1. The selected baselines are VAE or MetaNO, but there are many baselines are not compared, e.g., classical methods (not ML-based) for solving inverse problem, \"adaptive operator learning for infinite-dimensional Bayesian inverse problems\" [Gao, et al, 2024], \"Deciphering and integrating invariants for neural operator learning with various physical mechanisms\"[Zhang, et al, 2024], etc. \n2. Questions related to Figure 2, the derived loss function and the experiment: \n-What is the first term in the loss function in page 1? Is it \"f(x)\" in Figure 1? It seems that f(x) is the input of the encoder, but it doesn't appear in the Figure 1.\n-Which term in the loss function in page 7 related to the true value of b^{\\eta}(x) in the supervised learning?\n-If b^{\\eta}(x) is given, is it also the input of MetaNO?\n- Could you explain what is the physical meaning of b^{\\eta}(x) and \\theta in its related PDE in the experiment in supervised learning?\n3. There is format issue in page 2. \n4. In the key contributions, you mention robust generalization. What does \"robust\" refer to?\n5. The title can be improved. The key application seems to be the inverse problems in PDE.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "It combines the predictive accuracy and physical interpretability in a unified ML-based modelling framework, improving the automation level and usage of data in this area.", "weaknesses": "This is a straightforward application of VAE and Neural Operators for solving inverse problems. The presentation can be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8JbWn5lrj9", "reviewer": "Reviewer_zLDe", "summary": "This paper proposes DisentangO, a neural operator designed to disentangle physically meaningful latent factors within PDE-governed systems. The model aims to unify forward and inverse PDE learning, enhance interpretability, and achieve latent identifiability under varying supervision levels. It introduces a variational hypernetwork formulation where the latent variables encode distinct physical mechanisms, supported by theoretical identifiability proofs. Experiments across three settings (supervised, semi-supervised, and unsupervised) demonstrate that DisentangO can recover structured latent spaces and achieve competitive prediction performance on forward and inverse problems.", "questions": "The paper claims that DisentangO enables physically interpretable latent variables. Could the authors elaborate more concretely on how each latent dimension corresponds to specific physical quantities or mechanisms, and whether these mappings can be verified quantitatively?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper explores building interpretable neural operators capable of discovering physically meaningful latent factors from data. This is an important and promising research area. The experimental design is well-structured, progressing from fully supervised to semi- and unsupervised scenarios, effectively illustrating the model’s generality and stability. Notably, the unsupervised material learning results are engaging: the latent traversal visualizations show that each latent dimension corresponds to interpretable material properties. The paper provides a conceptually coherent attempt to connect neural operator modeling with disentangled representation learning.", "weaknesses": "(1) While the experimental structure is good, the experiment setups remain relatively low-dimensional and simplified. The generalizability of DisentangO to complex systems remains uncertain.\n\n(2) The paper provides only limited quantitative analysis. The evaluation mainly relies on qualitative visual evidence, with few metrics or diagnostic studies to support the claimed physical disentanglement. It would strengthen the work if the paper could include more quantitative or property-based analyses to better assess the model’s physical interpretability and internal behavior.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)