Uncertainty-Informed Meta Pseudo Labeling for Surrogate Modeling with Limited Labeled Data
Authors: Xingyu Ren
Pengwei Liu
Pengkai Wang
Guanyu Chen
Qinxin Wu
Dong Ni
Venue: NeurIPS 2025 poster ()
Keywords: Surrogate modeling
Semi-supervised regression
Meta pseudo label
Primary area: general_machine_learning
TL;DR: We propose a model-agnostic UMPL framework that leverages teacher-student uncertainty to refine pseudo labels and improve generalization with unlabeled data.

Abstract: Deep neural networks, particularly neural operators, provide an efficient alternative to costly simulations in surrogate modeling. However, their performance is often constrained by the need for large-scale labeled datasets, which are costly and challenging to acquire in many scientific domains.
Semi-supervised learning reduces label reliance by leveraging unlabeled data yet remains vulnerable to noisy pseudo-labels that mislead training and undermine robustness.
To address these challenges, we propose a novel framework, Uncertainty-Informed Meta Pseudo Labeling (UMPL).
The core mechenism is to refine pseudo-label quality through uncertainty-informed feedback signals. Specifically, the teacher model generates pseudo labels via epistemic uncertainty, while the student model learns from these labels and provides feedback based on aleatoric uncertainty.
This interplay forms a meta-learning loop where enhanced generalization and improved pseudo-label quality reinforce each other, enabling the student model to achieve more stable uncertainty estimation and leading to more robust training.
Notably, This framework is model-agnostic and can be seamlessly integrated into various neural architectures, facilitating effective exploitation of unlabeled data to enhance generalization in distribution shifts and out-of-distribution scenarios.
Extensive evaluations of four models across seven tasks covering steady state and transient prediction problems demonstrate that UMPL consistently outperforms the best existing semi-supervised regression methods. When using only 10% of the fully supervised training data, UMPL achieves a 14.18% improvement, highlighting its strong effectiveness under limited supervision. Our codes are available at https://github.com/small-dumpling/UMPL.

Uncertainty-Informed Meta Pseudo Labeling for Surrogate Modeling with Limited Labeled Data
Xingyu Ren Pengwei Liu
Pengkai Wang
Guanyu Chen'
Qinxin Wu!
Dong Ni
'Zeejiang University, Hangzhou, China
(12332063, 22up2,22032026, guanyu, 22360247, dni) dni)@zju edu. cn edu.
Abstract
Deep neural networks, particularly neural operators, provide an efficient alternative to costly simulations in surrogate modeling. However, their performance is often constrained by the need for large-scale labeled datasets, which are costly and challenging to acquire in many scientific domains. Semi-supervised learning reduces label reliance by leveraging unlabeled data yet remains vulnerable noisy pseudo-labels that mislead training and undermine robustness. To address these challenges, we propose novel framework, Uncertainty Informed lnnferainty-Ifformeedd Meta Pseudo Labeling (UMPL... The core mechenism is to refine pseudo-label quality through uncertainty-infol infoorme feedback signals. Specifically, the teacher model generates pseudo labels via epistemic uncertainty, while the student model learns from these labels and provides feedback based on alcatoric uncertainty. This interplay forms a meta-learning loop where enhanced generalization and improved pseudo-label quality reinforce each other, enabling the student model to achieve stable uncertainty estimation and leading more robust training. Notably, This framework is aggossic and can be seamlessly integrated into various neural architectures, facilitating effective exploitation of unlabeled data to enhance generalization in distribution shifts and out-of-distributte distriibution scenarios. Extensive evaluations of four models across seven tasks covering steady state and transient prediction problems demonstrate that UMPL consistently outperforms the best existing semi-supervised regression methods. When using only 10% of the fully supervised training data, UMPL achieves a 14.18% improvement, highlighting its strong effectiveness under limited supervision. Our codes available https://gtthub..om// http://wwww.u.oooooommauuaaunnnn --dumlinngUUMPP
Introduction
High-fidelity simulations are indispensable tools scientific computing and engincering, enabling precise modeling of complex physical systems governed by partial differential equations (PDEs). They serve as foundational methodologies across diverse domains, including aerodynamics, chemical processing, and energy systems [64] 566 59 62 39]. However, the underlying systems grow increasingly nonlinear and coupled, the computational demands of high-fidelity simulations grow prohibitively high, limiting their feasibility for real-time tiletimme deployment and large-scale iterative work- flows [5] B2]. Recent advances in deep learning. particularly neural operator- based approaches, offer promising alternative by introducing scalable, data-driven surrogate models capable of capturing complex spatiotemporal dynamics and learning solution mappings between function spaces (2 3).
Nevertheless, the effectiveness of data-dríven approaches is still heavily dependent on the quality and availability of training data. cases where data are limited, these models tend to overfit
These authors contributed equally.
Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

StopTiUpdale Model
Heteroscedastoe Regression
Unlabeted
Flow Data Flow
Labuled
Gradient Flow
Student Model
Physics Space
Decedlr
Comnepaninng
Soltion
Labeld Beaetett)
Erocader
Blocka
Deceder
Prysial
paurccers
Gworesy
9
Liteseed
Dalo
Teacher Model
Uniorta
Decxder
Uncatarga
Clsurved
0
Blocke
Ouoritess
Memory Pe.1..2-13
Preduced Sate 34
Tencher Model
Processor Blocks
2
Updoted Sturent Model 98.9
rescback gralk (25 6:ff
Step2:Updatt Teacher Model with Feedbockk Loss
1-M1L190 Bq)
Figure 1: Overview of the UMPL pipeline. UMPL leverages both labeled and unlabeled data to enhance generalization under distribution shifts. The teacher generates pseudo labels via epistemic uncertainty, while the student learns from predicted states and their corresponding aleatoric uncer- tainties using heteroscedastic regression loss. The teacher is iteratively refined with feedback from the student's 's performance on labeled data, forming a closed-loop training process that progressively improves pseudo-label quality. Both the teacher and student can be versatile forecasting models.
and generalize poorly, restricting their use in situations where obtaining accurate simulations is expensive or unfeasible [33. To mitigate these challenges, semi-supervised learning (SSL) has emerged as promising solution by leveraging structural similarities between limited labeled data and abundant unlabeled data. Common SSL strategies often include pscudo-labeling [31 19] 67].
self-training [69 8 668, and contrastive learning [ 4). While SSL has shown promise in computer
vision, its application to complex surrogate modeling remains underexplored [52]. The presence of
strict physical constraints in scientific systems makes them particularly sensitive to the noise and
inconsistencies introduced by conventional SSL methods [57]. underscoring the urgent need for
physically consistent semi-supervised frameworks.
In this paper, we propose Uncertainty-informel Meta Pseudo Labeling (UMPL), novel semi- supervised learning framework explicitly designed for physical systems surrogate modeling. UMPL extends the meta pseudo-labeling paradigm by incorporating teacher-student architecture informed by uncertainty, addressing key challenges such as limited labeled data and distributional shifts. The teacher model generates both predictions and corresponding epistemic uncertainty construct incertainty-aaar pseudo labels. The student model is trained on these labels while estimating their heteroscedastic aleatoric inherent uncertainty. Throughout the training, the pseudo-label quality is assessed against a small labeled set, providing feedback that enables the teacher to refine its outputs iteratively. As a result, the student model can produce accurate predictions along with the relative uncertainty estimates. Extensive experiments across diverse physical-systen datasets and forecasting architectures demonstrate that UMPL consistently outperforms existing semi-supervised learning baselines. An overview of the UMPL. framework is presented in Figure T In summary, the contribution of our paper can be summarized as follows:
Problem Formulation. To the best of our knowledge, this is the first attempt to integrate pscudo-labeling strategies with the modeling of high dimensional PDE systems on non- uniform meshes, aiming address challenges for out-of-distribution generalization and distribution shifts arising from limited labeled data.
for Robust Generalization. We design a teacher-student framework that assigns distinct uncertainty modeling strategies to each role, inspired by their functional differences and underlying physical data characteristics. The teacher models epistemic uncertainty to gauge confidence under distribution shifts, while the student models heteroscedastic aleatoric uncertainty to capture pseudo-label noise. This design improves generalization and uncertainty estimation on unseen data. Furthermore, the framework flexible, allowing interchangeable predictive models and uncertainty estimators.
Role-Aware

---

Uncertainty- Informmed Feedback. Building upon the dual unceerrncetai modeling, we intro- duce an uncertainty-mfoomene feedback mechanism: the student provides aleatoric uncer- tainty signal to guide the teacher in refining uncertain regions. This targeted feedback significantly improves the quality of pseudo labels across iterations, especially in high- uncertainty or underrepresented regimes.
Empirical Validation. Extensive experiments conducted under various settings demonstrate the proposed UMPL framework's 's superior performance and robustness compared to existing semi-supervised learning methods.
2
Problem Setup
Consider the problem of learning solution operators for PDEs defined on a spatial domain Ω2 The objective is lear mapping A S, where A and S denote the input and solution function spaces over Ω. Each input x = (g, a, s) consists of geometry E TR TRNxCy XC2 and associated physical quantities R x.. and physical parameters observed on g: the corresponding solution is E I xVC, where N is the number of discretized spatial points. We approximate G using a neural network Go. which comprises three components:
(1) a state encoder & that maps input features to latent embeddings E IR N xdx, (2) a latent evolution module P that transforms to predicted latent states 2, and (3) decoder D that maps
the output solution ÿ. For transient systems, the prediction proceeds autoregressively over the time steps. Our goal is to closely approximate the underlying mapping with the parameterized neural network. For training and evaluation, the supervision and evaluation loss is defined using the relative L2 error:
y ÿ 12
Li 0)
y E122
where y and ÿ denote the prediction and target data, respectively. c is a small constant introduced to ensure numerical stability. In practice, we consider the training data consists of small labeled set D, = (#f 3f) 1 and a larger unlabeled set D,, = {xt, with n,, > n4. This setup reflects real-world constraints in scientific computing, where labeled data obtained from simulations is expensive, while unlabeled inputs are more readily available. The learning objective is to approximate the underlying solution operator G by utilizing labeled and unlabeled data.
3 Methodology
3.1 Asymmetric Uncertainty Estimation in Teacher and Student Models
To capture different sources of uncertainty, our framework adopts a dual-model design: the teacher estimates epistemic uncertainty via Monte Carlo (MC) Dropout, while the student models aleatoric uncertainty through direct variance prediction. While MC Dropout and ensemble methods approx- imate Bayesian inference, they incur high computational costs [66 53 66 2]. In contrast, direct prediction methods are efficient but fail to capture model uncertainty, often yielding overconfident predictions on -ut-o--distribution data 45 To balance these trade-offs, we apply MC Dropout in the teacher to quantify epistemic uncertainty, particularly relevant in simulation- based datasets with reliable labels but sparse input coverage. At the same time, the student learns from the teacher's s pseudo labels and estimates aleatoric uncertainty to refine its predictions. This mutual guidance improves pseudo-label quality and promotes more robust learning.
EMC Dropout for Teacher Model. To further improve efficiency, we introduce Exponential Moving Average-enhanced MC Dropout (EMC Dropout), which incorporates exponential moving averages (EMA) into the estimation process. Inspired by Sequential MC Dropout [7]. EMC Dropout incrementally updates the predictive mean and variance using a temporal sliding window, thereby preserving the Bayesian properties of dropout while avoiding repeated stochastic forward passes.
During the training progress, we set ür as the mean value of prediction at time and variance o?. At the initial time (t = 0), we perform a complete MC estimation using M stochastic passes to obtain the initial predictive mean jáo and variance of. When get the prediction ÿt at time we update the mean value fu+t and variance oit
j = aatt + (1 cc)
(2)
62 = aeli - Pc+1) ) + (1 -
3

---

where E (0, serves as gating factor that controls the balance between the influence of the most recent observation and the accumulated historical estimates. higher value of a places more emphasis on recent data, while lower value favors historical stability.
From a Bayesian perspective, dtopoou-based methods estimate the posterior predictive distribution by injecting stochasticity into the network's 's parameters, typically requiring multiple forward passes per input. EMC Dropout incorporates the EMA mechanism to reduce this computational overhead and track the predictive mean and variance over time. Rather than repeatedly sampling the stochastic model output, EMC Dropout incrementally updates running estimates during training. This strategy preserves the epistemic uncertainty captured by dropout while offering more efficient approximation of Bayesian marginalization. Further analytical details are provided Appendix [B.]
Theorem 3.1. Asymptotic consistency of EMC Dropout. Let (x(r)) denote the prediction of a neu- ral network with dropout applied at time step I, and fut. of be the predictive mean and variance maintained over time with the gating factor a. Suppose that ft(x) ~ p(ylæ, (r, wt). where Wy ~ q(w) represents a stochastic sample from the dropout-induced approximate posterior. Then, under station- arity and ergodicity assumptions on the sequence {f(())) the EMC estimates satisfy:
ju g (2; (x; w)), of
+ V ((w)) [x) as t + Do.
(3)
Assume and a are the expectation estimation and variance of MC Dropout, we have:
lip E - -1)
a Bgtw)) [[f((;; w) - --)] Var ((f((r;w)) 11)
t-0c
2
(4)
lim B (o? - - s
2 2 a
The proof of Theorem 3.1 is provided in Appendix B.2 It establishes that, under sufficiently mixing sampling process, the EMA estimates of the predictive mean and variance converge almost surely to the Monte Carlo estimates of the Bayesian predictive distribution induced by dropout.
Model-Based Uncertainty Estimation for Student Model. To account for the noise and uncertainty inherent in pseudo labels provided by the teacher model, we train a student model to predict both the target value and an input-dependent variance. Inspired by uncertainty quantification techniques in PDEs forward problems (63, we introduce additional uncertainty decoder to estimate this variance. Given an input ær the model uses an auxiliary decoder D. to produce the corresponding predictive uncertainty ē E IR XC, from the latent state ă. Then, we optimize the student model using L2 relative error based heteroscedastic regression loss based on the pseudo labels:
N,
11: 11 e|12
Dil12
+ log o? ).).
Cu(y, [0, ō])
(5)
N,
==1
where ý denotes the prediction of the target label y. The loss function down-weights errors for samples with higher predicted aleatoric uncertainty, reflecting their lower label reliability (25). Its formal derivation is based on maximum likelihood estimation under the assumption of heteroscedastic observation errors (24), enabling more effective student optimization and providing informative feedback for the teacher.
3.2 Eeteraaint--nnfommen Feedback for Teacher Model Update
We adopt incertainty-informe strategy that replaces standard meta-gradient computation to enable efficient teacher updates without incurring the high cost of per-sample gradient tracing. While the standard MPL (47) method relies on categorical pseudo labels and is limited to classification. it does not generalize well to regression tasks with continuous outputs. Alternatives like Noisy TS [57] use noisy teacher predictions and scalar feedback but require manually tuned noise and apply uniform feedback across the input space, leading to spatial inconsistency and limited adaptability. In contrast, our method introduces structured, point- wise feedback informed by model uncertainty. enabling coherent and efficient teacher updates in high-dimensional regression settings. Details of the feedback loss during training are illustrated in Figure [ The feedback loss produced by our method on unlabeled data more closely approximates the actual loss, indicating more reliable guidance than Noisy TS [57] which is a current meta pseudo-labeling approach designed improve model performance in ODE-based dynamical system modeling.

---

Ground Truth
Noisy TS
Noisy TS
UMPL
UMPL
Kendall-r
Foodback Loss
Actual Erorr
Feedback Loss
Actual Error
Correlation
Figure Figure Comparison of feedback loss and true pseudo-label error during training for
UMPL and Noisy TS [57] using AeroGTO (38) as the base forecaster. The rightmost subplot illustrates the K Kendall-T correlation between feedback loss and true pseudo-label error across 100 unlabeled cases, showing that UMPL's aleatoric uncertainty-guidee feedback more reliably reflects pseudo-label error.
eeverfainty-Informen Pseudo Label Generation. We design the teacher model to integrate predic- tive uncertainty into pseudo label generation, thereby enhancing the reliability and informativeness of the supervisory signals. For simplicity, the following definitions and derivations provided for single prediction step, without incorporating multi-step autoregression. Given the teacher's variance os at time t, get the pseudo labels:
V (0T) + E(g) oc.
(6)
where [(g) denotes random field obtained through Gaussian smoothing the geometry g. which encourages continuity in the generated pseudo labels. Specifically, the geometric mesh composed of points and edges, and the smoothing is performed based on the local connectivity defined by the mesh structure:
9: 9;112 (-1s: 212
|9:
E(gs) = sign
Σ jEZ((S)
exp
Ti
T3 N(0,1,
(7)
where ((i) is the geometric neighborhood of the point i, defined by edge connectivity for meshes or k-nearest neighbors for point clouds. 9i 9j are spatial coordinates, I controls the smoothing scale, and ry (0, 1) is sampled noise. The sign E {-1, +1} encodes smoothed perturbations with spatial coherence.
Derivation of the Feedback Gradient.
To optimize the teacher model based on the student's feedback, the objective is minimize the following loss:
min ((055((00))
where (P'()T) = argmin Lu (0r,000))
(8)
8s
where Li and L,, denote the losses labeled data and pseudo-labeled data, respectively. The argmin operation in the above equation poses challenge for gradient-basee optimization, as requires the parameter Os attain its optimal value before the subsequent step can proceed. This inherently breaks the end-to-end diffcrentiability of the training pipeline. To overcome this issue, we employ one-step approximation that allows for tractable and efficient optimization.
(L = 0g 7s
Co
(9)
Here, the optimization goal becomes:
min L,(0s nss V (0r, 0s)).
(10)
6T
In principle, if the gradient of the objective concerning 0r is computable, end-to-end optimization via gradient descent becomes feasible. Denoting Eq. 10 Lp. the gradient can be derived using the
5

---

REINFORCE rule 600 as follows:
dhr ② Li yl, S 2i Bo ((..(0)))
10s- - nis vos VusCuliu" ( 0s)]]
00T
80T
T
=1s- aCi S(x 0'ss)
log pl Pl
( OT)
(11)
00g
00s
00T
0y=0s/
=ns (Vog Cin,, S(mr; (s))
Voy " ((xx:0s)))
Vo, (rr
wh (LLL( Or).
where h (3n S(m;; 0s)) - ((m1;())))) denotes the change in student performance after training on pseudo label and NLL == - llggopylll plyla; 0) is a scaled negative log-likelihood (NLL) loss. If h 2 0, minimizing Ly increases the likelihood of similar pseudo labels in future updates. otherwise, it suppresses them. More details about the feedback loss can be found in Appendices
B.3 and B.4 Combined with the supervised loss L :- Ci (xx;; OT)), the teacher's update is performed as follows:
0- - '/
Vor (AT + Lr).
(12)
Theorem 3.2. Generalization error bound of the student.
Let F denote the hypothesis class of neural surrogate models with fixed architecture and bounded parameters. With the training set consisting of n; labeled and nu pseudo-labeled samples, for any 5 (0, 1), with probability at least 1 8, the following inequality holds for the student model S(z 0s) F where x', denotes an unseen input drawn from the underlying data distribution:
log(1/6) 2(m n.))
((a)))) (Vu, Strui 0s)) < Ceemi (S(0s)) + 2T. Rt Rmy (F) +
+ Ap epeendo>
(13)
where Luems denotes the empirical loss computed over both labeled and pseudo-labeled data. RRy+R. (F) is the empirical Rademacher complexity of the hypothesis class F; L represents the Lip- schitz constant of the loss function. and is universal constant. The coefficient Ap E [0, 1] controls the relative contribution of the pseudo-labeled samples. Epscudo EE [ (1r,s 0s)) denotes the expected error between the pseudo-labels and the student's 's predictions on unlabeled inputs. The proof can be seen in Appendix B.2
stel
This bound implies that the student model can achieve improved generalization performance when the reduction in model complexity exceeds the additional error introduced by the pseudo-labels.
3.3 Training Workflow of the UMPL Framework
We propose a unified training framework that optimizes teacher and student models through mutual feedback. To improve robustness against noisy pseudo-labels, we incorporate Progressive Uncertainty Filtering (PUF), which leverages EMC Dropout to estimate predictive uncertainty. Based on this, binary mask m f0, 13 constructed by selecting the top-k% most uncertain predictions. The corresponding masked regions (m, 1) are excluded from the student's 's loss computation and the feedback used to update the teacher. As training progresses and the reliability of pseudo labels improves, the masking ratio is gradually decreased, enabling the student to learn from broader range of samples and provide more informative feedback to the teacher.
After initialization, the training process follows three main steps: (1) Pseudo-label generation:
The teacher model computes the prediction and variance with Eq generating pseudo-labels
( E) (2) Student training on pseudo labels: Train the student with an unlabeled loss de- fined on pseudo labels with Eq and estimate the improvement h as the student feedback. (3) Teacher and student's 's update: The teacher is updated using both the supervised loss and the feed- back loss (Eq. [2], while the student updated exclusively on labeled data. This strategy, referred to as Supervised Anchoring (SA), ensures the student remains grounded in reliable supervision, prevent- ing error accumulation during self-training, especially under domain shifts where pseudo-labels may be unreliable. The design of the algorithm ensures continued exploration of unseen data space and mitigates the risk of model collapse. A detailed description of the algorithm presented in Algorithm I More details about training can be found in Appendix C

---

Algorithm UMPL Algorithm
Input:
Labeled data: Di = {( vi Kー unlabelled data: Da {at J= teacher and student neural operator: T(=07). S(;; 0g). epoch N Nepoch
Output: Trained student: (%))
for ł 0,
NNpochh 1 do
1. Sample batch of labeled data {(2 3/) and an equal-sized batch of unlabeled data at.
2. Generate pseudo rollouts from the teacher and record the old student's performance on the labeled data:
Th = 5.. (0r) + E(g) Öt:
5 Li(i (z) f's))
3. Update the student with pseudo loss and record the new student's performance on the same labeled data:
0,t1 flz 115 Vgg S
E. Cily) S(z) ())
4. Calculate the feedback loss:
CF = (L. - 1) -NLL 04)
5. Update the teacher with its labeled loss and the feedback loss and update the student with its labeled loss:
0 04 7/T
V 03
(CT LF),
0 00st1 ns L gt42
end for return S(: 5(:00%
Experiment
4.1 Experimental Settings
Benchmarks. We study datasets from three domains, as summarized below. Partial Differential Equations. We consider four PDE benchmarks: (i) Lid-driven cavity flow (16), where constant top boundary velocity induces circulation. We explore steady and transient settings, using Reynolds number(Re) and lid velocity (v) as orthogonal parameters for out-of-distribttion generalization and distribution shifts evaluation. (ii) Darcy flow [9]. a classical model describing fluid transport in porous media. (iii) NSM2D (37], the 2D Navier-Stokes equations augmented with magnetic field component. (iv) Plasma ICP, where low-resolution simulations with incomplete chemical reaction mechanisms are used for pretraining, followed by fine-tuning on limited high-resolution data to assess robustness under distribution shifts. Real-world Data. We employ the Black Sea dataset 10.. which contains real-world ocean temperature measurements over time, to evaluate temporal out-of-distribution generalization. Large-scale 3D Simulation. We use the Ahmed dataset (34), where the input is the 3D geometry of vehicle, and the output is its corresponding aerodynamic drag coefficient. This benchmark evaluates the model's ability to generalize aerodynamic predictions across diverse vehicle shapes.
Baselines. We evaluate our method on four representative surrogate modeling architectures, in- cluding two basnt-based models (DeepONet 41 and Transolver [6]] and two mesh-based models (Me)) [46] and AeroGTO
38 ). to assess its performance across different data representations. We compare UMPL against four baseline useudo-labeling strategies (more details see Appendix D): Supervision Only. The model is trained exclusively on labeled data. Pseudo Label
(PL) 31 The teacher remains fixed, and the student trained on both labeled data and pseudo labels generated by the teacher. Mean Teacher [54]. A widely used and stable approach in which the teacher is updated via EMA of the student's parameters. Noisy TS [57]. A meta pseudo-labeling method where the teacher receives feedback from the student and generates pseudo labels with added
7

---

Table 1: We compare the performance of our method against four baselines across four different models under a labeled data ratio of r = 10% for all tasks. Cyan Yellow and Green indicate the best, second-best and worst L2 loss values, respectively. \ indicates non-convergence.
BENCHMARKS
Model
Method
Dancy Flow
Ahmed
NSM2
ICP
Stationayy
Wi OOD
w/ 000
In/T
Out-I
Supervision Oaly
7.98e-2
3.07e-1
6.07e-2
1.69-1
1.4e-1
2.41e-1
Parado Label
8.376-2
2.82e-1
5.65e-2
1.73e-1
1.5ee1
2.35e-1
DeepONe 4
Meom Teacher
8.360-2
2.77c-1
5,576-2
1.77e-1
1.46e-
2.22c-1
Noiey TS 32
7.83e:2
2.57e:
5.38e-2
1.42e
1.24e
1.67e
UMPL_[our]
6.94e.2
2.23e
4.57e-2
1.08e-1
9.61e:2
1.4ke11
PROMOTION
11.36% %
13.22%
15.05%
23.94%
22.50%
14.37%
Supervision Only
8.06e-2
9.11e-2
5.62e-1
3.82e-2
1.02e-1
9.84e2
1.13e1
Pscudo Label 3
7.33e-2
8.76e-2
4,95e-1
2.22e-2
1.13e-1
9.97e-2
1.08e1
Moon Teacher X
7.33o-2
8.320-2
4.41e-1
1.97e-2
1.07e-1
9.83e-2
1.07e-1
MGN 46
Noay 18
6.49e.2
6.92e-2
3.85e 1
2. 2.20e
9.48e-2
8.51e-2
9.67e-2
UMPL[ours]
5.74e
6.34e 2
3.51e 1
1.64e 2
7.91e:2
8.19e.2
8.99e 2
PROMOTION
11.55 %
8.38%
8.83%
16.75%
16.56%
3.76%
7.03%
Supervision Only
6.14e-2
7.69c-2
2.71e-1
1.90-1
3.69c-2
9.55e-2
9.34e-2
1.12c-1
Pscudo Label 31
5.24e-2
7.30e-2
2.35e-1
1.85e-1
1.79e-2
1.08e-1
9.16e-2
9.79e-2
Transolverf6l
Mean Teucher
5.34o-2
7.21e-2
2.21e-1
1.72e-1
2.19e-2
1.31e-1 I
9.26e-2
9.68e-2
Notey TS Xt
4.66c-2
5.89c-2
1.79c-1
1.56c-1
1. 1.64e-2
9.53c-2
8.81e-2
9.17c-2
UMPL(eurs]
4.37e
5.17e
1.51e-1 1
1.41e-1
1.24e
7.32e
7.74e
8.53e
PROMOTION
6.22%
12.22%
15.64%
9.61 %
24.39%
23.18%
12.14%
6.97%
Supervision Only
3.83c-2
6.88e-2
1.72c-1
9.30c-2
3.51e-2
9.44e-2
8.95e-2
1,10-1
Pscudo 31
3.17e-2
5.85e-2
1,51e-1
7.20x-2
8.75e-3
1.17e-1
8.24e-2
1,03-1
AeroGTO X
Mran
3.03e-2
5,86e-2
1.45e-1
6.976-2
2.17e-2
1.24e-1
9.07e-2
1.02e-1
Nonay
2.856-2
4.71o-2
1.39c-1
6.246-2
1.09--2
7,98c-2
7.42o-2
9.38c-2
UMPL[eurS] 2.56e-2
3.63e-2
1.21e-1
5.25e
6.91e.3
6.33e-2
6.66e
8.12e-2
PROMOTION
22.92%
12.94%
20.67%
10.78%
13.43%
random noise. As data augmentation and noise injection may harm model accuracy not employ any augmentation techniques during training.
23
57
we do
Tasks. We evaluate UMPL under three regimes with a unified protocol. Limited-label setting. On 20%, Ahmed, Darcy Flow, and NSM2D, we vary the labeled ratio (10%, 20%, 30%]. For each stratified labeled subset fixed and the remaining training samples are treated as unlabeled. The test split is held fixed across r to ensure comparability. Out-of-distribution generalization. On the stationary Lid dataset, training and test sets are drawn from disjoint parameter regions to probe out-of- distribution parameter space. On the Black Sea dataset, models trained earlier, contiguous time windows and evaluated on non-overlapping future windows to probe out-of-ding off-istribbtion temporal space. Distribution shifts. We evaluate domain adaptation the time-dependent lid dataset via cross- Reynolds-
-yynolds-number number transfer ( RRe = 500 <> 5000) and on the Plasma ICP dataset by adapting from low-fidelity simulations high-fidelity measurements. Results averaged over five independent runs with different random seeds. Details regarding the datasets, models, and experimental settings can be found in Appendix E
Table 2: Evaluation on four models across datasets under label ratios r 10%, 20%, 30%.
Table 3: Ablation results of dif- ferent combinations of SA, PUF, and Feedback modules.
Black Sea 20%
dime-dependent Lid2d 20%
Model
10%
DeepONet DePPONatUMPL
1.93c-1
1.16-1
9.95c-2
3.34c-1
2.390-
2.16c-1
SA
PUF
Feedback
La Error
1.19e-1
8.33e
8.09-22
1.90e-1
1.59e-
1.38-1
1.06e-1
MGN
1.05e-1
9.28e
7.12e:2
3.00s
X
3.70e.
UMPL
8.996-2
8.05e.2
5.56e-2
2.65e.1 I
9.60e-2
4.65e-2
x
8.46e-2
V
6,72c-2
Transolver
1.02c-1
7.78c-2
6.28c-2
2.15c-1
9.53e-2
6.276-2
x
8.1oc-2
TransolverMPPL
8.9e-2
6.54e-2
5.06e-2
2.04e-1
7.85e-2
5.17e-2
5.93c-2
AeroGTO
1.00e-1
7.27e:2
5.98e
7.24e:2
3.54e
1.93e-2
AePGTO-MPI.
2.
5.88e
4.57e:2
4.63e.2
1.85e-2
1.06e-2

---

033
0.5
2.6
0.23
1.0
0.39
0.33
2.46
0.35
0.28
8:37
0.45
2.5
00 Commattion
Thee Thee-lpadder Add trilted MGN
Leuring
23%
51%
TS (6) Darcy Floe trained with DeyOöd
ES
Tin diet
4002 5500
10000
1e0
G7
UMBS Eive
Mllaatttttion livadiic Statenary 1.6
icrlieed Armiirr
Figure 3: (a) Correlation between predicted drag coefficient C2 and uncertainty from UMPL. and the fully supervised model, compared to ground truth the Ahmed test set (51 unseen geometries). (b) L2 loss over the whole parameter domain on the Stationary Lid dataset. +, *, and denote labeled, unlabeled, and test points, respectively. The red dashed boundary marks the in-distribution region. outside areas represent the citoof-distribution region. (c-e) Visualization results are presented across different datasets, with transient cases shown the final time step.
4.2 Main Results
Performance Comparison. Table presents the performance of UMPL across three representative tasks under limited-labed conditions. On stcady-state datasets (Darcy Flow, Ahmed Body) and the transient dataset (NSM2d), UMPL consistently outperforms all baselines, yielding an average improvement of 12. 13% over the second-best approach. In out-of-distribution scenarios (Stationary Lid, Black Sea). UMPL achieves the lowest Lg errors both within and beyond the training distribution, surpassing the second-best method by 20.19% and 11.37%, respectively. Likewise, distribution shift benchmarks (Time-dependent Lid2d and Plasma ICP) demonstrate superior performance. As shown in T consistetlyy improves accuracy across varying label ratios, with particularly notable gains under the most challenging label distribution setting. These results collectively highlight the method's 'ss strong generalization capability and data efficiency.
Temporal and Spatial Generalization. UMPL consistently outperforms all basclines across diverse tasks, demonstrating strong generalization capabilities in both temporal and parametric domains. To further illustrate this, we present detailed visualizations of the Stationary Lid dataset. As shown in Figure b), UMPL achieves lower L2 eтrоrs than the previous best- persooerforming method, Noisy TS, particularly in out-of-distribution regions. Moreover, UMPL maintains lower testing loss and better training stability compared to the PL method, which exhibits early-stage overfitting, The corresponding learning curves and detailed comparisons are provided in Appendix [F]
Visualization and Analysis. Figure 3(a) illustrates the performance gains of UMPL over the Supervised Only baseline the Ahmed Body dataset. Our method not only achieves more accurate predictions, but also produces better-callbrated uncertainty estimates. contrast, the Supervised Only model yields overconfident and homogeneous uncertainty, failing to reflect the true variability and uncertainty in the data. Figure bbb) presents the error distribution on the test set, showing that UMPL produces tighter and more favorable error profiles compared to existing methods. Figure c--) c-e) further compares UMPL with baselines across additional datasets, consistently demonstrating its advantage in both prediction accuracy and uncertainty calibration. Figure 4 () shows performance on the Black Sea dataset, where UMPL achieves lower prediction error and provides well-structured uncertainty estimates. Figure bb) presents a temporal rollout on the NSM2d dataset, highlighting UMPL's ability to capture complex system dynamics with lower error and more robust uncertainty estimation compared baseline approaches. Together, these results demonstrate that UMPL offers stronger generalization under both distribution shifts and out-of-distribution scenarios. Moreover, the

---

Diaribution
Out
Dutribation
CT
Error
(a) Black Seo trained with Transolver
Natey Erer
Prediction
Timestrp
GT
Teaher Erree
Nots
UMPL
UMPL
Prediction
(b) NSM22 trained with PhysGTO
Figure 4: Visualization results across multiple methods and models are presented, where all temporal predictions visualized the final time step. The results show that the UMPL framework significantly reduces prediction error and provides more reliable uncertainty estimates.
student model trained under our uncertainty informmed meta pseudo-labeling framework consistently provides reliable and interpretable uncertainty estimates across diverse tasks.
4.3 Abation Studies
We perform ablation studies on the Stationary Lid dataset to investigate the impact of each component within the UMPL framework. The experiments reveal that removing either the uncertainty-biat based pseudo-label filtering (PUF) the supervised alignment (SA) module leads to clear degradation in model performance, while eliminating both results in the most significant decline. Notably, using only the supervised component reduces the method to conventional pseudo-labeling and fails to match the performance of the full framework. These observations underscore the complementary roles PUF and SA, demonstrating that both are essential for achieving robust and accurate predictions. Further analysis on hyperparameter sensitivity included in Appendix F.5
Discussion and limitations
Limitations. While the proposed method consistently improves performance, still exhibits several limitations. It assumes that training and testing data are identically distributed (i.id.), causing performance degrade under significant distribution shifts, and struggles adapt highly dynamic and complex environments. Furthermore, the approach introduces computational and storage overhead due to the required EMC Dropout in the teacher model. Its loss function also necessitates aligned output shapes, which restricts its applicability to scenarios involving non-uniform or adaptive meshes. Future work should focus on improving computational efficiency and better leveraging meta pseudo-labels for more complex applications, like adaptive 3D flow field modeling.
Conclusion. This paper tackles the challenge of the surrogate model under out-o--distribution generation and distribution shift scenarios with limited labeled data. We propose UMPL, a semi- supervised framework that leverages uncertainty refine pseudo-labeling: the teacher estimates epistemic uncertainty while the student captures aleatoric uncertainty. These signals guide learning from confident pseudo labels and enable feedback via labeled data, allowing the teacher to reinforce reliable predictions and suppress errors. Experiments show that UMPL reduces error in uncertain regions and enhances generalization. Future work includes extending UMPL to tasks like 3D flow prediction.
10

---

References
[1] Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U Rajendra Acharya, et al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information fusion, 76:243-297, 2021.
[2] Soyed Tuhin Ahmed, Michael Hefenbrock, and Mehdi B Tahoori. Tiny deep ensemble: Un- certainty estimation in edge ai accelerators via ensembling normalization layers with shared weights. In Proceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design, pages 1-9, 2024.
[3] Kamyar Azizzadenesheli, Nikola Kovachki, Zongyi Li, Miguel Liu-Schiaffini, Jean Kossaifi, and Anima Anandkumar. Neural operators for accelerating scientific simulations and design. Nature Reviews Physics, 6(5):320-33) 328, 2024.
[4] Hritam Basak and Zhaozheng Yin. Pseudo-label guided contrastive learning for semi semi-supervised medical image segmentation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19786-1977 19797, 2023.
[5] Ramandeep Behl, Sonia Bhalla, Ángel Alberto Magreñán, and Sanjeev Kumar. An efficient high order iterative scheme for large nonlinear systems with dynamics. Journal of Computational and Applied Mathematics, 404:1 13249, 2022.
[6] Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty neural network. In International conference on machine learning, pages 1613-1622, PMLR, 2015.
[7] Pamela Carreno, Dana Kulic, and Michael Burke. Adapting neural models with sequential monte carlo dropout. In Conference on Robot Learning, pages
PMLR, 2023.
[8] Baixu Chen, Junguang Jiang, Ximei Wang, Pengfei Wan, Jianmin Wang, and Mingsheng Long. Debiased self- Soff-training for semi-supervised learning, Advances Neural Information Processing Systems, 35:32424-3244
2022.
[9] Gengxiang Chen, Xu Liu, Qinglu Meng, Lu Chen, Changqing Liu, and Yingguang Li. Learning neural operators riemannian manifolds. National Science Open, 36)):2024001,, 2024.
[10] Stefania A Ciliberti, Marilaure Grégoire, Joanna Staneva, Atanas Palazov, Giovanni Coppini, Rita Lecei, Elisaveta Peneva, Marius Matreata, Veselka Marinova, Simona Masina, et al. Moni- toring and forecasting the ocean state and biogeochemical processes in the black sea: Recent developments the copernicus marine service. Journal of Marine Science and Engineering, 9(10): 146, 2021.
[11] Jiequan Cui, Zhisheng Zhong, Shu Liu, Bei Yu, and Jiaya Jia. Parametric contrastive learning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 715-724, 2021.
[12] Weihang Dai, Xiaomeng Li, and Kwang-Ting Cheng. Semi-supervised deep regression with uncertainty consistency and variational model ensembling via bayesian neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7304-7313, 2023.
[13] Subhayan De. Matthew Reynolds, Malik Hassanaly, Ryan N King, and Alireza Doostan. Bi- fidelity modeling of uncertain and partially unknown systems using deeponets. Computational Mechanics, 71 ((6)::251-126)) 2023.
[14] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference machine learning, pages 1050-1059. PMLR, 2016.
[15] Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, et al. survey of uncertainty deep neural networks. Artificial Intelligence Review, 56(Suppl 1D:1513-155 2023.
11

---

[16] UKNG Ghia, Kirti N Ghia, and CT Shin. High-re solutions for incompressible flow using the navier-stokes equations and multigrid method. Journal of computational physics, 48(3):387- 411, 1982.
[17] Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, pages 12556-12569. PMLR, 2023.
[18) Yao Hu, Hing-Ho Tsang, Nelson Lam, and Elisa Lumantarna. Physics-informed neural networks for enhancing structural seismic response prediction with pseudo-labelling. Archives of Civil and Mechanical Engineering, 24(1):7,002
[19] Zijian Hu, Zhengyu Yang, Xuefeng Hu, and Ram Nevatia. Simple: Similar pscudo label exploitation for semi-supervised classification. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 15099-15108, 2021.
[20] Chuanbo Hua, Federico Berto, Michael Poli, Stefano Massaroli, and Jinkyoo Park. Learning efficient surrogate dynamic models with graph spline networks. Advances in Neural Information Processing Systems, 36, 2024.
[21] Eyke Hüllermeier and Willem Waegeman. Alcatoric and epistemic uncertainty in machine learning: An introduction concepts and methods. Machine learning, 110(3):457-06, 2021.
[22] Neal Jean, Sang Michael Xie, and Stefano Ermon. Semi-supervised deep kemne learning: Re- gression with unlabeled data by minimizing predictive variance. Advances in Neural Information Processing Systems, 31, 2018.
[23] Jindou Jia, Zihan Yang, Meng Wang. Kexin Guo, Jianfei Yang, Xiang Yu, and Lei Guo. Feedback favors the generalization of neural ODEs. In The Thirteenth International Conference on Learning Representations, 2025.
[24] Robert L Kaufman. Heteroskedasticity regression: Detection and correction. Sage Publica- tions, 2013.
[25] Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017.
[26] Sungjun Kim and Jungho Kim. Meta pseudo label tabular-related regression model for surrogate modeling. Expert Systems with Applications, 261:125520, 2025.
[27] Kazuma Kobayashi, James Daniell, and Syed Bahauddin Alam. Improved generalization with deep neural operators for engineering systems: Path towards digital twin. Engineering Applications of Artificial Intelligence, 131:107844, 2024.
[28] Georgios Kostopoulos, Stamatis Karlos, Sotiris Kotsiantis, and Omiros Ragos. Semi-supervised regression: A recent review. Journal of Intelligent & Fuzzy Systems, 15(3):1483-1500 2018.
[29] Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1 -97, 2023.
[30] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
[31] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient sumi-supe learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3. page 896. Atlanta, 2013.
[32] Xiaojie Li, Mingxue Yan, Xuewen Zhang, Minghao Han, Adrian Wing-Keung Law, and Xunyuan Yin. Efficient data-driven predictive control of nonlinear systems: review and perspectives. Digital Chemical Engineering, page 100219, 2025.
12

---

[33] Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. Journal of Machine Learning Research, 24(388):1-26 2023.
[34] Zongyi Li, Nikola Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Otta, Moham- mad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadenesheli, et al. Geometry-informed neural operator for large-scale 3d pdes. Advances in Neural Information Processing Systems, 36, 2024.
[35] L. Lima, A. Aydogdu, R. Escudier, S. Masina, S. A. Ciliberti, D. Azevedo, E. L. Peneva, S. Causio, A. Cipollone, E. Clementi, S. Cretí, L. Stefanizzi, R. Lecci, F. Palermo, G. Coppini, N. Pinardi, and A. Palazov. Black sea physical reanalysis (emems bs-currents) (version 1) [data set]. 2020.
[36] Jeremiah Liu, John Paisley, Marianthi-Amin Kioumourtzoglou, and Brent Coull. Accurate uncertainty estimation and decomposition in ensemble learning. Advances in neural information processing systems, 32, 2019.
[37] Pengwei Liu, Zhongkai Hao, Xingyu Ren, Hangjie Yuan, Jiayang Ren, and Dong Ni. Papm: A p proxy model for process systems. arXiv preprint arXiv:2407.05232, 2024.
(38) Pengwei Liu, Pengkai Wang, Xingyu Ren, Hangjie Yuan, Zhongkai Hao, Chao Xu, Shengze Cai, and Dong Ni. Aerogto: An efficient graph-transformer operator for learning large-scall aerodynamics of 3d vehicle geometries. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 18924-18932, 2025.
[39] Pengwei Liu, Qinxin Wu, Xingyu Ren, Yian Wang, and Dong Ni. A decp-learming-beee based surrogate modeling method with application to plasma processing. Chemical Engineering Research and Design, 21 211:299-317, 2024.
[40] Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in neural information processing systems, 29, 2016.
[41] Lu Lu, Pengzhan Jin, Guofei Pang. Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218-229) 2021.
[42] Varun Madhavan, Amal S Sebastian, Bharath Ramsundar, and Venkatasubramanian Viswanathan. Self-supervised pretraining for partial differential equations. arXiv preprint arXiv:2407.06209 2024.
[43] Ritam Majumdar, Amey Varhade, Shirish Karande, and Lovekesh Vig. Can physics informed neural operators self improve? arXiv preprint arXiv:2311.1388. 2023.
[44] Grégoire Mialon, Quentin Garrido, Hannah Lawrence, Danyal Rehman, Yann LeCun, and Bobak Kiani. Self-supervised learning with lie symmetries for partial differential equations. Advances in Neural Information Processing Systems, 36:28973-29004, 2023.
[45] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan and Jasper Snock. Can you trust your model's 's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019.
[46] Tobias Pfaff, Meire Fortunato, Alvaro Sanzhez- and Peter Battaglia. Learning mesh- based simulation with graph networks. In International conference learning representations, 2020.
[47] Hieu Pham, Zihang Dai, Qizhe Xie, and Quoe V Le. Meta pseudo labels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11557-11568, 2021.
[48] Charles Phelps. Phelps database, 2023. Accessed: 2024-07-31.
[49] Rahul Rahaman et Uncertainty quantification and deep ensembles. Advances neural information processing systems, 34:20063-20075. 2021.
13

---

[50] Md Ashiqur Rahman, Robert Joseph George, Mogab Elleithy, Daniel Leibovici, Zongyi Li, Boris Bonev, Colin White, Julius Bernr, Raymond A Yeh, Jean Kossaifi, et al. Pretraining codomain attention neural operators for solving multiphysics pdes. Advances in Neural Information Processing Systems, 17:1100335-10066 2024.
[51] Mamshad Nayeem Rizve, Kevin Duarte, Yogesh S Rawat, and Mubarak Shah. defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. arXiv preprint urXiv:2101.06329, 2021.
[52] Michael Rotman, Amit Dekel, Ran Ilan Ber, Lior Wolf, and Yaron Oz. Semi-supervised learning of partial differential operators and dynamical flows. In Uncertainty in Artificial Intelligence, pages 1785-1794, PMLR, 2023.
[53] Ronald Scoh. Qualitative analysis of monte carlo dropout. arXiv preprint arXiv:2007.01720, 2020.
[54] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged con- sistency targets improve semi-supervised deep learning results. Advances in neural information processing systems, 30, 2017.
[55] Akshay Thakur and Souvik Chakraborty. Md-nomad Mixture density nonlinear manifold decoder for emulating stochastic differential equations and uncertainty propagation. arXiv preprint 1XXYv22404.15331, 2024.
[56] Wouter H Vermeer, Justin D Smith, Uri Wilensky, and C Hendricks Brown. High-fidelity agent- based modeling to support prevention decision-making An open science approach. Prevention Science, pages 1-12, 2022.
[57] Yu Wang, Yuxuan Yin, Karthik Somayaji NS, Ján Drgoña, Malachi Schram, Mahantesh Halap- panavar, Frank Liu, and Peng Li. Semi-supervised learning of dynamical systems with neural ordinary differential equations: A teacher-student model approach. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 15698-15705, 2024.
[58] Yuchao Wang, Haochen Wang, Yujun Shen, Jingjing Fei, Wei Li, Guoqiang Jin, Liwei Wu, Rui Zhao, and Xinyi Le. Semi-supervised semantic segmentation using unreliable pseudo-labels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4248-4257, 2022.
[59] Ziming Wang, Xiaotong Liu, Haotian Chen, Tao Yang, and Yurong He. Exploring multi-fidelity data in materials science: Challenges, applications, and optimized learning strategies. Applied Sciences, 13(24):1317,, 2023.
[60] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforee- ment learning. Machine learning, 8:229-256. 1992.
[61] Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: A fast transformer solver for pdes on general geometries. arXiv preprint wxXiv2400.002366, 2024.
[62] Sipei Wu, Haiou Wang, and Kai Hong Luo. A robust autoregressive long-term spatiotemporal forecasting framework for surrogate- -ased turbulent combustion modeling via deep learning. Energy and AI, 15:100333, 2024.
[63] Tailin Wu, Willie Neiswanger, Hongtao Zheng, Stefano Ermon, and Jure Leskovec. Uncer- tainty quantification for forward and inverse problems of pdes via latent global evolution. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 320-328. 2024.
[64] WANG Xingjian, YEH Shiang- Ting, YANG Vigor, et al. A high-fidelity design methodology using les-based simulation and pod-based emulation: A case study of swirl injectors. Chinese Journal of Aeronautics, 31(9): (0())18555-111 1869, 2018.
[65] Shingo Yashima, Teppei Suzuki, Kohta Ishikawa, Ikuro Sato, and Rei Kawakami. Feature space particle inference for neural network ensembles. In International Conference on Machine Learning, pages 25452-25468. PMLR, 2022.
14

---

[66] Tal Zeevi, Rajesh Venkataraman, Lawrence H Staib, and John A Onofrey. Monte-carlo frequency dropout for predictive uncertainty estimation in deep learning. In 2024 IEEE International Symposium on Biomedical Imaging (ISBI), pages 1-5. IEEE, 2024.
[67] Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi semi-supervised learning with curriculum pseudo labeling, Advances in neural information processing systems, 11118888-18419, 2021.
[68] Xingchen Zhao, Niluthpol Chowdhury Mithun, Abhinav Rajvanshi, Han-Pang Chiu, and Supun Samarasekera. Unsupervised domain adaptation for semantic segmentation with pseudo label self-refinement. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2399-2409, 2024.
[69] Yang Zou, Zhiding Yu, Xiaofeng Liu, BVK Kumar, and Jinsong Wang. Confidence regularized self-training. In Proceedings of the IEEE/CVE international conference on computer vision, pages 5982-5991, 2019.
15

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
We have made clear claims about contributions and scopes in the abstract and introduction.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
16

---

Justification:
We have the seperated limitation section in the appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
We provide detailed theoretical proofs in the main paper and appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We describe the details of our model and training strategy in the paper for reproduction.
Guidelines:
17

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We offer the implementation code reproduce work.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
soteeeesubmmiiioooooooooooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//ipps.cc/publi///guu
CodeeSumiiii wettubbmmssiiioi ssionoolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
18

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: we have introduced experiment settings and details, and even add more details in the appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to a level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report enror bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We do not report specific error bars for the following reasons: (a) In fluid dynamics modeling, using a fixed random seed shows consistent performance, and the performance difference with different seeds is small.
(b) Related work in this field does report error bars.
(c) ensure fairness, we fix all random seeds and conduct experiments on the same machine.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have discussed the information about computer resources.
19

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't I make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttss:/ http://pp//eeareep.....oaaaaaaaa
Answer: [Yes]
Justification:
We follow the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The UMPL. method not only makes significant contributions to academic research but also shows broad potential in practical applications. It helps address distribution changes in complex PDEs systems.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
20

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do release any data or model that has high risks for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-us use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have cited the original paper that produced the code package and dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pwperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide documentation with code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
21

---

14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our work does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
22

---

m Table 4: Table of notations.
Notation
Meaning
Problem Formulation
spatial domain for PDEs
g
the mapping between input space and solution space the input function space the solution function space the input function for neural networks the input geometry spatial coordinates the physical quantities the input geometry the coefficients, such as viscosity
A
S
a = (g.a,5)
9 N
E R N
y E NXC.C
the corresponding solution the encoder, preocessor and decoder for neural operators
E, P,P,D
2 E RNx
the latent states
predicted latent states and corresponding output solution Labelled data
Di {11) v Inf
Dn (ou)
unlabelled data
Ui
the geometric neighborhood of the point i
Notations in UMPL Training
Ci 9) (u.v.00.0) CF
the relative L2 error for labelled data the heteroscedastic regression loss for pseudo labels the feedback loss for the teacher the EMC estimation time
Pe, Ōz
Pr, oz
the expectation estimation and variance of MC Dropout at time the uncertainty estimation from the model
E(g)
random field obtained through Gaussian smoothing over g the prediction of the model at time
fz)
((w)
sample from the troooout-iduuce posterior at time
0g.
the parameters for the teacher and student the scaled negative log- log-likel (NLL) loss improvement for student from aseudo-label training the gating factor of forgetting in EMC Dropout sampled Gaussian noise at point i the smoothing scale factor the binary mask for the progressive uncertainty filtering the masking ratio
- log (()
h
a
rs
k Notations in Theoretical Analysis
Raitng (F)
the empirical Rademacher complexity of the hypothesis class J. the weight of the pseudo-labeled component the weight decay coefficient expected model precision dropout rate the expected relative error of the pseudo-labels
Ap
Xa
p
erel
pseudo
Methodology
PL
pseudo labels
UMPL
uncertainty-informeed meta pseudo labels
PUF
progressive uncertainty filtering
SA
supervised anchoring
EMA
Exponential Moving Average
EMC
MC Dropout with EMA
23

---

In this appendix, we complement the main paper with additional theory, implementation details, and extended experiments. (ejendixAJsurveey related work and situates our contributions. Appendix
B provides supplementary theory. including Bayesian interpretation of MC Dropout, proofs of Theorems 3.1 and 3.2, gradient derivations for the teacher update, and an analysis of feedback- gradient bottlenecks under a deterministic teacher. Appendix [C] reports training configurations and optimization details, while Appendix [D] describes the compared approaches. Appendix [B details datasets, model architectures, and experimental settings. Appendix iraluations-addiiionn comparisons, computational cost analysis, robustness assessments, and ablations-ales with sensitivity studies uncertainty perturbations, PUF thresholds, feedback strength, and other hyperparameters to elucidate model behavior. Finally, Appendix G discusses broader impact.
F presents extended
A
Related works
Neural Operator for surrogate modeling. Neural operator-based methods, such as Deep Operator Networks (DeepONet) [41], Fourier Neural Operators (FNO) [33], and MeshGraphNet (MGN) [46),
function spaces, providing end-to-end frameworks for modeling parametric partial differential equations (PDEs). Transformer- based architectures, including Graph Neural Operator Transformers (GNOT) 17.. Transolver [61], and AeroGTO [38), further push the boundaries of dynamic modeling by effectively handling large- scale datasets and capturing complex spatiotemporal dependencies. These models have demonstrated strong generalization capabilities and offer a scalable, data-driven approach to surrogate modeling 277.
have shown promise in learning mappings between infinite-dimense
Semi-supervised learning for Regression. While SSL has seen significant success in image- related tasks, such as classification and segmentation [1] 67 58), its use in regression problems remains relatively underexplored (28). Methods like SSDKL [22] utilize pseudo-labeling along with uncertainty minimization. Additionally, ensemble-base consistency strategies have been proposed to improve regression performance [12]. Other efforts have focused on self-supervised learning methods to learn helpful information from heterogeneous sources 44 500 or integrating pseudo labels to enhance the accuracy of physics-informed neural networks (PINNs) when solving ordinary differential equations (ODEs) [8 43). Noteworthy examples include MPLR (26) and Noisy TS (57), which extend the meta pseudo label framework [47 to handle tabular data and ODE-based regression. However, these approaches often face challenges in generalizing to high-dimensional PDE systems due the increased complexity of solutions 22 This indicates an urgent need for more scalable
SSL frameworks designed explicitly for dDiveriven problems.
Uncertainty Estimation. Uncertainty estimation is critical in deep learning tasks that demand reliable generalization and safety-aware decision [1 [5]. Uncertainty can be broadly categorized into epistemic uncertainty, arising from limited data model capacity, and aleatorie uncertainty, caused by inherent noise in observations
(21). Bayesian neural networks (BNNs) 6 provide a principled way model both types (25. but exact inference is often intractable due computational challenges. Practical approximations like the Monte Carlo (MC) Dropout enable efficient estimation of epistemic uncertainty by sampling stochastic subnetworks at test time [14. At the same time, ensemble methods aggregate predictions from multiple models to improve robustness and calibration 30 49 [2]. Recent studies propose latent-space methods that evolve compact representations of system states and their associated uncertainties (63), thereby improving uncertainty quantification through more expressive and structured representation learning.
B
Supplementary Theoretical Analysis
B.1 Bayesian Interpretation of MC Dropout
Given a training dataset Dí, the goal of Bayesian inference is to compute the posterior distribution over neural network weights p(w Di), which captures epistemic uncertainty about the model parameters. For an evaluation input x. De. the predictive distribution of the model output ÿe is given by marginalizing over this posterior:
p(ūel | Le. D.) = p(ue Ixx, w) p(w D.) dw
(14)
24

---

where p(de Ze, w) denotes the model output (likelihood) under fixed weights and p(w) De) is the posterior distribution over weights. Since this integral is analytically intractable in deep neural networks, MC Dropout is employed an efficient approximation method. MC Dropout interprets dropout applied during both training and inference as variational approximation. variational distribution g(w) over the weights by randomly zeroing out units with a Bernoulli mask. Formally, for each weight matrix W,, we define:
defines a
W = P: diag(zi),
Fi.j Bernoulli(p:)
(15)
where P; are the learned variational parameters and 2; are dropout masks. This results in a mixture distribution over sub-networks, where each sample corresponds to different realization of the active units, Under this approximation, the predictive distribution is computed as:
f pl Fe; w) q(w) dw
plue (e) Rs
EE p(ue Tes we), wy (ww)
(16)
T
To quantify predictive uncertainty, we estimate the predictive variance by computing the sample variance of the model outputs across T stochastic forward passes, along with the contribution from the assumed model noise. Let " denote the model output at the /-th sample. Then the predictive variance is estimated as:
T 0
1 T
T
T
T
Var(ße xe) ~
7
(1)
(
(17)
T
where the first term 7- accounts for the uncertainty under a Gaussian likelihood assumption with model precision т, and the second term corresponds to the epistemic uncertainty due to model parameters. The overall predictive variance thus combines both sources of uncertainty. The model precision can be derived from the variational framework as function of weight decay Ad, dropout rate p. prior length-scale and dataset size N, as follows:
pl?
T 2N
(18)
This provides principled way to interpret standard dropout training as approximate Bayesian inference, enabling deep models to produce not only predictions but also reliable uncertainty estimates with minimal computational overhead.
B.2 Proof of Theorem 3.1 and Theorem 3.2
T.eorem3.1. Assume { satisfies ft = f(; wi), where ~ ((w) is i.i.d. and:
ft E [a, +1fl S M / [==
(19)
We define the exponential moving average estimate as: M++1 = aft + (1 - a) #1 =1= = fi
(20)
Then for any time f, the EMA estimation satisfies the following mean square error bounds:
E((( 11) < 2 a
a
Var(J)
(21)
Proof: For e; = ft μ4, we have:
ex11 e+11= = (1 a1- + a(Jz - 11)
(22)
Square and find the expectation:
Ejez11] (1 11-aj E) + a a² 2
(23)
25

---

and El(fi --) Var(f) we have:
B Ble++ll S (1 (le))) Var())
(24)
This is non-homogeneous linear recurrence relation with constant term.
We set 2i = Elc]) a = (1 a)), a² Var(f), then have:
xe+1 az;
(25)
Expand the first few items:
Var(f)
22 0rl +
23 S azg b f aaar + +b + d²x1 ab +b
24 da + + ab +b
(26)
It S d'~1x) 1 + b(1 + a + a² + + 2)
a'-1a 1 2 + b. 1 - a
1-a
When ++ 00, (1 ax)² we have:
a² Var(f)
lim Fz. 5
(27)
800
1 (1 a))
Therefore we have:
a²
lim B[e]]
VVr(J) = a Var(J)
(28)
10000
2a a²
2 -
The upper bound its solution can proved by recurrence inequality as follows:
lim sup Ele? < a t
-2a Var(f)
(29)
which complete the proof.
Beyond the convergence of the predictive mean, we further analyze the consistency the variance estimator in EMC Dropout. We consider the variance estimate à² updated recursively via the EMA formula:
. a(fe 1 + (1- a)o?
(30)
where fe denotes the stochastic prediction at time with dropout, and fu+1 is the EMA mean estimator. Let μr E[fi] and x² Vift] denote the true predictive mean and variance under the dropout-induced approximate posterior q(w), and define the squared deviation = (ʃz - - The goal is to quantify the asymptotic error of ae with respect o², We analyze the mean squared error [[ - 0²)²] under the assumption that fz is stationary and ergodic. Following second-order error decomposition and bounding cross-terms involving the EMA mean eπror & fr - 44, we derive the following bound:
lim limsup 1-000
[ 02)2]
S 2 2-a Varl(fi =
(31)
This result mirrors the convergence behavior of the EMA mean estimator and confirms that the variance estimate of asymptotically consistent. The bound depends on the variance of the squared deviation, which reflects the kurtosis of the predictive distribution. As in the case of fr. the smoothing factor a E (0, 1) governs the bias-variance trade-off: smaller values yield more stable estimates with slower adaptation, while larger values allow faster response at the cost of higher variance. Equation 31
26

---

thus establishes a theoretical guarantee for the reliability of EMC Dropout in approximating the uncertainty captured by standard Monte Carlo dropout, without incurring the cost of repeated forward passes.
Theorem 3.2. Generalization Bound under Relative L2 Loss in Semi-Supervised Regression
Let be a hypothesis class neural surrogate models with fixed architecture and bounded parameters. Given a training set consisting of ny labeled samples and nh pseudo-labeled samples, then for any E (0, 1), with probability at least 1 - ò, the following inequality holds for any f(0) E F, including ((: 0g):
log(1/6) E (x.v.) (Ci( f(ra 0))] < Csemi (f())+ Rmin (F)+e.
+ Ap s
ppeeudo (32)
2(m + m..)
where Yu E Du is the unlabeled data. (f(m) is the empirical loss on both labeled and pseudo- labeled data for student model. Rm,+n, (F) is the empirical Rademacher complexity of F. L 2M
Vinin
the Lipschitz constant of the loss function. is universal constant. X E [0, 1] is the weight the pseudo-labeled component. ezel "peeddo == EgnDD [Li (Wa: (T); the expected relative error the
pseudo-labels.
Proof: Let ((f(0) = ((x.g.) [Cr(ya, f 0))] denote the expected loss under the data distribution. and let (() = nr+ - (v: ((;; 0)) + Σド= 1= (0 fzz, 01) denote the empirical loss over the ny labeled and ny pseudo-labeled examples. Then we assume the loss function C; is L--Lipshitz with respect to fa;; where L 2M for [f(x) yl s M.
Vinn
From the standard Rademacher generalization bound, we have that with probability at least 6:
log(1/5)
L(f(0)) & [(f())) + 2L Rn (F) + c. V 2(m4 - nw)
(33)
Now, the pseudo-laheled loss [()( is computed based on ăj rather than true labels. To account for the potential error introduced by the pseudo-labels, introduce additional term, which quantifies the expected relative error between pseudo-labels and the ground truth.
Thus, we obtain Eq 32
B.3 Details about gradient for the teacher's update
The traditional optimization process using pseudo tags be described as follows:
05 argmin Er. [C(T(Fui 0T) S(a, 0s))]
(34)
0g
(0r.0))
Here, T' and denote the teacher and student models, respectively; 0 represents the model parameters; C denotes the unlabeled loss function; and 0 refers to the optimal parameters of the student model obtained via the pseudo-labeling strategy. This process is performed on unlabeled data, as traditional semi-supervised learning frameworks typically combine the supervised loss on labeled data with consistency loss on unlabeled data.
To update the teacher model, we leverage the student model's 's performance on labeled data, which can be mathematically quantified the loss function evaluated the labeled samples. This relationship can be expressed as follows:
Ez,,,r [C Szz; 0F1))] = Li (0
(35)
This can be interpreted as functional form where 6T serves as the input variable and 0g as the optimization parameter. Accordingly, the optimal student parameters can be denoted (0T(()))
The loss in Eq 35 above can be defined as (((0FL(0r))
27

---

In this manner, the meta- medalioo process is accomplished: the parameters of one model are regarded as imputs a functional expression, whereas the parameters of another model optimization variables. By minimizing loss function defined over these variables, the optimal solution can be effectively derived.
Accordingly, updating the teacher model within this framework entails minimizing the loss function specified in Eq 35
min (,(05" where (0r) = argmin Lu ( 0s)
(36)
8s
The argmin operation above is not amenable to gradient- based optimization, as requires waiting until fg converges its optimum before proceeding the next step. This dependency clearly disrupts end-to-end training. To address this issue, this paper proposes one-step approximation:
(0T() Os L0T,, 0s)
(37)
Here, the goal becomes:
min Li(0s ns VegL(0r,63) 0s))
(38)
Pr
Therefore, if the gradient of dy can be computed for this expression, it becomes feasible to apply gradient descent for end-to-end optimization. Let the above equation be denoted CF; the detailed solution process is as follows:
dCF a Li (u. S
(2;; Byg- (T (0s nss V Cu( S(z,; (()))))))))
(39)
00T
00T
For convenience, we denote the updated parameters of the student model as g'g:
= =5 3T [0s - 7s Vos Cu (1T-, S(rai 0s))]
(40)
Then Eq 39 can thus be rewritten
OCF
ô 00s
00's
Li (2i, S(z;
10r
0g
00T
(41)
1x/2
1x|S|
Here, we denote the shape of Or as (T') x and the shape 0g as |S] x 1, respectively. The left-hand side of the above equation be efficiently addressed using gradient descent, as the gradient can be approximated by computing the difference between the student model parameters before and after updating on the labeled dataset.
03 = 0s - 175 Vo. Vog S(x; 0s))
(42)
Therefore, the gradient can be readily approximated by computing the difference between the model parameters 0 before and after the update.
ns V Cil,, S(z 0s)) 05
(43)
So now focus on the right side of the equation:
28

---

00's
0
2ggh~ (Tuutrr)
[0s n/s vg Lula (axi 0s))]
307
80r
|
(44)
, 0g))
Fag ~ (x.:(r))
0g - 1ls
d0
00s
Ag 0g
gs. (U = ac., S((..:0)) J0g
09
we denote gs gradient descent, and the equation above becomes:
and gT == 30,
gs is easily computable via
08=08
a0's
-1s
E pg -T(ws:r)) gs (VT") ]
(45)
00T 00,
It is important to note that gs itself does not depend on 0y; rather, the dependency arises from the fact that 9u is generated through a pseudo-labeling mechanism that is parameterized by the teacher model. This process involves the application of the Leibniz integral rule:
0 00r
E 9I==T(Te (a. gs (1%
Σ
Iua Br) gs (1% p(uu
00,
5u
Σ
p( Iui (T) gs (3%1e)
00
(46)
Σ p(uu Iaa 0r)
log p(u
Iay Or) gs (iT-)
001
9u
E o (T
Es
gs
(0Th
log
p(O I 0T)
D0,
At this point, the partial gradient can be computed using the NLL loss term. For clarity, we now organize the expressions the left- and right-hand sides of the equation as follows:
OLr dLi (y,, S (11;; 0's)
O0
a0r
00g
der
0s=8's
||] x
1x|9|
dLi (Mi, (z:; 8's))
775
00g
E Bg TTTT (
T gs
d0T
log p(u"
Iu OT)
By 0%
|8|x1
1x
[x|8]
(47)
The expectation term in the above equation can only be approximated through sampling, which is typically performed over mini-batches. Taking a single sample from the batch as an example, its corresponding gradient is given by:
29

---

T
OLF
0Li(M, ((x;;'s))
(1 Strui 0s)
log p(u (au 0r)
=1S
00r
00s
00s
00r
0gm0s
=1ss (V og Li ((m; 022))
Vog fuu (Wa" S (wzu: 0s)))
V VorP(Uu if ( (eni 0r)
(48)
scalar:sh
<h Sch.NEE (NLL(
which we complete the proof.
Then we show the Taylor estimation for h:
As know 0'g 0s v7. Let v/ 0os Lu ((r,; 0s)), according the Taylor's equation
we get:
L(0's) L(0s 11 L(0s) 175 (og
I((g) 7s V 0. L. WTT
(49)
-[(8g) 1TS 0g Lu - 24
-[(0s) h
Therefore, we get the estimation of h:
h L(0s) L(0'))
(50)
B.4
Computational Bottlenecks in Feedback Gradient Computation with Deterministic Teacher
In the absence of the proposed teacher, the gradient of the feedback loss be computed directly. However, this leads to substantial computational overhead and proves inefficient under the widely adopted reverse-mode moorse-modeaaoo automatic differentiation framework.
Specifically, based on the problem formulation provided in Section 3.2 expression for directly optimizing the feedback loss:
we derive the explicit
0LF
0 00g
00's 00
Li (2R, S(1100s))
80,
0==88s
1x/2
1x|8|
(51)
OC,
-]s
06 g
00
00s
1x|s]
|
We can further expand the unlabeled loss in Eq the following derivation:
51
For brevity, the subscript u is omitted throughout
|| 01,112
CT
105
651)
1
ΣΣ
+ log
Nij
(0
11 xlll
(52)
ac.
1 ΣΣ
aS(r:j; 0s)
00g
NoJ
00g
Combine Eq 51 and 52
get:
30

---

OCF -|
dLi
(S(z 0s)
(ayy 0T)
00T
06
a0s
10T
(53)
1x/7] 1x
|s|x1
[x|T]
The direct computation of 53 requires evaluating gradients each element in the rollout, leading to prohibitive computational and memory costs. Given a batch of size B, with TI' time steps, N points, state dimensionality C, and parameter dimensionality S, the memory complexity scales as ( x T x N x C'x x x T), due to the need to preserve the entire computational graph. To overcome these limitations, Section 3.2 introduces UMPL get interpretable approximation to the feedback gradient. In particular, we adopt the REINFORCE rule (60) to estimate the feedback gradient, which removes the backpropagation path from the student to the teacher and substantially reduces memory and computation, at the expense of longer runtime from repeated forward and backward passes.
C Training Details
Prior to the formal UMPL training process, we perform pretraining stage without dropout, yielding an initial model with parameters denoted as 0o. The teacher is initialized from the warm-up weights = warma, whereas the student is randomly initialized (0g = Orand) for the UMPL stage. To construct memory bank for the unlabeled data D 1ny
enhance training stability and robustness,
storing the mean [ you and variance {o obtained from the teacher's s initial stochastic forward passes. This memory bank enables principled estimation of epistemic uncertainty in the teacher network by leveraging the EMC framework, thereby facilitating a more robust distinction between intrinsic model uncertainty and stochastic noise arising during training.
Furthermore, we adopt Progressive Uncertainty Filtering (PUF) strategy to suppress unreliable pseudo labels. Specifically, each training step, compute a binary mask E 10 based on the predictive variance estimated via EMC Dropout. The top-k% most uncertain predictions (m, = 1) are excluded from both the student's ' loss computation and the feedback used update the teacher. As training progresses and pseudo-label quality improves, the masking ratio gradually decreases, allowing the student to incorporate broader range of samples and provide more informative feedback to guide the teacher's s update.
The unified training process consists of the following steps:
1. Get predictions from the teacher:
17 (Or) T(x Or)
(54)
Update predicted uncertainty with Eqn. 30 and get uncertainty of
3. Generate pseudo labels:
T = (Tr) + E(g) ot.
(55)
4. Calculate the error of the student model on D and Train the student with unlabeled loss (C%):
cf = Li(n,, ((I1,0s)
L C.(m TT m S(x 0s))
(56)
0' Os ns V 2s
5. Calculate the error of the updated student model D and feedback loss with
C =La(un (x1,0ss))
(57)
Cpp = - L'y.
31

---

6. Train the teacher with both labeled loss D/ and the feed back loss p'p:
C Ly [T,,, 0r))
(58)
8 - 0r or Vor (C4 + Cpp)
To ensure stability throughout training, we apply Supervised Anchoring (SA) strategy where the student consistently trained on labeled data after the last step, regardless of pseudo-label reliability, This mitigates error propagation during self-training and is especially crucial under distribution shifts where pseudo-labels may misleading. Beyond SA, our algorithm minimizes model-collapse risk by coupling divergent teacher-student initialization and independently sampled labeled anchors with an uncertainty-guudee feedback loop (student's 's aleatoric signals) and uncertainty aawaee pscudo-labeling (teacher's s epistemic perturbation with progressive uncertainty filtering), curbing error reinforcement and stabilizing convergence.
D Details of Compared Approaches
Pseudo Labeling(PI.) 31], In this baseline, the teacher model is fixed throughout training and is used to generate pseudo labels for the unlabeled data. The student model is then trained jointly on the labeled data and the pseudo-labeled unlabeled data with a supervised loss. While this method leverages additional information from the unlabeled set, our experiments reveal that it can underperform compared to fully supervised training. This performance degradation is primarily attributed to the teacher's 's inaccurate noisy predictions the unlabeled samples, which may introduce erroneous learning signals and ultimately misguide the student model.
Mean Teacher[54). The Mean Teacher framework improves upon standard pseudo-labeling by dynamically updating the teacher model as an exponential moving average (EMA) of the student's parameters during training. At each iteration, the teacher generates pseudo labels for the unlabeled data. the same time, the student is trained minimize both the supervised loss of labeled data and the consistency loss concerning the teacher's predictions. Despite its stability advantages, Mean Teacher has a similar drawback to standard pseudo-labeling. If the teacher's predictions on unlabeled data are inaccurate, they can propagate noise and mislead the student. In our experiments, set the EMA decay coefficient to 0.99, which balances temporal smoothness and adaptation.
Noisy Teacher-Student (TS) This method extends the standard teacher-studen framework by introducing stochastic perturbations to the teacher model and incorporating feedback from the student to guide the generation of pseudo labels. Specifically, the teacher produces rollouts under added Gaussian noise, which are then used to train the student model. The student's performance on labeled data provides feedback signal to update the teacher, helping to mitigate the accumulation of errors caused by inaccurate or noisy pseudo labels. This feedback mechanism improves the student's stability and prediction accuracy, particularly in low-label regimes. In our implementation, the noise magnitude applied to teacher's outputs is fixed at 0.05, following original formulation.
E
Detailed description of datasets, models, and experimental settings
E.1
Datasets
Table 5: Basic information of datasets. Note that No. train includes both labeled and unlabeled data.
Datasct
Domain
Nodes(avg.)
Timesteps
Operator Mapping
# Phys.
No. train/test
Darcy Flow
2D
2290
a(x) + w(x)
u [0]
1000/200
Stationary Lid
2D
1475
BC.+Puram + u(w)
[U V. P
861/100
Time-dependent Lid2d
2D+time
IC.+Parum u(a,
UUV.P.
900/100
NSM2d
2D+time
2630
201
IC.+Param - a(x, T'
U.
900/100
Black Sea
2D+tme
5000
15
IC.+Param - u(2, T)
u
[T]
1615/238
Plasma ICP
2D+tme
3424
81
IC.+Pram - a(a,
500/150
Ahmed
3D
110k
Geo.+Puram
500/51
32

---

As shown in Table S we employ seven datasets spanning diverse domains, such as fluid dynamics. plasma ,ad large-scale 3D geometry simulations. A summary of the datasets, their underlying operator learning tasks, and data generation sources is provided below.
Darcy Flow. We adopt the Irregular Darey dataset ( which is based the Darcy flow equation over 2D irregular domain with a thin rectangular notch. The governing equation is V (a (uVu) f. where the source term fixed to f = The input diffusion field a(x) generated by Gaussian random field μ ~ (0,, (-A + 251)- followed by piecewise transformation t(p) to produce binary-valued coefficients: t(μ) 12 if μ 2 O, and otherwise. The computational mesh contains 2290 nodes with Dirichlet boundary conditions. The dataset provides 1200 labeled samples, of which 1000 are used for training and 200 for testing.
Stationary Lid & Time-dependent Lid2d. The Lid2d dataset is based on the lid-driven cavity problem, canonical benchmark in computational fluid dynamics that characterizes recirculating flow induced by a moving top boundary (the "lid") within square domain. A constant horizontal velocity is applied the lid, while the remaining three walls follow no-slip boundary conditions. We construct two versions of this problem: a stationary (stcady-state) version and a time-dependent (transient) version, each governed by different formulations of the incompressible Navier-Stokes equations.
Stationary Lid2d: In the stationary case, the flow is assumed to have fully developed and no longer changes over time. Therefore, the governing equation reduces to the steady-state Navier-Stokes form:
u Vu +
Au
Vp = 0,
Re
(59)
V u 0,
where u = (u(2, y), v(x, v)) is the velocity field, p(r, y) is the pressure, and Re is the Reynolds number. The spatial domain is discretized using non-uniform triangular mesh with 1475 nodes. The steady-state dataset generated over a uniform grid of Re E [100, 10000) and lid velocity E (0.1,100, forming a 2D parameter grid of 31 31 samples. Among them, 81 samples centered in the grid are selected for training, 100 are randomly chosen for testing, and the remaining 780 are used unlabeled data.
Be
10000
(c)
Rs 500
Tax 5000
Figure 5: Visualization of the Lid2d dataset. (a) Velocity profiles along x = 0.5 and y = 0.5 in the stationary Lid dataset. The hollow circles represent reference values from the literature [16], showing excellent agreement with our simulation results. (b)-(c) Mesh structure and velocity magnitude
v 2 + uz visualization for stationary and time-dependent Lid2d datasets, respectively. For the time-dependent case, the final time step is visualized.
Time-dependent Lid2d. In contrast, the time-dependent case follows the full unsteady Navier-Stokes system:
du
1
Vu
Au Vp.
I1
Rlc
(60)
2
0.
33

---

subject to the initial condition (u, v. = O and the same boundary conditions as in the stationary case. Simulations are conducted over non-uniform triangular mesh with 730 nodes, spanning total simulation time of 10 seconds with 101 time steps. We consider two Reynolds numbers (Re = 500, Re 5000) and uniformly sample lid velocities v E [0.1, 1.0]. For each Re, 1000 transient trajectories are generated. Among these, 100 samples are fixed for testing, while the remaining 900 are treated as unlabeled data for semi-supervised learning. Visualization about this dataset is shown in F
NSM2d. The NSM2d dataset is derived from the 2D incompressible Navier-Stokes equations with an additional magnetic force term. The simulation follows the same physical setting as the original reference [37] including the time-invarian magnetic intensity, inflow velocity profile, and boundary conditions. The domain is [0, 4] X [0, 1], with jet inflow imposed at = O. centered at a vertical position Yo E [0.4, 0.6), and Reynolds numbers ranging from 100 1500. While preserving the original PDE formulation, we discretize the spatial domain using a non-uniform triangular mesh with 2630 nodes. One thousand simulations are generated using COMSOL, each consisting of 201 time steps over 10-second interval. A visualization of the dataset is shown in Figurd6
(a)
(b)
Figure 6: Visualization of the NSM2d dataset. (a) Mesh of the NSM2d dataset. (b) Visualization of the velocity magnitude
r2 + u at the final time step.
Black Sea. The Black Sea dataset comprises daily real-world measurements of ocean currents and temperatures [35]. We use data from 01/01/200 /017 to regular grid with spatial resolution of 1/27 x 1/36", but due to the irregular geometry of the sea, fewer than 50% of grid points lie within the valid ocean region. We apply Delaunay triangulation to the valid region to construct mesh, resulting in spatial discretization with 5001 nodes consistent across all models. Following (20), we use the water temperature at a fixed depth of 12.54 meters as the predictive variable.
The raw data is provided on
For model evaluation, we adopt a temporal extrapolation and interpolation split. Specifically, we designate the final three months of the dataset as the extrapolation test set. In comparison, three additional non-consecutive months are randomly selected from the earlier portion of the dataset as the interpolation test set. The remaining data is used for training under semi-supervised setting. During training, the model takes the temperature field of a single day as input and is trained to predict the temperature evolution over the next 5 consecutive days. For testing, the model is evaluated on its ability to forecast the next 14 days, given only the temperature from a single input day. The dataset is available to download from the Copernicus Institute The geometry of this problem and the solution field are shown in the following Figure 7
8000
13.4
17.9
224
20.0
Figure 7: Visualization of the Black Sea dataset with mesh and the solution field T.
Available at https://data.marie.com http://dhh....iiic.oooomooeess/d MULTIYEAR PHY _00000000000 004/description
34

---

Plasma ICP. This dataset models an argon/oxygen inductively coupled plasma reactor, incorporating multiphysies phenomena such magnetic fields, plasma dynamics, laminar flow, and heat transfer. The model is derived from the COMSOL. Application Library with partial chemical reaction mechanisms adopted from 48
To construct a multi-fidelity learning scenario, we generate two
levels of simulation fidelity:
Low-fidelity simulations conducted sparser mesh with approximately 10,000 elements and exclude specific physical and chemical processes.
High- Sigh-fidelity simulations use a denser mesh with around 25,000 elements and preserve the complete physical mechanisms, providing more accurate but computationally expensive results (about 1 hour per sample for computing).
Varying reactor conditions, including chamber pressure, inlet flow rate, argon/oxygen ratio, and chuck temperature, generate 650 high-fidelity simulations. Among them, 500 samples used as labeled or unlabeled data during training, while the remaining 150 samples are reserved exclusively for testing under distribution shifts. The training protocol follows standard multi-fidelity learning pipeline: models are first pretrained on the low-fidelity dataset, which provides broader but less detailed approximation of the physical system. This is followed by fine-tuning and evaluating the high-fidelity dataset, enabling the assessment of model generalization across fidelity gaps under limited supervision. Each simulation spans total physical time of second, discretized using 81 non-uniform time steps. lispaace
The time stepping follows logarithmic scale, specifically both early fast transients and slower dynamics. A visual comparison between low- and figh-fidelity solutions is presented in Figure g highlighting differences mesh resolution and solution fields.
0), better to capture
(8)
(b)
Figure 8: A visual comparison between low- and high-fidelity mesh and solutions. (a) shows the low-fidelity data, and (b) the high-fidelity data. From left to right, each panel presents the mesh electron density (nz), electric potential (Ø), and gas temperature (T), highlighting the differences in both mesh resolution and solution fields across fidelities.
Ahmed. The Ahmed Body dataset is constructed from parametric variations of simplified vehicle geometry, characterized by six shape parameters-lengee width, height, ground clearance, slant angle, and fillet aodius-alone with the inlet velocity to account for changes in Reynolds number. 551 design configurations are generated using Latin Hypercube Sampling, with 500 for training and 51 for testing. Aerodynamic simulations performed using OpenFOAM with the SST k-w turbulence model. The resulting surface pressure distributions are used to compute the drag coefficient Ca based on the following standard integral:
2 C. = 12 Jg
p(x) (n(x) dx,
(61)
where e is the inlet velocity, the frontal area, p(z) is the surface pressure, nxx) is the surface normal, and i is the streamwise direction. Details of the geometry and pressure computation follow
Availlabee at reactor- 109191
argon -xygen inductively plasma
35

---

standard aerodynamic practices and can be found at Modulus Datasets This dataset defines mapping from vehicle geometry and flow conditions to aerodynamic drag, using the surface mesh (about 10k nodes) as model input. A visualization of the mesh and pressure is shown in Figure 9
(a)
(b)
Figure 9: Visualization of the mesh (a) and surface pressure (b) for the Ahmed dataset.
E.2 Models
This section presents the models employed in our experiments, including two point-cloud-hased architectures (DeepONet[4]) and Trans) and two mesh-based architectures (MeshGraphNet (M6)) Aerogto 38 ). To ensure a fair comparison, we retain the original network structures for all approaches without modification. All models are trained and evaluated autoregressively for temporal prediction tasks, where future states are recursively predicted based on previous outputs. In addition the primary output, estimate data-dependent uncertainty by attaching an auxiliary decoder to each model, following the model-based me3 This auxiliary decoder learns to predict the predictive variance conditioned on the input data, allowing the models to provide uncertainty estimates alongside pointwise predictions.
DeepONet(41). DeepONet is neural operator architecture composed of a branch network that encodes the input function and a trunk network that encodes spatial coordinates. In our experiments, both networks consist of three fully connected layers with GELU activations and 128 hidden units. The outputs of the two networks are combined via element-wise addition and passed through a two- layer output network produce the final prediction. We train the model using the Adam optimizer, with a cosine annealing learning rate schedule that decays from le-3 to le over the course of training epochs.
Transolver(6]]. Transolver is a transformer-based architecture tailored for learning physical dy- namics on irregular spatial meshes. It represents mesh nodes unordered sequences and employs a specialized slice-token attention mechanism to capture both local and global interactions. In our implementation, Transolver consists of 8 transformer layers with 8 attention heads, hidden dimen- sion of 128, and a slice-token mechanism with 64 slices. The input combines node coordinates and physical state variables, optionally augmented with time embeddings for temporal predictions. All features are first processed through an MLP and added to a learnable positional placeholder before passing through the transformer blocks. The model is trained with the AdamW optimizer, with an initial learning rate of 54 and a weight decay of 15
(cGG MGN is graph neural network designed to learn physical dynamics over unstructured meshes. models each simulation domain as a graph, where nodes represent mesh points and edges encode spatial relationships. The architecture consists of an encoder that embeds node and edge features, a processor composed 12 message-passsingg steps to propagate information across the mesh, and decoder that maps the final node embeddings predicted physical states. All MLPs in the model use 2 hidden layers, 128- dimeeniiai embeddings, ReLU activations, layer normalization, and dropout for regularization. In our experiments, MGN is trained with the Adam optimizer with a learning rate that decays from - to 1e-5
AeroGTo AeroGTO is hybrid graph- traph-tranffommer architecture designed for learning aerody- namic responses over unstructured 3D meshes. It combines local message passing via a graph neural network (GNN) with global correlation modeling through projection-based attention mechanism
Available at ahmed modyy_tss)
htpp:s/ww.......aooooooooooooooe datasets
36

---

(GTO-Atten). The model encodes node and edge features using MLPs with Fourier- Fourier-based positional encodings. processes them through sequence of 4 Mixer blocks, each composed of GNN and a transformer-style attention layer with 128 learnable tokens and 4 attention heads. With a hidden dimension 128, AeroGTO captures fine-grained geometric structures and long-range dependencies across the mesh. The final predictions are produced through residual decoder with learnable scaling factor. The model is trained using the AdamW optimizer, with a learning rate that decays from le to le-6.
E.3 Experimental Details
This section outlines the experimental settings, including training schedules, data splits, and task- specific configurations.
Training Schedule. We establish a fully supervised baseline ("Supervised Only") for each task type, with training epochs determined by task complexity. Specifically, stationary tasks are trained for 500 epochs, transient tasks for 1000 epochs, and the Ahmed dataset trained for 300 epochs. In the case of distribution shift experiments, models are first pretrained on source-domain data for 1000 epochs, followed by fine-tuning on a small, shifted-domain subset for an additional 300 epochs. Semi-supervised methods, including Pseudo Labeling (PL), Mean Teacher, Moisy Teacher-Student, and UMPI., follow the same total training schedule as their corresponding supervised baselines. semi-supervised training begins from the supervised initialization and continues for the same number of epochs.
Split. For semi-supervised learning, we randomly select a fixed proportion E [10%, 30%] of the training data to serve as labeled samples, while the remaining data is treated as unlabeled. The Stationary Lid dataset is an exception, where 81 samples are consistently selected as the labeled subset, and all other samples are used as unlabeled data.
Distribution Shifts Setup. We adopt a two-stage training strategy to evaluate model robustness under distribution shifts involving pretraining and fine-tuning across distinct data distributions. Specifically, in the Time-dependent Lid2d dataset, we perform cross-Reynolds transfer, where models pretrained on Re 500 are fine-tuned on data from Re 5000, and vice versa, simulating distribution shift in flow regimes. In the Plasma ICP dataset, we evaluate fidelity transfer, where models are pretrained on low-fidelity simulations and then fine-tuned on high-fidelity data assess generalization across simulation accuracies. In both cases, pretraining is conducted for 1000 epochs on the source domain, followed by fine-tuning for 300 epochs on a small, labeled subset (r E [10%, 30%)) of the target domain.
UMPL Settings. For all datasets, UMPL training is conducted with fixed dropout rate of 0.1, a = 0.2, and an initial top-k rate of 0.5. The dropout rate refers to the probability used in MC estimation of the teacher model, controlling the stochasticity during inference. The parameter governs the EMA update of the teacher model, balancing current student feedback with past teacher parameters. The initial top-k rate determines the mask ratio used for pseudo-label selection: the beginning of training, only the top-k percent of predictions with the highest confidence are selected for student learning and feedback, while the rest are masked out mitigate the impact of noisy supervision.
F
More Experiment Results
This section presents additional experimental results, including extended comparison studies and detailed ablation analyses.
F.1 Computational Cost Analysis
All experiments were conducted on a single NVIDIA RTX 4090 GPU. We evaluate the computational efficiency of the proposed method using the AeroGTO model across three representative tasks-2D steady-state, 2D transient, and 3D simulation. The computation time and memory consumption for each case summarized to provide a fair comparison of the overall computational cost.
Here, Supervised+uq refers to the student model used in UMPL.. As shown, UMPL introduces almost no additional memory overhead, thanks to the REINFORCE-style update [60] that avoids the costly
37

---

Table 6: Computation time and memory usage across datasets.
Darcy (bs=4)
NSM2d (bs=4)
Ahmed (bs=1)
Method
Memory (M)
Training Time
Memory (M)
Trnining Time (h)
Memury (M)
Training Time (h)
Supervised
2297
0.45
9727
3.75
9375
3.53
Pscudo Label
3843
1.08
16567
9.04
15947
8.73
Mean Teacher
3843
1.21
16567
10.17
15947
9.57
Noisy TS
2304
1.77
9813
13.21
9477
12.48
Supervise++UQ
3243
0.96
13578
7.69
12986
7.36
UMPL
3257
2.21
13792
16.48
13290
15.56
Table 7: Comparison of the NSM2d and Time-dependent Lid2d dataset across different methods and models. Cyan Yellow and Green indicate the best, second-best, and worst L2 loss values, respectively.
BENCHMARKS
Model
Method
dependent LidZd
dependeni Lit 5000)
10%
20%
Sapervision Only
3.07e-1
2.54e-1
2.24e-1
1.81e-1
1.46.11
1.27e.1
4,88c-1
3.34c1
3.0fe-1
2.82c11
2.43e-1
2.09e-1
2.19e-1
1.40-11
1.276-1
5.02c-
3.15e-1
2.63e-1
DeepONet
2.776-1
2.39c-1
2.02e-1
2.03e-1
1.33e-1
1.12c-1
Noisy
2.57e-1
2.24e-1
1.73e-1
1.46e-1
1.22e-1
1.02e-1
UMPL(curs)
2.23c-1
1.9161
1.52o-1
1.29e-1
1.06e-1
8,93e-2
2.51c-1
2.12c-1
1.87e-
PROMOTION
13.22%
14.73%
12.13%
11.64%
13.11
10.49%
10.03%
12.75%
8.33%
Supervision Only
1.76e
1.37e.1
9.78e-2
3.21e
8.17e-2
Pazudo
2.94e11
1.54e-1
2.72e-1
1,26e-1
7.76e-2
6.47e-2
MGN
Mean Teacher Noisy TS
4.41e-1
2.79e-1
1.33e-1
2.71e-1
1.15e-1
5.58e-2
3.10e-1
1.6e-1
6.09--
3.85e-1
1.99-1
1.07e-1
2.55e-1
9.60e-2
4.45e-2
3.04e-1
1.33e-
4,74e-2
UMPL()urs)
3.51e-1
1.61e-1
8.03e-2
2.36e-1
8.02c-2
2.82c-2
2.93c-1
1.12c-1
4.48c-2
PROMOTION
8.83%
19.09%
24.95%
7.59%
16.30%
36.60%
11.37%
16.38
5.48%
Supervision Only
2.71e-1
1,29c-1
5.41o-2
2.15e-1
7.15e-2
3.65e-2
2.760-1
1.176-1
8.89e-2
Pacudo Lahel
2.35e-1
1.17e-1
5.04e-2
2.12a-1
6.86e
3.23e-2
2.70e-1
1.12e-1
8.53e-2
Transolver
Mean Teacher
2.21e-1
1.15e-1
4.99-2
2.10e-1
6.58e-2
3.11e-2
2.57c-1
1.10c-1
8.18e-2
Noisy TS
1.79e-1
8.85e-2
4.00e-2
2.09e-1
6.38e-2
2.96e-2
2.54e-1
1.09-1
8.04e-2
UMPL()urs)
1.51e-1
7,50e-2
3.83e-2
2.04e-1
5.89e-2
2.53e-2
2.40e-1
1.05e-1
7,80e-2
PROMOTION
15.64%
15.25%
4.25%
2.39%
7.68%
14.52%
5.51%
3.66%
2.98%
Supervision Only
1.72c-1
9.35o-2
5.130-2
5.02o-2
2.23e-2
1.21e-2
9.46o-2
4.84o-2
2.66e-2
Pacudo Label
1.5le-1
9.04e-2
4.54e-2
5.37e-2
1.91e-2
1.03e-2
7.66e-2
4.20e-2
2.26e-2
AeroGTO
Mean Teacher
1.45e-1
8.71e-2
4.29e-2
4.59e-2
1.4e-22
9.30e-3
7.05e-2
3.55e-2
1.90e-2
Noisy TS
1.39c-1
8.23e-2
3.96e-2
4.24e-2
1.20e-2
6.89e-3
6.34e-2
3.06e-2
1.75e-2
1.27e-1
7.54e-2
3.69e-2
3.51e-2
1.22o-2
5.18c.3
5.74e-2
2.57e-2
1.61e-2
PROMOTION
8.63%
8.27%
6.81%
17.20%
7.04%
24.71%
9.39%
15.69%
7.83%
backpropagation chain from the student the teacher. This significantly reduces computational and memory complexity, at the cost of increased training time.
F.2
Extended Comparison Results
We conduct further comparisons actoss different models and various proportions of labeled data. Table 7 reports the performance of different methods and models on the NSM2d and Time-dependent Lid2d datasets under varying amounts of labeled data. Rable&]summarian dataset, comparing methods both in temporal interpolation (In-T) and extrapolation (Out-T) scenarios. Table presents similar comparison of the Plasma ICP dataset. Across all these tasks, UMPL consistently outperforms baseline methods, demonstrating its effectiveness under limited supervision.
results on the Black Sea
Figure 10 a) shows raincloud plots of prediction versus ground truth on the Ahmed dataset, revealing that UMPL. yields significantly better generalization on cilioo-ddissibuutoonOOOOO test samples than bascline approaches. Figure (0rb) illustrates the training curves the Stationary Lid dataset. While
38

---

Table 8: Comparison of the Black Sea dataset across different methods and models. Cyan Yellow
and Green indicate the best, second-best, and worst L2 loss values, respectively.
Black Sea
Model
Method
10%
20%
30%
In-T
Out-T
In-T
Out-T
In-T
Out-T
Supervision Only
1.45c-1
2.41o-1
1.10-1
1.23e-1
1.06-1
9.26c-2
Pieudo Lahel
1.51e-1
2.35e- 1
1.02-1
1.14-11
1.02e-1
9.07e-2
DeepONet
Mean Teacher
1.45e-1
2.23c-1
9.61e-2
1.07e11
9,98c-2
8.89c-2
Noisy IS
1.24e-1
1.67e-1
8.61e-2
9.63e-2
9.81e-2
8.06e-2
UMPI (ouns)
9.61e-2
1.43e-1
7,97e-2
8:70--2
9.09e-2
7.09e-2
PROMOTION
22.50%
14.37%
7.43%
9.65%
7.33%
12.03%
Supervision Only
9,84c-2
1.13c-1
8.720-2
9.83e-2
7.31e-2
6.92c-2
Pseudo Label
9.97e-2
1.08e
8.19e-2
9.11e.2
6.57e-2
6.876e
Moan Texcher
9.83e-2
1.08e-1
7.91e-2
9.05e-2
6.4le22
6.73e-2
MGN
Noisy
8.51e-2
9.67e-2
5.77e-2
6.24e-2
UMPI (curs)
8.39e-2
9.59e-2
6.52e-2
7.61e-2
5.83e-2
PROMOTION
1.41%
0.83%
8.20%
8.31
6.52%
Supervision Only
9.34c-2
1.11c-1
7.56o-2
7.99o-2
5.49o-2
7,07c-2
Picudo
7.29e.2
6.75e.2
9.68e-2
7.19e:2
6.76e
Transolver
Noisy TS
8.81e-2
9.17e-2
6.78e-2
6.94e-2
5.01e-2
6,12e-2
UMPLicurs)
8.74e-2
9.13e-2
6.26e-2
6.82e-2
4.47e-2
5.65e-2
PROMOTION
1.35%
0.43%
7.66%
1.73%
10.77%
7.68%
Supervision Only
8.95e-2
1.10e-
7.08c-2
7.46e-2
5.1
6.85e-2
Picudo Label
8.24e-2
1.03e-1
6.98e-2
7.43e:2
4.81e:2
6.19e.2
Mean Teacher
9.07e-1
1.02e-1
6.79e-2
7.17e:2
4.92e:2
6.15e
ActoGTO
Noisy TS
7.42o-2
9,38c-2
6.09c-2
7.04o-2
4.53e-2
5.67c-2
UMPL
6.66e-2
8.72e-2
5.48o-2
6.28e-2
4.13e-2
5.02e-2
PROMOTION
10.29%
7.05%
9.92%
10.78%
8.76%
11.58%
Table 9: Comparison of the Plasma ICP dataset across different methods and models.
Cyan
Yellow and Green indicate the best, second-best, and worst Lg loss values, respectively.
Method
Model
Percentage
w/o Fine caning
Supervision Only
Pseudo Label
Moan Teacher
Noisy TS
UMPL
PROMOTION
190e-1
1885-1
1.72e-1
1.56e-1
Transoiver
30%
2.49c-1
1.54e-1
1.44-1
1.43e11
1.21e-1
13.22%
AeroGTO 10%
1,77e-1
9.306-2
7.20e-2
6,97e-2
6.24e-2
5.25e-2
15.19%
1.776-1
7.640-2
6.46e-2
5.91c-2
4.74e-2
4.13e-2
12.47%
the PL. method exhibits clear overfitting the early training stages, UMPL, avoids this issue, indicating superior generalization in the presence of distribution shifts.
Figure 13 and Figure 14 visualize performance comparisons and corresponding uncertainty estimations between UMPL and baseline methods on the Darcy Flow, NSM2d, Black Sea, and Plasma ICP datasets, respectively. These results demonstrate that UMPL only achieves lower generalization errors but also provides reliable uncertainty estimates.
Furthermore, Figure Figure 12
Notably, also compare UMPL with heteroscedastic regression models trained solely labeled data, using AeroGTO as the base surrogate model. As shown in Figure (Blcckk Sea datasct) and Figure 16 (NSM2d dataset), UMPL delivers more accurate and calibrated uncertainty estimates unseen test data. In contrast, supervised-only models tend to produce overconfident but erroneous predictions.
E33 Quantitative Validation of Uncertainty-Eree Correlation
To quantitatively validate that the predicted uncertainty reliably correlates with the actual error magnitude, conduct comparative experiments on the AeroGTO framework using the Darcy
39

---

1.95
1.46
0.49
0.1
AcaGTo
0.00
Epach
Cu
(b) Stationary
Figure 10: (a) Raincloud plot for Ca prediction in Ahmed dataset. (b) Training and testing loss curve on the Stationary Lid dataset.
Nobe Uottt Erer
Prodicde
Nea Ener
EMPLEE Ener
Pretides
(a) NSM22 trained with AcroGTO
NSM2 trained with Transolver
Figure 11: Comparison of our method against baseline methods on the Darcy Flow dataset and uncertainty estimation our method.
Timestep
Natey
UMPL Error
UMPL Encortiinty Predicion
(n) NSM2 trained with MGN
Tiinste
GT
Noiy
UMPI
UMPL Uncertintt Profectio
(b) NSM2A trained with Transolver
Figure Comparison of our method against baseline methods on the NSM2d dataset and uncertainty estimation of our method.
40

---

Tamby
Pl.neer
Hua Teader
DNrL
Bleck See 1n-T Tiratned AmroTTO
Nack Ohe-T erained with ArROCTT
Figure 13: Comparison of our method against baseline methods on the Black Sea dataset.
Heerry
-20
444
+00
Tiandor
146
+50
Mox Taaker
Forr
Ponee JCP trained vat Unmesrer
Pleaee ICPiralned with AcroCTO
Figure 14: Comparison of our method against baseline methods on the Plasma ICP dataset.
Ear
twmmker
bm
temio
tal Riack See tranne with UMPL
D Rivck See trained with Supervision Only
Figure 15: Prediction errors and uncertainty estimates of our method compared to the Supervision Only method on the Black Sea test dataset.
41

---

Thnestrg
440
Eroor
Uncertaintr
GT
Freor
(a) NSM2d trained with UMPL.
Tinatep
GT
Uncertainty
Error
Uneeraiity
NSM2d trained with Supervision
Figure 16: Prediction errors and uncertainty estimates of our method compared to the Supervision Only method the NSM2d test dataset.
Flow and NSM2d datasets. The evaluation considers both accuracy and computational efficiency, employing miscalibration area (MA) and root mean squared calibration error (RMSCE) calibration metrics, and L2 error as the prediction accuracy metric.
Table T0 summarizes the comparison among different uncertainty estimation approaches, including Monte Carlo Dropout, SVGD [40], feature-WGG [65], and ensemble- variants. Our proposed UMPL achieves the lowest MA and RMSCE with competitive training efficiency, demonstrating its ability to generate well-calibrated and reliable uncertainty estimates.
Table 10: Comparison of uncertainty estimation methods on the Darcy Flow dataset using AeroGTO as the base forecaster.
Method
MA
RMSCE
L2
Training Time (h)
Memory (M)
EMC Dropout (Teacher)
0.2123
0.2492
0.0388
0.52
2302
MC Dropout
0.1781
0.2333
0.0376
3.31
16249
SVGD 40
0.1736
0.1954
0.0390
9,98
2253
feature- WGD 65
0.1654
0.1880
0.0322
3.50
2567
Latent-UQ (Student)
0.1561
0.1778
0.0382
0.96
3243
Ensemble [30]
0.1847
0.2044
0.0319
9.68
2395
Ensemble Latent-UQ
66
0.1297
0.1455
0.0294
13.39
3887
UMPL
0.0891
0.1011
0.0256
2.21
3257
As shown in Table II UMPL consistently achieves superior calibration and predictive accuracy com- pared with MC Dropout and Latent-UQ, indicating its robustness across different flow configurations.
These results highlight UMPL's strong capability to maintain favorable balance between accuracy and efficiency while achieving reliable uncertainty calibration. Moreover, both the teacher and
42

---

Table 11: Comparison uncertainty estimation methods on the NSM2d dataset.
Method
MA
RMSCE
L2
Latent-UQ (Student)
0.1959
0.2404
0.1728
MC Dropout
0.2841
0.3183
0.1734
UMPL
0.1127
0.1393
0.1213
student components in UMPL can be replaced by more advanced architectures to further enhance performance, albeit at the cost of higher computational complexity.
In addition, we replaced UMPL's uncertainty components with more advanced estimators, as shown in Table [2] While these alternatives achieved better calibration, they brought only marginal improve- ments in predictive accuracy and incurred significantly higher computational cost.
Table Calibration and efficiency comparison with separate Teacher (T) and Student (S) compo- nents.
(T)
(S)
RMSCE
Training Time (h)
Memoor
EMC Dropout SVGD
Latent-UQ
0.0891
0.1011
0.0256
2.21
3257
Latent-UQ UQ
0.0873
0.0983
0,0248
20.42
3677
featuro-WGD
Latent-UQ
0.0842
0.0951
0.0243
6.53
5597
Ensemble UQ
F4 Performance UMPL under Non-Gaussian Uncertainty
To investigate how UMPL handles physical systems characterized by asymmetric or heavy-tailed uncertainty, we designed one-dimensional Burgers-type PDE with a small diffusion coefficient as testbed and adopted DeepONet the backbone model. This setting introduces shocks and discon- tinuities, resulting in a highly non- Gaussian regime where models assuming Gaussian uncertainty often degrade. The governing equation is given by:
Ou
Ou
+
+ cu 0,
Øt
Ox
dr
u(x
0) Ou
ax (a+ 1) x (-1,1]
(62)
0.
dr
Table 13 reports the quantitative comparison. The baseline UMPL,, which assumes Gaussian uncer- tainty, exhibits degraded calibration performance under this non-Gaussian condition. To mitigate this issue, replaced the teacher's epistemic uncertainty model with feature-WGD (65), which better captures heavy-tailes priors, and substituted the student's 's aleatoric uncertainty model with Mixture Density Network (MDN) [55]. capable of modeling multimodal and skewed distributions. This enhanced version substantially improves both calibration and predictive accuracy at the cost of increased computation.
Table 13: Results on a one-dimensional Burgers equation under non-Gaussian uncertainty using DeepONet the backbone. "/" indicates uncertainty estimate.
Method
MA
RMSCE
L22
Training Time (h)
Memory (M)
Supervised
/
0.0499
1.42
1261
UMPL
0.3004
0.3448
0.0528
1.62
1439
Improved UMPL
(feature-Wae WGD MDN)
0.1408
0.1590
0.0432
2.61
1789
These results indicate that assuming Gaussian uncertainty limits model calibration and predictive fidelity in non-Gaussian regimes. By integrating more expressive epistemic uncertainty model
43

---

(feature- WGDD and a heavy-tailed noise model (MDN), UMPL achieves substantially better robust- ness and accuracy with moderate computational overhead. Overall, the framework demonstrates flexible, modular design that enables an effective trade-off between efficiency and predictive reliability across diverse uncertainty structures.
E,5
Extended Ablation Analysis
F.5.1 Sensitivity and Robustness Analysis
We evaluate the sensitivity and robustness of UMPL the out-of-distribution Stationary Lid-Drivee Cavity dataset using AeroGTO as the backhone. We first probe robustness to aleatoric perturbations by injecting Gaussian noise of varying magnitude into the student's predicted uncertainty during training. As summarized in Table [4 uncertainty calibration (MA, RMSCE) degrades steadily with increasing noise, whereas predictive accuracy (L2) remains stable under small perturbations but drops sharply with large noise. Overall, UMPL is robust to moderate noise yet sensitive to severe distortion.
Table 14: Sensitivity to injected Gaussian noise in the student's predicted uncertainty on the OOD Stationary Lid dataset.
Noise Magnitude (a)
MA
RMSCE
L2
0.05
0.0703
0.0828
0.0634
0.10
0,0717
0.0879
0.0634
0.20
0,0779
0.0942
0.0636
0.50
0.0908
0,1053
0.0663
1.00
0.2156
0.2655
0.1499
We then study two design hyperparameters under the same setting: the PUF threshold (which filters pseudo-labels) and the feedback strength (which weights uncertainty guided correction). Results in Tables 16]]66oww that lower PUF threshold can better filter noisy labels on harder datasets, though its effect is mild here. For feedback strength, too small values suppress pseudo-label correction (reducing UMPL. toward plain pseudo-labeling), while overly large values inject noise and degrade label quality and student learning.
Table 15: Sensitivity the PUF threshold the OOD Stationary Lid dataset.
PUF Threshold
MA
RMSCE
L
100%
0.0679
0,0802
0.0633
95%
0.0661
0.0783
0.0632
90%
0.0707
0.0809
0.0638
Table 16: Sensitivity to feedback strength the OOD Stationary Lid dataset.
Uncertainty Strength
MA
RMSCE
L
x0.1
0.0711
0.0838
0.0752
x1
0.0679
0.0802
0.0633
x5
0.0825
0.0974
0.0882
x x10
0.1273
0.1503
0.1367
Beyond these factors, we further investigate three UMPL hyperparameters on the same dataset: (i) the EMC update coefficient a in the teacher, (ii) the dropout rate used for Monte Carlo estimation, and (iii) the initial top-k selection ratio for pseudo-label masking. In each experiment, one parameter is varied while others are held at their respective optima. The results in Figure 17 indicate non-monotonic trends for each parameter, highlighting UMPL's sensitivity to these choices and the importance of careful tuning. Exploring adaptive data-driven strategies for hyperparameter selection left for future work.
44

---

1--2
1e-2
Drapout
1r-2
7.2
7.5
5.8
7.0
6.4-
6.5
0.0
0.2
0.4
0.6
0,8
1.0
0.00
0.01
0.02
0,03
DU04
0.05
0.11
0.2
0.4
Dropout
Instaal Top-k%
Figure 17: Ablation results for key UMPL hyperparameters the Stationary Lid dataset: EMC update coefficient a, MC-dropout rate p. and initial top-k ratio for pseudo-label masking. Each curve varies one parameter while fixing others at their optimal values.
F.5.2 Noise Robustness Analysis
While our main benchmarks do explicitly include noisy observations and the current teacher primarily captures epistemic uncertainty, we further study noise robustness on Darcy Flow (AcroGTO backbone) by injecting input-dependent noise varying levels. Results in Table [7]shw performance degradation as noise increases. To improve robustness under noisy observations, we augment the teacher to incorporate both epistemic and aleatoric uncertainty, following [25], and introduce an ensemble student, which enhances predictive performance in the noisy regime.
Table 17: Darcy Flow with input-dependent noise: calibration (MA, RMSCE) and accuracy (L2).
Noise Level
MA
RMSCE
L2
5%
0.0996
0.1117
0.0257
10%
0.1076
0.1207
0.0264
20%
0.1286
0.1443
0.0302
40%
0.1755
0.1975
0.0386
40%, Improved UMPL
0.1335
0.1498
0.0348
G
Borader Impact
The UMPL framework significantly enhances surrogate modeling performance, particularly under and distribution shift scenarios. This improved generalization is valuable for tasks such fluid dynamics modeling, climate prediction, and molecular simulations, where models often face data distributions that differ from training conditions. By enabling uncertainty-awe aware learning with limited supervision, UMPL reduces reliance large labeled datasets and improves model robustness. In future work, we plan to extend UMPI. to real-world scenarios and more complex applications, bridging the gap between research and deployment in science and engineering domains.
45


Reviews:
[{"review_id": "xoZYqZr2NO", "reviewer": "Reviewer_fwWx", "summary": "This paper tackles the challenge of building surrogate models for physics-governed systems when only a small, costly set of simulation labels is available alongside abundant unlabeled data, proposing a semi-supervised teacher–student framework in which a lightweight Exponential-Moving Monte-Carlo Dropout (EMC-Dropout) teacher predicts system states and provides epistemic uncertainty, while a student network learns from those pseudo-labels, estimates aleatoric uncertainty, and feeds it back to the teacher. Training is further stabilized by Progressive Uncertainty Filtering, which gradually discards the noisiest pseudo-labeled regions, and by Supervised Anchoring, which periodically re-grounds the model on true labels. Evaluated by plugging into four distinct neural operators—DeepONet, MeshGraphNet, Transolver, and AeroGTO—across seven CFD and physics benchmarks spanning partial differential equations, real-world datasets, and large-scale 3-D simulations, the method achieves roughly a 14 % error reduction with only 10 % labeled data and markedly improves out-of-distribution and distribution-shift generalization.", "questions": "1. Could you elaborate on the computational resources your method requires—e.g., hardware, run-time, and memory overhead—so readers can better gauge its practical cost?\n\n2. Within the teacher–student framework, is there a risk of model collapse, where the teacher and student converge to similar outputs on unseen data yet still generate poor labels relative to the true target distribution? - How to protect against this?", "limitations": "Yes — the limitations are discussed in Appendix Section K.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. **Novel teacher–student strategy** – The framework introduces an innovative teacher–student loop that produces more reliable pseudo-labels while simultaneously training the surrogate model.\n\n2. **Architecture-agnostic design** – Because the method is model-agnostic, it can be plugged into virtually any neural-operator or surrogate-model architecture with minimal modification.\n\n3. **Thorough empirical validation** – The authors support their claims with extensive experiments across multiple datasets and architectures, demonstrating consistently strong performance.\n\n\n\n**Weaknesses**\n\n1. **Potential computational burden** – Running two networks, multiple Monte-Carlo dropout passes, and iterative feedback may be resource-intensive. A dedicated section detailing the hardware used, run-times, and memory footprint would clarify the true computational cost.\n\n2. **Clarity and presentation** – Several aspects of the manuscript hinder readability:\n   \n   a. Key variables and symbols should be defined in the main text rather than deferred to the appendix.\n   \n   b. Algorithm 1 should be included in the main body to better understand the whole procedure.\n   \n   c. Although Figure 1 is helpful in principle, its current form does not clearly depict the full workflow and should be redesigned for greater clarity.", "was_revised": true, "final_justification": "Authors have addressed most of my questions. I will maintain my score at 5.", "rebuttal": "Thank you for your positive feedback and for recognizing the significance and potential of our method.\n\n> Q1: Clarity and presentation – Several aspects of the manuscript hinder readability.\n\n**A1:** We thank the reviewer for the helpful suggestions. For a and b, we will include key variable definitions and move Algorithm 1 to the main text in the revised version. For c, we will revise Figure 1 in the revised version to better reflect the complete UMPL workflow. The updated figure will explicitly show the teacher–student interaction, the three-stage training loop (pseudo-label generation, student update, teacher update), and the roles of Progressive Uncertainty Filtering (PUF) and Supervised Anchoring (SA), thereby enhancing the clarity and interpretability of the framework.\n\n>Q2: Could you elaborate on the computational resources your method requires?\n\n**A2:** Thanks for the question. All experiments were conducted on a single NVIDIA RTX 4090 GPU. We report results on AeroGTO model across three representative tasks—2D steady-state, 2D transient, and 3D simulation—as summarized below.\n\nDataset|Darcy (bs=4)||NSM2d (bs=4)||Ahmed (bs=1)||\n-|-|-|-|-|-|-\nMethod|Memory(M)|Training Time(h)|Memory(M)|Training Time(h)|Memory(M)|Training Time(h)\nSupervised|2297|0.45|9727|3.75|9375|3.53\nPseudo Label|3843|1.08|16567|9.04|15947|8.73\nMean Teacher|3843|1.21|16567|10.17|15947|9.57\nNoisy TS|2304|1.77|9813|13.21|9477|12.48\nSupervised+uq|3243|0.96|13578|7.69|12986|7.36\nUMPL|3257|2.21|13792|16.48|13290|15.56\n\nHere, Supervised+uq refers to the student model used in UMPL. As shown, **UMPL introduces almost no additional memory overhead**, thanks to the REINFORCE-style update that avoids the costly backpropagation chain from the student to the teacher. This significantly reduces computational and memory complexity, **at the cost of increased training time**.\n\n>Q3: Within the teacher–student framework, is there a risk of model collapse? How to protect against this?\n\n**A3:** We thank the reviewer for the insightful question. We agree that model collapse is a potential risk in teacher–student frameworks. To mitigate this, our design incorporates several mechanisms:\n\n- **Divergent Initialization Strategy:** The teacher and student models are initialized with different parameters, and during each training loop, they are exposed to independently sampled labeled data. This helps reduce the risk of early-stage over-alignment due to shared errors.\n\n- **Uncertainty-Guided Feedback Loop:** The student provides point-wise feedback based on aleatoric uncertainty, which captures spatial error sensitivity and directs the teacher’s updates toward high-error regions. This breaks blind imitation and reduces the chance of reinforcing incorrect signals.\n\n- **Uncertainty-Aware Pseudo Labels:** Pseudo labels are perturbed using the teacher’s epistemic uncertainty, replacing hard targets with uncertainty-aware guidance. This alleviates label bias and stabilizes convergence.\n\n- **Progressive Uncertainty Filtering (PUF):** Before being used for training, the teacher’s pseudo labels are filtered by uncertainty. Low-confidence samples are discarded, improving pseudo-label quality and enhancing training stability.\n\nWe will include a detailed discussion of these safeguards in the revised version to better address this potential risk."}, {"review_id": "80HvOFQCth", "reviewer": "Reviewer_VXJf", "summary": "This paper tackles the challenge of building robust surrogate models for complex physical systems, especially under conditions of limited labeled data, distribution shifts or OOD scenarios. Traditional DNNs and neural operators require large labeled datasets, which are often costly or impractical to obtain in scientific domains but the problem arises as, even though semi-supervised learning offers a way to leverage abundant unlabeled data, it suffers from noisy pseudo-labels that can degrade model performance.\n\nTo tackle this the authors propose Uncertainty-Informed Meta Pseudo Labeling (UMPL), a semi-supervised learning framework that improves pseudo-label quality by incorporating uncertainty estimates in a setup of a teacher-student meta-learning framework where they have roles based on uncertainty. Specifically, the teacher model generates pseudo-labels along with epistemic uncertainty, which reflects the model’s confidence and helps identify uncertain predictions under distribution shifts. The student model however learns from these pseudo-labels in order to estimate the aleatoric uncertainty, capturing inherent data noise and label uncertainty. This separation of uncertainty types aligns with the distinct roles of teacher and student and leads to more robust and interpretable uncertainty estimates. The student provides feedback to the teacher based on aleatoric uncertainty, guiding the teacher to refine pseudo-labels iteratively. This meta-learning loop enhances pseudo-label quality and model generalization, especially in uncertain or underrepresented regions of the data.", "questions": "As mentioned above:\n\nAs I am unfamiliar with the setting of modeling high-dimensional PDE systems I struggle to find the motivation for the choice of relative L_2 error compared to just the standard L2 loss. Can the authors explain this motivation?\n\nHow long did it take to run the experiments and did you use GPUs or CPUs? Could you please elaborate more on why do you not report statistical significance? Can you please include significance for at least a couple of experiments in Table 1 or 2? \n\nTypo in Appendix H title: \"experiential settings\"", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The paper is well written and the motivation is clear. Although I am not that familiar with the practical problems and frameworks that solve (or at least tackle how to solve) problems like pseudo-labeling of high-dimensional PDE systems, it seems that the problem is well formulated and the authors motivate it well.\n\nRegarding the contribution of the paper, in their second bullet point the authors claim that the role-aware design leads to more robust and interpretable uncertainty estimates, however without backing it up with any claims. It would be good if the authors could point out or summarize the sections, results (figures/tables) which back this claim up. At the moment it seems a bit unjustified. The same comment stands for the third bullet point, in which the authors claim that the targeted feedback significantly improves the quality of pseudo labels across iterations without claiming why that is the case.\n\nAs I am unfamiliar with the setting of modeling high-dimensional PDE systems I struggle to find the motivation for the choice of relative L_2 error compared to just the standard L2 loss. Can the authors explain the motivation why the loss choice in (1)? Also, why have the authors chosen an encoder embedding? Is this a popular model choice in the student-teacher literature? Is there a reason why one would expect this would work better for modeling PDE systems? Both fall a bit flat unmotivated.\n\nThe same comment stands for their choice of MC Dropout. Although the simplest to implement and most straightforward to train from my experience with BNNs, the authors should argue more why they have chosen it, and not for example SVGD[1], fSVGD[2] or any other Bayesian NN choice, and I think that they should perform benchmarks/baselines to corroborate their choice. In Appendix C the authors give a nice Bayesian interpretation of MC Dropout which I am thankful for, but I was a bit disappointed that they did not deliberate anything about why specifically choosing it over other methods. \n\nRegarding Figure 2. it is not really clear what the point of the figure is, the error between the proposed method and Noisy TS is differently distributed and with different magnitudes and its not clear to me from the plot which one is better.\n\nRegarding using REINFORCE as the solution to minimize the loss in (8) and deriving its gradient, it would be nice if the authors add a few comments regarding the REINFORCE rule; are there any downsides to using it, is it the standard in other literature, what are its benefits and so on. I am not well versed in this literature and to me, although all the design choices that the authors have made towards proposing the UMPL framework, they fall a bit flat. In general, is there any reason why we would expect each of them to work better separately and together, how do they interplay and can the authors show that other choices for the building blocks of UMPL would've resulted in inferior performance? The ablation study in Table 3 was indeed nice and provided one small step towards this but giving at least one or two examples of what happens with other choices of uncertainty estimation (e.g., Bayesian NNs) would've been nice to include. \n\nOne argument towards this could be the superior results that the authors observe in Tables 1 and 2, but these also fall a bit short as I don't know if these results are statistically significant or not. In answers to question 6 of the Q&A the authors mention that they introduce experiment setting and details but I couldn't find how long did it take to run your experiments and whether the authors ran them on GPUs vs CPUs. Furthermore, it is mentioned that the statistical significance is not reported as in fluid dynamics modeling using a fixed random seed shows consistent performance. I believe this claim needs to be backed up and at least commented in the main paper or the appendix. If the results were computationally expensive and you have limited resources this is fine but then this should be commented on too. If you did mention this somewhere in the appendix I would be happy if you can point me to it, but if your experiments did not require extensive computational resources I think that statistical significance needs to be added both to Tables 1 and 2 as well as the L2 error in Table 3.\n\n[1] Liu, Q. and Wang, D., 2016. Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in neural information processing systems, 29.\n[2] Wang, Z., Ren, T., Zhu, J. and Zhang, B., 2019. Function space particle optimization for Bayesian neural networks. arXiv preprint arXiv:1902.09754.", "was_revised": true, "final_justification": "I am still hesitant and reluctant due to the lack of statistical significance of the results. The method seems to be extremely more computational heavy compared to the other methods both in terms of memory and training time. I am especially reluctant as the authors said they have not included statistical significance due to space constraints which seems odd.\n\nHowever, as I am not well versed in the field and the after reading the other reviewers' comments, I have chosen to update my score from 3 to a 4. However, I still have some concerns and worries so will not give a larger score.", "rebuttal": "Thank you for your careful reading of the paper and your constructive comments. We address each point below.\n\n>Q1:Request to clarify and support the claimed benefits of role-aware uncertainty modeling and targeted feedback.\n\n**A1:** In response to the request for evidence supporting the claims of “role-aware design” and “targeted feedback,” we provide the following clarifications:\n\nRegarding the second bullet point: “Role-aware design leads to more robust and interpretable uncertainty estimates” — our framework uses role-specific uncertainty: the teacher models epistemic uncertainty for distribution shifts, while the student captures aleatoric uncertainty for input-dependent noise, forming a complementary structure. This design is supported by:\n\n- Theoretical justification:\nSection 3.1 provides the theoretical foundation, where Theorem 3.1 introduces EMC Dropout for efficient epistemic uncertainty estimation on the teacher side, Equation (5) formulates the aleatoric uncertainty optimization for the student, and Theorem 3.2 establishes an upper bound on student error under noisy pseudo labels.\n\n- Empirical evidence:\nFigure 2 shows the student’s feedback loss aligns well with true error, demonstrating strong spatial adaptivity. Figures 3(a) demonstrates that our method, applied to the Ahmed dataset, provides more accurate uncertainty estimates compared to the approach that uses only the student model trained on labeled data.\n\nRegarding the third bullet point: “Targeted feedback significantly improves the quality of pseudo labels”\nThis is supported both theoretically and empirically:\n\n- Theoretical justification:\nSection 3.2 introduces a point-wise feedback mechanism guided by the student’s aleatoric uncertainty. The pseudo labels are perturbed (Eq. 6), and the teacher is updated using a first-order gradient approximation (Eq. 10–12), which avoids expensive meta-gradient computation.\n\n- Empirical evidence:\nFigure 2 shows that feedback signals correlate strongly with true label error, guiding the teacher to focus on high-error regions and improving pseudo-label quality over time. Tables 1 and 2 demonstrate that UMPL consistently outperforms methods without targeted feedback (e.g., Pseudo Label, Mean Teacher), and Table 3 shows a 42.6% performance drop when the feedback module is removed, confirming its critical role.\n\nWe acknowledge that the current version lacks direct quantitative analysis of pseudo-label quality progression. In the revised version, we will include comparisons of pseudo-label error curves over iterations to more explicitly demonstrate how targeted feedback improves pseudo-label quality.\n\n>Q2:Why to choose relative L2 error?\n\n**A2:** Thank you for the question. We adopt relative L2 error due to its clear advantages in scientific modeling, particularly in the following aspects:\n\n- It is **well-suited for multi-scale systems or quantities with heterogeneous units**, where standard L2 loss tends to overweight large-magnitude regions and ignore small but important ones.\n\n- In **multi-physics problems** (e.g., Plasma ICP), where predicted variables differ by orders of magnitude, standard L2 loss leads to gradient imbalance. **Relative L2 provides scale-invariant optimization that treats all fields fairly.**\n\n- This choice is **widely adopted in neural operator literature** (e.g., FNO, Transolver, AeroGTO), both as a training loss and evaluation metric, due to its physical interpretability and robustness.\n\nTherefore, relative L2 error not only serves as an effective evaluation metric but also as a more appropriate training objective.\n\n>Q3:Why to choose an encoder embedding? Is this a popular choice in the student-teacher literature? Is it work better for modeling PDE systems?\n\n**A3:** Thank you for the question. The use of encoder embeddings is motivated by the following:\n\n- **Unified representation of heterogeneous PDE inputs:** PDE problems involve geometry, physical parameters, and boundary conditions. The encoder maps these multimodal inputs into a common latent space, enhancing generalization. For instance, in the Plasma ICP case, the encoder helps integrate irregular mesh geometry and spatial parameters into a consistent representation.\n\n- **Widely used across domains:** Teacher–student frameworks commonly use encoder embeddings in tasks such as image classification[1] and defect detection[2], enabling stable pseudo-labeling and alignment between models.\n\n- **Adaptation to PDE-specific challenges:** For datasets with non-uniform meshes or complex geometries (e.g., Darcy Flow, NSM2d), the encoder learns geometry-invariant representations to improve spatial robustness. This strategy is also used in operator models like DeepONet and AeroGTO to align heterogeneous inputs into a unified latent space.\n\nWe will clarify this design choice in the revised version.\n\n>Q4: Figure 2 isn't clear which method is better.\n\n**A4:** Thank you for the valuable feedback. In the revised version, we will replace the last subplot in Figure 2 with a comparison of Kendall correlation between feedback loss and true pseudo-label error for UMPL and Noisy TS. **On 100 unlabeled samples, UMPL achieves much stronger correlation**: 59 samples exceed 0.8 (vs. 5 for Noisy), and only 2 fall below 0.6 (vs. 11 for Noisy). This highlights that UMPL’s aleatoric uncertainty-guided feedback more reliably captures pseudo-label error, leading to more stable and robust training.\n\n>Q5: Comments regarding the REINFORCE rule.\n\n**A5:** Thank you for the question. We adopt REINFORCE to estimate the gradient in Eq.(8), primarily to avoid the costly backpropagation path from student to teacher, thus **significantly reducing memory and computational overhead.** REINFORCE is a standard technique and has been widely used in meta pseudo-labeling methods in computer vision. Its main limitation is **increased time cost due to repeated forward and backward passes.** We will clarify this design choice and trade-off in the revised version.\n\n>Q6: Why not choose other uq methods?\n\n**A6:** Thank you for the insightful question. As noted, we chose EMC mainly for its computational efficiency. While more accurate uq methods exist, EMC offers a good balance between cost and performance. **Our main goal is to use unlabeled data to boost prediction accuracy on unseen data.** We have added experiments comparing different uq methods on AeroGTO for Darcy Flow dataset in terms of both accuracy and efficiency, using miscalibration area (MA) and root mean squared calibration error (RMSCE) as evaluation metrics:\n\nMethod|MA|RMSCE|L2|Training Time(h)|Memory(M)\n-|-|-|-|-|-\nEMC Dropout(Teacher)|0.2123|0.2492|0.0388|0.52|2302\nMC Dropout|0.1781|0.2333|0.0376|3.31|16249\nSVGD|0.1736|0.1954|0.0390|9.98|2253\nfeature-WGD [3]|0.1654|0.1880|0.0322|3.50|2567\nLatent-uq(Student)|0.1561|0.1778|0.0382|0.96|3243\nEnsemble [4]|0.1847|0.2044|0.0319|9.68|2395\nEnsemble latent-uq [5]|0.1297|0.1455|0.0294|13.39|3887|\nUMPL|0.0891|0.1011|0.0256|2.21|3257\n\nAs shown in the results, the teacher performs worse than advanced Bayesian methods (e.g., SVGD, feature-WGD), and the student underperforms compared to Ensemble latent-uq. However, both are more efficient. As discussed in Q7&A7, **Advanced replacements improve uncertainty estimation with minimal accuracy gains, at a much higher computational cost. Our framework remains flexible, enabling such substitutions depending on practical trade-offs.**\n\n>Q7: Other choices for the building blocks of UMPL?\n\n**A7:** We replaced the student and teacher components using the same setup as in Q6&A6. The performance comparison is shown below:\n\nMethod|MA|RMSCE|L2|Training Time(h)|Memory(M)\n-|-|-|-|-|-\nT(EMC Dropout)+S(latent-uq)|0.0891|0.1011|0.0256|2.21|3257\nT(SVGD)+S(Latent-uq)|0.0873|0.0983|0.0248|20.42|3677\nT(feature-WGD)+S(Latent-uq)|0.0842|0.0951|0.0243|6.53|5597\nT(feature-WGD)+S(Ensemble latent-uq)|0.0773|0.0877|0.0239|28.43|6273\n\nThese results confirm that replacing components leads to better performance, at the cost of increased computational overhead.\n\n>Q8: Computational resources & statistical significance?\n\n**A8:** Thanks for the question. All experiments were conducted on a single NVIDIA RTX 4090 GPU. We report results on AeroGTO model across three representative tasks—2D steady-state, 2D transient, and 3D simulation—as summarized below.\n\nDataset|Darcy (bs=4)||NSM2d (bs=4)||Ahmed (bs=1)||\n-|-|-|-|-|-|-\nMethod|Memory(M)|Training Time(h)|Memory(M)|Training Time(h)|Memory(M)|Training Time(h)\nSupervised|2297|0.45|9727|3.75|9375|3.53\nPseudo Label|3843|1.08|16567|9.04|15947|8.73\nMean Teacher|3843|1.21|16567|10.17|15947|9.57\nNoisy TS|2304|1.77|9813|13.21|9477|12.48\nSupervised+uq|3243|0.96|13578|7.69|12986|7.36\nUMPL|3257|2.21|13792|16.48|13290|15.56\n\nDue to main text space constraints, we reported only the mean over 5 trials with different seeds. Statistical significance results will be included in the appendix of the revised version.\n\n>Q9: Typo in Appendix H title.\n\n**A9:** Fixed it.\n\nReference\n\n[1] Wang, Zhenbin, et al. \"Metateacher: Coordinating multi-model domain adaptation for medical image classification.\" Advances in Neural Information Processing Systems 35 (2022): 20823-20837.\n\n[2] Zhao, Sinong, et al. \"Meta pseudo labels for anomaly detection via partially observed anomalies.\" Engineering Applications of Artificial Intelligence 126 (2023): 106955.\n\n[3] Yashima, Shingo, et al. \"Feature space particle inference for neural network ensembles.\" International Conference on Machine Learning. PMLR, 2022.\n\n[4] Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. \"Simple and scalable predictive uncertainty estimation using deep ensembles.\" Advances in neural information processing systems 30 (2017).\n\n[5] Wu, Tailin, et al. \"Uncertainty quantification for forward and inverse problems of pdes via latent global evolution.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 1. 2024."}, {"review_id": "xVFey6AKwo", "reviewer": "Reviewer_9hkT", "summary": "The paper proposes a semi-supervised teacher-student framework for emulators with limited simulation data. The teacher generates pseudo-labels using MC-Dropout to predict epistemic uncertainty, while the student learns from these labels while predicting aleatoric uncertainty via variance prediction. The student's uncertainty provides feedback to refine the teacher's pseudo-labels in a meta-learning loop and adds spatial coherence. Evaluation of the system is presented from scientific tasks and distribution shifts.", "questions": "1. Both epistemic (MC-Dropout) and aleatoric (heteroscedastic variance) uncertainty predictors assume symmetric Gaussian distributions. How does UMPL handle physical systems with asymmetric or heavy-tailed uncertainties (e.g., shock waves, bifurcation points) where these assumptions fail? Did you evaluate on such systems?\n2. Why are standard uncertainty calibration metrics absent? Can you show quantitative proof that the predicted uncertainty reliably correlates with error magnitude (not just direction) in OOD regions?\"\n3. Could you report computing resources?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n* Original feedback closed loop between teacher-epistemic and student-aleatoric \n* The mesh-aware spatial coherence for the pseudo-labels\n* Extensive experimental section on various architectures\n\n**Weaknesses**\n* Questionable uncertainty method: the paper relies on MC-Dropout to estimate the teacher-epistemic, known to struggle when the true uncertainty isn't nicely behaved and bell-shaped (Gaussian) – common in messy real-world physics like shocks or abrupt changes. \n* Reliable uncertainties not shown: there should be some experimental proof that the uncertainty prediction are trustworthy using checks like calibration metrics or reliability diagrams. Aren' t the errors remain high in complex tasks like NSM2D?\n* Potentially overly easy experiments: benchmarks used clean, smooth simulation data without the real-world noise or sharp changes one could encounter. This likely explains why even the basic \"Pseudo Label\" baseline beat supervised learning – a result that often doesn't hold up with noisier or more complex data.  I am not convinced that the method would hold against the hard distribution shifts or more complex noise common in real data", "was_revised": true, "final_justification": "The authors have addressed most of my questions. I am increasing my score at 5.", "rebuttal": "Thank you for the positive feedback.\n\n>Q1: Quantitative proof that the predicted uncertainty reliably correlates with error magnitude.\n\n**A1:** Thank you for the insightful question. We have added experiments comparing our method with other uncertainty estimation approaches on AeroGTO for the Darcy Flow dataset in terms of both accuracy and efficiency, using miscalibration area (MA) and root mean squared calibration error (RMSCE) as evaluation metrics:\n\nMethod|MA|RMSCE|L2|Training Time(h)|Memory(M)\n-|-|-|-|-|-\nEMC Dropout(Teacher)|0.2123|0.2492|0.0388|0.52|2302\nMC Dropout|0.1781|0.2333|0.0376|3.31|16249\nSVGD [1]|0.1736|0.1954|0.0390|9.98|2253\nfeature-WGD [2]|0.1654|0.1880|0.0322|3.50|2567\nLatent-uq(Student)|0.1561|0.1778|0.0382|0.96|3243\nEnsemble [3]|0.1847|0.2044|0.0319|9.68|2395\nEnsemble latent-uq [4]|0.1297|0.1455|0.0294|13.39|3887|\nUMPL|0.0891|0.1011|0.0256|2.21|3257\n\nAdditionally, we conducted a comparison on the NSM2d experiment:\n\nMethod|MA|RMSCE|L2\n-|-|-|-\nLatent-uq(Student)|0.1959|0.2404|0.1728\nMC Dropout|0.2841|0.3183|0.1734\nUMPL|0.1127|0.1393|0.1213\n\nThese comparisons highlight **UMPL's strong performance over baselines with a good balance between accuracy and efficiency.** The teacher and student components can be flexibly replaced with more advanced methods for further gains, albeit at higher computational cost.\n\n>Q2: The method may not hold up with the noise or real-world more complex data.\n\n**A2:** Thank you for the thoughtful comment. We have considered challenging scenarios in our experiments. For example, the Black Sea dataset is based on real ocean measurements with strong seasonal patterns and complex temporal evolution. It poses a difficult temporal OOD generalization problem, yet UMPL still achieves performance gains (albeit smaller than on other datasets). The Plasma ICP dataset involves distribution shift from low- to high-fidelity simulations. UMPL outperforms baselines using only 10% of high-fidelity data, showing strong data efficiency under realistic conditions.\n\nWe acknowledge that our benchmarks do not include noisy data. The current teacher model focuses on epistemic uncertainty and does not explicitly model data noise. To address this, we conducted an additional experiment on the Darcy Flow dataset using the AeroGTO backbone, introducing varying levels of input-dependent noise.\n\nNoise|MA|RMSCE|L2\n-|-|-|-\n5%|0.0996|0.1117|0.0257\n10%|0.1076|0.1207|0.0264\n20%|0.1286|0.1443|0.0302\n40%|0.1755|0.1975|0.0386\n40%,Improved UMPL|0.1335|0.1498|0.0348\n\nResults show that model performance degrades with added noise. **To improve robustness with noise, we incorporated both epistemic and aleatoric uncertainty in the teacher** following [5], and added an ensemble student model. This enhanced predictive performance under noisy conditions, which we will detail further in the revised version.\n\n>Q3: How does UMPL handle physical systems with asymmetric or heavy-tailed uncertainties?\n\n**A3:** Thank you for the insightful question. Our teacher and student models indeed assume Gaussian scenarios, which is suitable for most standard cases. However, we acknowledge that performance may degrade under shocks or discontinuities. To evaluate this, we designed an experiment using a 1D Burgers-type PDE with small diffusion coefficient:\n\n$\\frac{\\partial u}{\\partial t}-\\mu \\frac{\\partial ^2 u}{\\partial x^2}+u \\frac{\\partial u}{\\partial x}+\\epsilon u^3=0,u_0(x)=ax^3-(a+1)x,x\\in[-1,1],-\\mathbf{n}\\cdot\\frac{\\partial u}{\\partial x}|_{x=-1,1}=0$\n\nUsing DeepONet as the backbone, results show that the original model struggles under such non-Gaussian settings:\n\nMethod|MA|RMSCE|L2|Training Time(h)|Memory(M)\n-|-|-|-|-|-\nSupervised|/|/|0.0499|1.42|1261\nUMPL|0.3004|0.3448|0.0528|1.62|1439\nImproved UMPL|0.1408|0.1590|0.0432|2.61|1789\n\n We improved performance by replacing the teacher's epistemic model with feature-WGD and the student's aleatoric model with MDN [6], which better capture heavy-tailed distributions. This comes at higher computational cost. **Our framework thus offers a flexible trade-off between efficiency and accuracy, with modular components adaptable to specific scenarios.** We will include these findings in the revised appendix.\n\n>Q4: Could you report computing resources?\n\n**A4:** Thanks for the question. All experiments were conducted on a single NVIDIA RTX 4090 GPU. We report results on AeroGTO model across three representative tasks—2D steady-state, 2D transient, and 3D simulation—as summarized below.\n\nDataset|Darcy (bs=4)||NSM2d (bs=4)||Ahmed (bs=1)||\n-|-|-|-|-|-|-\nMethod|Memory(M)|Training Time(h)|Memory(M)|Training Time(h)|Memory(M)|Training Time(h)\nSupervised|2297|0.45|9727|3.75|9375|3.53\nPseudo Label|3843|1.08|16567|9.04|15947|8.73\nMean Teacher|3843|1.21|16567|10.17|15947|9.57\nNoisy TS|2304|1.77|9813|13.21|9477|12.48\nSupervised+uq|3243|0.96|13578|7.69|12986|7.36\nUMPL|3257|2.21|13792|16.48|13290|15.56\n\nHere, Supervised+uq refers to the student model used in UMPL. As shown, **UMPL introduces almost no additional memory overhead**, thanks to the REINFORCE-style update that avoids the costly backpropagation chain from the student to the teacher. This significantly reduces computational and memory complexity, **at the cost of increased training time**.\n\nReference\n\n[1] Liu, Q. and Wang, D., 2016. Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in neural information processing systems, 29. \n\n[2] Yashima, Shingo, et al. \"Feature space particle inference for neural network ensembles.\" International Conference on Machine Learning. PMLR, 2022.\n\n[3] Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. \"Simple and scalable predictive uncertainty estimation using deep ensembles.\" Advances in neural information processing systems 30 (2017).\n\n[4] Wu, Tailin, et al. \"Uncertainty quantification for forward and inverse problems of pdes via latent global evolution.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 1. 2024.\n\n[5] Kendall, Alex, and Yarin Gal. \"What uncertainties do we need in bayesian deep learning for computer vision?.\" Advances in neural information processing systems 30 (2017).\n\n[6] Thakur, Akshay, and Souvik Chakraborty. \"MD-NOMAD: Mixture density nonlinear manifold decoder for emulating stochastic differential equations and uncertainty propagation.\" arXiv preprint arXiv:2404.15731 (2024)."}, {"review_id": "NEKFloA6pq", "reviewer": "Reviewer_FbuM", "summary": "This paper proposes Uncertainty-Informed Meta Pseudo Labeling, a semi-supervised learning framework tailored for surrogate modeling of physical systems governed by PDEs under limited labeled data. The framework builds upon a teacher-student architecture, where the teacher estimates epistemic uncertainty via EMC Dropout to generate pseudo-labels, and the student learns from these pseudo-labels while modeling aleatoric uncertainty via heteroscedastic regression. A key contribution is the use of uncertainty-informed feedback: the student’s uncertainty guides refinement of the teacher’s pseudo-labels, enabling a closed-loop meta-learning process. The method improves generalization under distribution shifts across multiple tasks.", "questions": "listed in the weakness", "limitations": "Yes. The paper includes a discussion on limitations in the appendix and acknowledges challenges such as computational cost and the assumption of reliable uncertainty estimation.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Integration of dual uncertainty modeling in a meta-learning loop.\n2. Efficient and theoretically grounded uncertainty estimation.\n3. Strong empirical results on PDEs and real-world datasets.\n\n\nWeaknesses:\n1. The key issue here is the contribution. Although this work proposed multiple ways to improve uncertainty estimation in terms of reliability and efficiency. However, the core part of this work is the SSL, which just use the uncertainty as a tool. I think the best way to justify the contribution should focus on SSL instead of the accuract uncertainty. \n2. It is good to achieve a better uncertainty estimation, but why don't just use most up to date uncertainty estimation methods or  calibration methods that achieve better performance on universal accepted benchmarks?\n3. The proposed uncertainty estimation method is not well evaluated on widely-accepted benchmarks, if the author believe the proposed method is better than other uncertainty estimation methods, you can include more experiments to justify it.\n4. The computational overhead analysis is lacking.\n5. How sensitive is the method to the quality of aleatoric uncertainty estimates from the student? The robustness of the method to different PUF thresholds or uncertainty injection strengths need to be better discussed.", "was_revised": true, "final_justification": "The author has solved my concern, which is the performance of the uncertainty estimation module.", "rebuttal": "Thank you for your constructive comments. We address each point below.\n\n>Q1: The core part of this work is the SSL, which just use the uncertainty as a tool.\n\n**A1:** We thank the reviewer for the valuable comment. We agree that our work falls within the scope of semi-supervised learning (SSL), where uncertainty is employed to enhance pseudo-label quality. **The central objective of UMPL is to improve model generalization to unseen data by introducing a role-aware dual uncertainty framework.** This framework leverages epistemic uncertainty for teacher selection and aleatoric uncertainty for student feedback, forming a closed-loop refinement mechanism. Through this design, UMPL effectively exploits unlabeled data to enhance prediction accuracy in low-label scientific modeling settings. We will revise the manuscript to better highlight this focus.\n\n>Q2: Why don't just use most up to date uncertainty estimation methods or calibration methods?\n\n**A2:** We appreciate the reviewer’s suggestion. **Our main goal is to improve model accuracy in low-label settings by leveraging unlabeled data.** While more advanced uncertainty estimators exist, they offer limited accuracy gains at significantly higher computational cost. UMPL adopts a balanced choice based on this trade-off, and already outperforms baselines (shown in Q3&A3). Moreover, **UMPL is modular and can flexibly integrate more advanced teacher/student models if needed.** We will clarify this design choice and comparative results in the revised version.\n\n>Q3: The proposed uncertainty estimation method is not well evaluated on widely-accepted benchmarks.\n\n**A3:** We thank the reviewer for the helpful suggestion. Systematic comparisons of uncertainty estimation in PDE-based modeling are scarce—[1] is the only work that benchmarks multiple methods on the LE-PDE network to our best knowledge. To validate our approach, we conducted additional experiments on the public Darcy Flow dataset using AeroGTO, comparing UMPL with several uncertainty modeling baselines. \n\nMethod|MA|RMSCE|L2|Training Time(h)|Memory(M)\n-|-|-|-|-|-\nEMC Dropout(Teacher)|0.2123|0.2492|0.0388|0.52|2302\nMC Dropout|0.1781|0.2333|0.0376|3.31|16249\nSVGD [2]|0.1736|0.1954|0.0390|9.98|2253\nfeature-WGD [3]|0.1654|0.1880|0.0322|3.50|2567\nLatent-uq(Student)|0.1561|0.1778|0.0382|0.96|3243\nEnsemble [4]|0.1847|0.2044|0.0319|9.68|2395\nEnsemble latent-uq [1]|0.1297|0.1455|0.0294|13.39|3887|\nUMPL|0.0891|0.1011|0.0256|2.21|3257\n\nWe use miscalibration area (MA) and root mean squared calibration error (RMSCE) as metrics to evaluate uncertainty quality. Results show that UMPL consistently achieves better uncertainty quality and predictive accuracy. We will include these results and clarify benchmarking choices in the revision. In addition, we replaced UMPL’s uncertainty components with more advanced estimators. **While these alternatives achieved better calibration, they brought only marginal improvements in predictive accuracy and incurred significantly higher computational cost.**\n\nMethod|MA|RMSCE|L2|Training Time(h)|Memory(M)\n-|-|-|-|-|-\nT(EMC Dropout)+S(Latent-uq)|0.0891|0.1011|0.0256|2.21|3257\nT(SVGD)+S(Latent-uq)|0.0873|0.0983|0.0248|20.42|3677\nT(feature-WGD)+S(Latent-uq)|0.0842|0.0951|0.0243|6.53|5597\nT(feature-WGD)+S(Ensemble latent-uq)|0.0773|0.0877|0.0239|28.43|6273\n\n>Q4: The computational overhead analysis is lacking.\n\n**A4:** Thanks for the question. All experiments were conducted on a single NVIDIA RTX 4090 GPU. We report results on AeroGTO model across three representative tasks—2D steady-state, 2D transient, and 3D simulation—as summarized below.\n\nDataset|Darcy (bs=4)||NSM2d (bs=4)||Ahmed (bs=1)||\n-|-|-|-|-|-|-\nMethod|Memory(M)|Training Time(h)|Memory(M)|Training Time(h)|Memory(M)|Training Time(h)\nSupervised|2297|0.45|9727|3.75|9375|3.53\nPseudo Label|3843|1.08|16567|9.04|15947|8.73\nMean Teacher|3843|1.21|16567|10.17|15947|9.57\nNoisy TS|2304|1.77|9813|13.21|9477|12.48\nSupervised+uq|3243|0.96|13578|7.69|12986|7.36\nUMPL|3257|2.21|13792|16.48|13290|15.56\n\nHere, Supervised+uq refers to the student model used in UMPL. As shown, **UMPL introduces almost no additional memory overhead**, thanks to the REINFORCE-style update that avoids the costly backpropagation chain from the student to the teacher. This significantly reduces computational and memory complexity, **at the cost of increased training time**.\n\n>Q5: The sensitivity and robustness analysis.\n\n**A5:** Thank you for the helpful suggestion. We evaluated sensitivity to aleatoric uncertainty on the OOD Stationary Lid-Driven Cavity dataset using AeroGTO as the backbone. Gaussian noise of varying magnitudes was added to the student’s predicted uncertainty during training.\n\nNoisy Magnitude($\\sigma$)|MA|RMSCE|L2\n-|-|-|-|\n0.05|0.0703|0.0828|0.0634\n0.1|0.0717|0.0879|0.0634\n0.2|0.0779|0.0942|0.0636\n0.5|0.0908|0.1053|0.0663\n1.0|0.2156|0.2655|0.1499\n\nThe result shows that while uncertainty calibration degrades steadily with noise, the predictive accuracy remains stable under small perturbations. However, large noise leads to sharp performance drops, indicating that **UMPL is robust to moderate noise but sensitive to severe distortion.**\n\nWe also evaluated the sensitivity to the PUF threshold and feedback strength under the same setting: \n\nPUF threshold|MA|RMSCE|L2\n-|-|-|-|\n100%|0.0679|0.0802|0.0633\n95%|0.0661|0.0783|0.0632\n90%|0.0707|0.0809|0.0638\n\nUncertainty strength|MA|RMSCE|L2\n-|-|-|-|\nx0.1|0.0711|0.0838|0.0752\nx1|0.0679|0.0802|0.0633\nx5|0.0825|0.0974|0.0882\nx10|0.1273|0.1503|0.1367\n\n**A lower PUF helps filter noisy labels on harder datasets, though its impact is mild on this dataset. For feedback strength, too small values lead to no pseudo-label correction (reducing UMPL to plain PL), while overly large values inject noise, degrading label quality and student learning.**\n\nReference\n\n[1] Wu, Tailin, et al. \"Uncertainty quantification for forward and inverse problems of pdes via latent global evolution.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 1. 2024.\n\n[2] Liu, Q. and Wang, D., 2016. Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in neural information processing systems, 29. \n\n[3] Yashima, Shingo, et al. \"Feature space particle inference for neural network ensembles.\" International Conference on Machine Learning. PMLR, 2022.\n\n[4] Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. \"Simple and scalable predictive uncertainty estimation using deep ensembles.\" Advances in neural information processing systems 30 (2017)."}]

Author rebuttal:

Meta-review: This paper introduces an uncertainty-informed method for constructing robust surrogate models of PDE-governed physical systems under limited labeled data. Specifically, the authors propose a student–teacher model with an uncertainty-informed feedback loop designed to improve both uncertainty estimates and the quality of pseudo-labels. Extensive evaluations across seven tasks demonstrate that their method consistently outperforms the best existing semi-supervised regression approaches, even with limited labeled data.  The paper received four reviews: two accepts and two borderline accepts. All reviewers recognize the novelty of using an uncertainty-informed feedback mechanism in a teacher–student framework to simultaneously refine pseudo-labels and uncertainty estimates. They also highlight the strong empirical results demonstrated across multiple simulated and real-world datasets using different architectures.  Nonetheless, reviewers raise several concerns. These include the choice of UQ method for the teacher model, the computational complexity of the proposed approach, performance under noisy real-world data, questions about the statistical significance of the reported results, and the adequacy of the uncertainty evaluation.  The authors’ rebuttal is extensive and detailed, providing new experimental results that largely address these concerns.
Decision: Accept (poster)