PICore: Physics-Informed Unsupervised Coreset Selection for Data Efficient Neural Operator Training
Authors: Anirudh Satheesh
Anant Khandelwal
Mucong Ding
Radu Balan
Venue: Accepted by TMLR ()

Abstract: Neural operators offer a powerful paradigm for solving partial differential equations (PDEs) that cannot be solved analytically by learning mappings between function spaces. However, there are two main bottlenecks in training neural operators: they require a significant amount of training data to learn these mappings, and this data needs to be labeled, which can only be accessed via expensive simulations with numerical solvers. To alleviate both of these issues simultaneously, we propose PICore, an unsupervised coreset selection framework that identifies the most informative training samples without requiring access to ground-truth PDE solutions. PICore leverages a physics-informed loss to select unlabeled inputs by their potential contribution to operator learning. After selecting a compact subset of inputs, only those samples are simulated using numerical solvers to generate labels, reducing annotation costs. We then train the neural operator on the reduced labeled dataset, significantly decreasing training time as well. Across four diverse PDE benchmarks and multiple coreset selection strategies, PICore achieves up to 78% average increase in training efficiency relative to supervised coreset selection methods with minimal changes in accuracy.

Published in Transactions on Machine Learning Research (11/2025)
PICore: Physics-Informed Unsupervised Coreset Selection for Data Efficient Neural Operator Training
Anirudh Satheesh
onirudhs@terppmall umd. edu
Deportment of Computer Science University of Moryland, College Park
Anant Khandelwal
akhaandzkwull7990gatccc....
Georgia Institute of Tecknology
Mucong Ding
meding edu
Department of Computer Science University of Moryland, College Park
Radu Balan
rebalan rebalan&@undl @urrdd edu
Department of Mathematics University of Moryland, College Pank
Reviewed OpenReview: https: // openreview. net/ forum? id= LOVSeuTJCI
Abstract
Neural operators offer a powerful paradigm for solving partial differential equations (PDEs) that cannot be solved analytically by learning mappings between function spaces. How- ever, there are two main bottlenecks in training neural operators: they require significant amount of training data to learn these mappings, and this data needs to be labeled, which can only be accessed via expensive simulations with numerical solvers. To alleviate both of we propose PICore, an unsupervised coreset selection framework
these issues Aimultaneeuse that identifies the most informative training samples without requiring access to ground- truth PDE solutions. PICore leverages physics-informed loss to select unlabeled inputs by their potential contribution to operator learning. After selecting compact subset of inputs, only those samples are simulated using numerical solvers to generate labels, reduc- ing annotation costs. We then train the neural operator on the reduced labeled dataset, significantly decrcasing training time as well. Across four diverse PDE benchmarks and multiple coreset selection strategies, PICore achieves up to 78% average increase in training efficiency relative to supervised coreset selection methods with minimal changes accuracy.
Introduction
Partial differential equations (PDEs) are foundational to modeling complex physical systems across science and engineering, from fluid dynamics quantum mechanics. Most PDEs are non-analytic and need be solved numorically Finite Difference Methods (FDMs), Finite Element Methods (FEMs), and Finite Volume Methods (FVMs) Cyrus al. 1968); Johnson 1988); Eriksson & Johnson (1995); LeVeque (2002). However, while these approaches yield high accuracy, they computationally expensive because they require simulation to be run to obtain solution. This is especially true for high-resolution multi- resolution PDEs, where simulations need to be re-run for each resolution.
Operator learning has emerged as a tool for accelerating PDE solutions by developing data-driven approxi- mations using neural networks instead of traditional grid-based discretizations. Neural operators (Kovachki et al. 2023) are family of neural networks that learn mappings between function spaces, such as initial conditions to solutions, which allows for resolution-invaraane predictions. Models such Fourier Neural

---

Published in Transactions on Machine Learning Research (11/2025)
Operator (FNO) Li et al. 2020 and U-Net Neural Operator (UNO) RRhhman et al. 2023 have shown state-of-the-art performance on various PDE benchmarks, and the ability to generalize to higher-order res- olutions with minimal performance drops. Additional work, such as Physics Informed Neural Operator (PINO) Li et al.) 2024c) and Markov Neural Operator (MNO) (Li et al.) 2021b), incorporates additional losses into neural operator training to improve performance and increase convergence speed.
Despite these advantages, there are two main data limitations of neural operators. First, they require significant amounts of training data to learn these mappings. Since PDE solvers require high-resolution data several time frames for accurate training, such training data can be several gigabytes large (Takamoto) ct al. 2022). This poses a challenge for training in resouro--constrained systems where such models would be trained and deployed, such as for weather prediction (Pathak et al. 2022 Bonev et al. 2023) and carbon storage (Tang et al., 2024). Secondly, this training data needs to be labeled by including both the initial condition and the ground truth solution. While generating initial conditions cheap, as they can usually be sampled from prior distribution, generating ground truth data requires running the full simulation through numerical solvers.
Coreset selection (Agarwal et al. 2005 Sener & Savarese 2017 is data-efficient training strategy that identifies a subset of the original training data that is most informative for model learning. Once this subset is identified, training only needs to be done on this subset, significantly reducing training time. However, this requires the full labeled training data to select a subset, which does not alleviate the cost collecting labels. On the other hand, active learning (Gu et al. 2021 Cao & Tsang 2022) minimizes data annotation costs by only labeling subset of the training data at each iteration. Active learning selects a subset by a proxy metric such as Bayesian (Zhao et al. 2021 Beluch et al. (008)) representation-basee methods (Yang) & LLogg 2022 Kim & Shin 2022) each training iteration, and trains only that subset. A limitation of many iterative active learning strategies is that repeatedly alternating between selecting points and updating the model can increase training time and reduce convergence speed. Thus, we pose the following research question:
How can we simallaneously reduce training time and labeling ground-trulh solutions for Neural Operalor learning?
We address this problem using unsupervised coreset selection by identifying the most informative training samples based on the physics-informed loss (Li et al. 2024c), a criterion that does not require any ground truth labels. Our approach can also be viewed as a single-shot active learning implementation, where subset of points is selected one pass rather than itcratively. By leveraging this loss, we can prioritize samples likely to improve model performance without. the need for expensive simulations. Ground truth labels are then generated only for this selected subset, significantly reducing the overall annotation cost. Finally, we train neural operator models on the reduced, high-quality dataset, leading to faster training times without compromising accuracy.
Our contributions are outlined as follows:
We propose PICore, novel unsupervised framework that uniquely integrates physics- informed losses with coreset selection. PICore eliminates the need for expensive ground-truth simulations during the data selection phase, simultaneously addressing the data annotation and training bottlenecks in neural operator training.
We demonstrate the modularity and generality of the PICore framework. Our method not tied to a specific architecture or selection algorithm, and we show its effectiveness across two different neural operators (FNO and UNO) and five distinct coreset selection strategies.
We present the first comprehensive benchmark for coreset selection in the context of ncural operator learning. Through extensive experiments on four diverse PDE datasots, we show that PICore achieves competitive accuracy supervised methods while improving end-to-end training efficiency by up to 78% relative to supervised coreset selection.
2

---

Published in Transactions on Machine Learning Research (11/2025)
2 Related Work
2.1 Neural Operators
While typical deep neural nets used to map and model finite-dimensional vector spaces, such text embeddings or images, neural operators map infiniteddimessinn vector spaces, such as the space of functions [Kovachki et al., 2021). Neural operators are then widely used to represent differential equation solutions due to their ability to have a family of solutions. In the context of solving partial differential equations, a neural operator can lake a ionction as an input (e.g. lemperature al. a point) and output a related function (e.g. heat over time at point).
Among the first modern neural operators, DeepONet (Lu et al. 2021) uses the universal approximation theorem for operators with a branch and trunk network to model inputs and outputs. The Fourier Neural Operator (FNO) [Li et al. 2020) expands on this by performing kernel operations in Fourier space, which results in a more expressive model with better performance on more challenging PDE datasets, such as Navier Stokes. U-Net Neural Operator (UNO) (Rahman et al., 2023 expands on FNO by using a U-Net basod structure to build deeper neural operators, and Convolutional Neural Operator (CNO) (Raonic et al. 2023 leverages convolutions preserve the continuous structure of PDEs, even when discretized. Additional work improves training by incorporating additional losses. Physics Informed Neural Operator (PINO) (Li et al. 2024ch usee the physics informed loss to anchor the output to conform to the PDE dynamics, and Markov Neural Operator (MNO) [Li ct al. 2022) uses dissipativity regularization to improve accuracy for more chaotic systems.
2.2
Data Efficient Machine Learning
2.2.1
Coreset Selection
For problems where training is too expensive or slow, coreset selection can accelerate training while preserving accuracy. Coresct selection methods can be largely categorized into two types: training-fres methods that leverage the geometric properties of the data, and training-baser methods that use model-specific information
to score data points. Training-Fi free methods involve random (Guo et al., 2022] Guppa et al.) 2023) and
goometry-informed selection (Welling) 2009 Chen et al., 2012). Recent work training-basee methods
be split into three groups: (i) submodular approachos to maximize the coverage of the sclected dataset (Wei
et al. 2015 Mirzasoleiman el al. 2020: Pooladzandi et al.
2022). (ii) gradient-based approaches to exactly
find the infiuence of data point Killamseety et al. 2021a Paul et al. 2021). and (iii) bilevel optimization
mothods to improve gencralization performance (Killamsetty et al. 2021c bj.
The traditional lestbed for coreset selection algorithms has been image classification lasks, but also has applications in Neural Architecture Search (NAS) (Shim et al. 2021]. efficient GAN training (Sinha et all 2020), continual learning (Yoon et al., 2021), and large language model (LLM) finetuning (Zhang et al.
2025 However, to the best of our knowledge, coresct selection has not been used for improving the training efficiency of neural operator learning.
2.2.2
Active Learning
In contrast to coreset selection, active learning, over multiple iterations and in an unsupervised environment, chooses previously unannotated data to label and trains on those newly labeled pairs (Li al. (025). The key differences are that active learning is unsupervised, choosing training samples with only features and that active learning is done over many iterations instead in single shot. Many algorithms transfer from coreset selection to active learning. Aside from the equivalent random selection, there are cluster based methods for active lcarning to find representative and typical examples (Sener & Savareso 2017 Hacohen et al. 2022) and uncertainty based methods that find data for which the model is either uncertain or degraded (Rahmati let al. 2024 Houlsby et al.) 2011 Ash et al.) 2020). Wu et al. (2023) and Musekamp et al. (2025) show several uncertainty based sampling methods and active learning methods for physics informed learning, but these are limited PINNs and not to neural operators, which are more powerful due to their ability map between function spaces, but inherently more difficult. to perform active learning.
3

---

Published in Transactions on Machine Learning Research (11/2025)
2.3 Active Learning and Data Efficiency for PDE Surrogates
Several works have explored improving data elliciency for PDE surrogate modeling through pretraining architectural constraints. The closest to our approach is Chen et al. (2024), who introduce an unsupervised protraining stratogy using Masked Autocncoders (MAEs) to learn useful representations that are later fine- tuned with a smaller labeled dataset. While this reduces labeling requirements, it does so through a two-stage training pipeline. In contrast, PTCore directly reduces both training cost and data annotation cost. within a single training cycle. [Hmmmasian & Farimani (2024) reduce the cost of generating high-resolution training data by pretraining neural operators on lower- diver--dmensioall problems. This approach, however, relies lactorized architecture such as the Factorized Fourier Neural Operator (FFNO) (Tran et al. 2021 whereas
PICore is operator-architecture agnostic and can be applied to a wide range of neural operator models.
Another related line of work is active learning for PDE surrogate modeling. Li et al. (024b) propose an active learning strategy tailored to FNOs, selecting new simulations by maximizing a utility cost ratio. This method effectively reduces simulation labeling cost but does not address end-to-end training efficiency. Similarly, Kim et al. (2025) use a surrogate model perform active learning at the timestep level, which improves efficiency only locally in time. In contrast, PICore reduces the annotation cost of full solution trajectorics in single pass. Beyond these architecturc--peeiffc approaches, Musckamp et al. (2025) study active learning for PINNs and Neural Operators more generally by evaluates several acquisition functions aimed at improving operator generalization under limited data. Overall, while prior work has made progress toward reducing labeling burden or improving sample efficiency, existing methods either rely on multi- stage pipelines, architectural constraints, or hcuristic based active sampling. PICore differs by providing an architecture-agnosiic, physics-informed coreset selection framework that improves both training efficiency and labeling cost in unified manner.
3 Preliminaries
3.1 Neural Operators for PDE Solution Generation
Many physical systems can be modeled using partial differential equations (PDEs), which describe the evolution of a function u over a domain. general PDE can be expressed as
F(u, a) 0, on C Rd
(1)
where E A represents input parameters such boundary conditions, initial conditions, physical cocf- ficients; F x A 2 is a differentiable and potentially nonlinear operator; and A, are Banach spaces over the bounded domain f
For stationary (time-independent) PDEs, the problom takes the form
F(u, a) = 0, on C R M h on an, where h defines the boundary condition the domain boundary M.
(2)
For dynamic (time-dependent) PDEs, the input a is restricted to the initial condition u/c=0, and the operator F is defined on the spatiotemporal domain x T:
F(u, a) 0, on Ω T, u h, on 0N T,
u on Ω x (0):
where T= 77 T) denotes the time domain. Examples of both stationary and dynamic PDEs are provided in Section
Unlike conventional neural networks that learn pointwisc mappings, neural operators approximate solutions by learning mappings between infmitteddimensionall function spaces:
G A:A+U.
(4)
4

---

Published in Transactions on Machine Learning Research (11/2025)
In practice, PDE dataset consists of pairs f(as, u4)) where each (a,, u4) corresponds to imput-output solution of the PDE. The neural operator G is approximated by Go through the optimization
N
Ge = arg min Ee
[o((,) uzlli2(0)
(5)
N
where θ is finito-dimensional parameter space.
3.2 Coreset Selection
Given a dataset (Iri, M)) coreset selection aims to find subset C D such that
arg min B(srs))-) [[(, (1:00))
(6)
S'C
where B is the percentage of the original dataset selected and 8s is the model trained on S. However, there
N = -(22H)(0)) possible subsets of size BN, so evaluating this objective directly is infeasible
are AN for large datasets. Instead, some works leverage submodular function S 2D R which ensures the diminishing return property
((TU - C
(7)
This results in greedy selection procedure, significantly reducing the subset search space. Another way to perform coresct selection is to scoring function and select the top-k data points. Finally, coreset selection can be represented a bilevel optimization problem, resulting in the following form
=
(S)) s.t. 0'(S") arg min 1'
arg arg min
Σ (F..I))S)
C( (6;
o
4 PICore
To address both issues of training time and data labeling costs for Neural Operator learning, we introduce PICore, an unsupervised coreset selection method that leverages a physics-informed loss to bypass the need for labeled training data during coreset selection.
Instead of using the ground truth PDE solution and supervised losses, the physics-informed loss evaluates the degree to which operator approximation Go(a) satisfies the governing PDEs defined in either the stationary form or the dynamic form. The physics-informer loss penalizes violations of the PDE (PDE residual) in the interior of the domain, as well as deviations from the given boundary and initial conditions. For neural operators, the physics-informed loss is defined as
Lrr(a; 0) (
((Go(()) 12(0)
+ + - =nl"
(9)
for stationary PDEs and
F(Go(a)) Lpr(a; 0) - ((((TXT)
-
+ (o(a))
(10)
((0X))
(2(0))
for dynamic PDEs.
Given solely an unlabeled dataset D = (ai) that can be cheaply generated (usually by sampling from prior distribution or sensor readings), PICore selects a coreset of D by solving
arg min Egg-s [Cps (a; (a::0))
(11)
S'
5

---

Published in Transactions on Machine Learning Research (11/2025)
(p)(0)
0.825
Numerical Simulation
Lr1(a2:0)
0.980
C
(p:(as::))
0.315
Go
(a:r(a);0)
0.634
Ejes y ddaa (a) Σjes
0.451
Figure 1: Overview of the PICore Framework. Given set of initial conditions and a pre-trained (warm- started) neural operator Go, compute the plysics-informed loss Lpr(a;;) 0) for each initial condition d, These losses are passed to coreset selection algorithm C, which identifies the most informative samples that deviate most from the underlying PDE. Each selected sample is assigned weight Y and is then simulated using a numerical solver to obtain the PDE solutions. The resulting labeled subset used to update Go using a weighted data loss, enabling efficient. Training by focusing on the most impactful data points. In the figure, blue arrows represent forward passes and red lines represent backward passes respectively.
using any existing coreset selection algorithm where 0 is the operator trained on S After selecting the coreset S, we simulate the true solutions u! = G(a,) for each a, E S using a traditional numerical solver, which forms the labeled subset D. {a,, u/))a,es. Finally, we train the neural operator Go on D. for T epochs with the standard supervised data loss
Casta (a;, w!) = ([Go(a)) ([2
(12)
Before coreset selection, we warm-start the neural operator with the physics-informed loss over the full dataset for small number of epochs Tw << T. Warm starting is common in prior coreset selection methods Killamsetty et al. 2021a) and is necessary as most coreset selection algorithms require gradient information, which is unusable with randomly initialized model. We provide the full algorithm in Algorithm I
Computing PDE Residuals One challenge with using the physics-informed loss in coreset selection is computing the PDE residual F(Ge(a),a). The residual requires computing derivatives of the neural operator with respect to the dimensional parameters, such @e Li et al. (202cc) uses function-wise differentiation method via Fourier differentiation to compute these values exactly, but this does not extend to general class of neural operators. We also tried vitoodiiferentiation methods, but these were highly computationally expensive, increasing the coreset selection time. Thus, we settled on simply using finite difference methods, which are efficient with linear time complexity the input resolution.
dmdt
5 Experimental Details
We conduct experiments on four representative PDE benchmarks spanning both stationary and time- dependent dynamics widely used the neural operator literature:
1D Advection Equation (time-dependent): A linear hyperbolic PDE representing pure transport dynamics, used to test propagation accuracy.

---

Published in Transactions on Machine Learning Research (11/2025)
Algorithm PICore: Plysics-Informed Coreset Selection for Neural Operators
Require: Unlabeled dataset {a: ii coreset size k 3N; learning rate a; pretrained operator Go:
physics-informed loss Car(a; 0); coreset selection algorithm Ceelecti warmuu steps Twi training steps T
1: Warm-start Go on unlabeled data using the plysics-informed loss:
2: for t = 1 to Tw do
for each 0; E do
0 (a;5)(
end for end for
7: Score each sample using physics- informed loss:
for each a, E D do
4 Lpr(a;; 10: end for
11: Select coreset indices using C
12: Crolect ((
13: Simulate ground truth for selected coreset:
14: De 0
15: for each i E S do
16: u¹ (a,)) {Run numerical simulation]
17: D. U-D
18: end for
19: Train Go Da using supervised loss:
20: for f = 1 to T do
21: for each (a:, u!) ) E De do
22: 0- 0 a V LdLataa u!:0)
23: end for
24: end for
1D Burgers' Equation (time-dependent): A nonlinear convection-diffusil PDE with periodic boundary conditions, modeling shock formation and dissipation.
2D Darcy Flow (stationary): A second-order elliptic PDE used to model pressure fields in porous modia given heterogencous permcability.
2D Navier-Stokes Incompressible Equation (time-dependentt): A nonlinear incompressible flow equation solved on a periodic domain.
Each dataset has 1000 generated trajectories, with 900 that can be used for training (varying based on the coreset selection percentage) and 100 for testing, which is comparable to existing neural operator literature (Li et al. 2021a 2024c). We generate 20 timesteps forward for Advection and Burgers, and only 10 timesteps for the Navier Stokes Incompressible dataset due to memory limits. Additional information on the datasets can be found in Section A We use the Fourier Neural Operator (FNO) (Li al. 2020 and U-Net Neural Operator (Rahman et al. 2023 as the base models for all experiments due to their implementation simplicity and performance. However, PICore selection algorithms in our experiments: CRAIG (Mirzasolciman et al. (020),, GradMatch (Killamsctty et al. 2021a), AdaCore (Pooladzandi el al. 2022). EL2N (Paul et al. 2021 and graNd (Paul el al. 2021). CRAIG, AdaCore, and GradMatch are submodular methods that try to match the gradient sum the coreset to the gradient sum of the entire dataset. GraNd and EL2N are score based methods that use the gradient or the loss. Additional information on these coresct selection algorithms can be found in Section B We use coresct selection percentages of 20%, 30%, 40%, 60%, and 80%. We report the results of each experiment, with the
work out of the box with any neural operator. We also use 5 coreset
7

---

Published in Transactions on Machine Learning Research (11/2025)
normalized root mean square error loss (NRMSE):
|| (a:) u, (ax))
llu.1122((xx)
used in Takamoto et al. 2022 We use this as normalized version of the data loss because the value of the if at each spatiotemporal point is very small, resulting small MSE values and potential gradient vanishing during training. We also use the uniform spatiotemporal discretization an input resolution of 64 for Ωt Since FNO and UNO are resolution invariant, also evaluate at higher resolutions for zero-shot super resolution in Section C.6 For all experiments use X = and μ 1, but this is relatively arbitrary, we did not conduct any hyperparameter tuning.
We use To = 25 warmup epochs and reset the neural operator to its initialization to ensure fair comparisons between supervised and physics-infon informed coreset selection. Then, we train neural operators for T = 500 epochs and report the average NRMSE over scods on a held-out test set the input resolution. We caleulate the acceleration as the total time taken for supervised coreset selection PICore (including data generation, warm starting, and training time) divided by the total time for the non-coreset baseline.
In addition to supervised coresct selection, we compare PICore to random subset selection, pure unsupervised training with the physics informed loss, and an active learning baseline based on uncertainly. Since most active learning baselines are for classification problems, we extend loss-s-uncertainty methods in Liu & Li (2023; Beluch et al. (2018) to neural operators. For the active learning baseline, we begin by randomly selecting 10% of the available data as an initial training set and generating the corresponding ground -tuth PDE solutions. We then train 10 independent copies of the neural operator on this subset for The epochs, which is the same training time for the other methods. After training, construct the final coreset from the remsining unlabeled data in a single step by selecting the points exhibiting the highest variance across the model predictions.
6 Results
6.1
Main Results
We report the core findings for PICore and supervised coreset selection across the four representative PDE datasets in Tables I and 4 In addition to the average test NRMSE over the best coreset selection algorithm for each method, we show the decrease in full training time (including data annotation costs through simulation) relative to the non-coreset selection baseline. Our results demonstrate that PICore consistently achieves competitive test performance compared to supervised coreset selection while providing substantial computational efficiency gains, primarily by reducing expensive data annotation (simulation) costs during the coreset selection phase.
PICore significantly improves training efficiency through reduced simulation costs. Across four representative PDE -adiettt--dveiiioo. Burgers, Darcy, and Navier-Stokes Incompressible-P9 consistently reduces the total training time by cutting down expensive simulation-based annotation. These efficiency gains become especially significant as the complexity of the PDE increases: Across the four datasets, PICore achieves average training time reductions 0.9%, 9.8%, 30.1%, and 78.0% compared to supervised coreset selection, calculated by averaging the relative acceleration improvements at each selection percentage (20%, 30,, 40%, 60%, and 80%). For example, at a 20% corcsct size, PICore achicves specdups of 5.01 on Darcy Flow (vs. 2.24x for supervised methods) and 5.00x on Navier-Stokes (vs. 1.14x) using UNO.
As shown in Tables 6 7 and S the relatíve contributions of training and data generation speedups vary by dataset difficulty. For simpler datascts such as Advection and Burgers, efficiency gains are driven primarily by reductions in training time. For example, Advection achieves a 79.7% improvement in training time but only a 1.47% improvement in data generation time at the 20% coreset level. In contrast, for more challenging datasots, the impact of training time reductions diminishes, while reductions in data generation time play a more significant role in overall efficiency gains. These results show that PICore scales well high-dimensional scientific problems where data annotation costs dominate training.
8

---

Published in Transactions on Machine Learning Research (11/2025)
PICore matches supervised coreset methods in test accuracy at reduced data budgets. Despite the substantial efficiency improvements, PICore remains competitive with strong supervised baselines in terms of test NRMSE. For instance, at a 20% coreset size, PICore achieves NRMSE values of 3.46 x 10-2 for Advection using FNO and 2.84 x 10-2 for Burgers using UNO, very close to the supervised mothod values of 3.42 10-2 and 2.93 x 10-2, respectively. This holds across different coreset sizes, with minor variations, showing that PICore can be as effective as supervised coreset selection, even at low coresct selection percentages. In fact, many dataset, model, and selection percentages combinations show that PICore improves upon supervised coreset selection, but there is largely no significant change in accuracy between the two methods. However, we observe that not all coreset selection algorithms perform equally as well. Due to the convexity assumptions and Hessian approximations with CRAIG (Mirzasoleiman et al., 2020) and AdaCore (Pooladzandi et al., 2022), they have higher NRMSE losses compared to the other algorithms. Thus, the tables almost always report GradMatch, GraNd, or E12N as the coreset algorithm to use for PICore and supervisod coresct selection.
Coreset. Selection methods outperform Random and Active Learning baselines on most datasets. Random subset selection consistently underperforms compared to both PICore and supervised coresct selection, with the performance gap widening as datasct complexity increases and the sclection per- centage decreases. For instance, on the Navier-Stokes Incompressible dalaset al 20% coreset size, random selection yields an nRMSE of 2.74 x 10-1, whereas PICore achieves 1.12 x 10-2 and supervised coreset selection achieves 9.57 x 10-2 These results highlight that leveraging PDE-specific information, whether through supervised loss signals or plysics-informed residuals, is crucial for attaining high-accuracy solutions with limited data. Nevertheless, the quality of the PDE residual approximation can introduce noise that diminishes performance, sometimes making PICore worse than random sampling on simpler problems such as Advection. In such cases, random sampling can reduce biases toward specific attributes of data points, thereby lowering errors attributable to those biases. We also observe that classical active learning occa- sionally outperforms PICore. For example, on the Advection dataset with UNO at medium coreset sizes (40 60%) and on the Navier-Stokes Incompressible dataset with FNO. However, across all other dataset- model combinations, active learning performs substantially worse, often by a wide margin.
Physics-informed training alone is insufficient to achieve high accuracy. Training a neural opera- tor solely with the physics-informed loss on the full unlabeled dataset yields significantly worse performance than all other baselines, despite requiring no simulation cost. For example, on the pressible dataset with FNO, the dbysice-informed-ont baseline achieves an nRMSE of 1.46 x 10-0, compared to PICore's 1.25 10-1 at 20% coresct selection, decrcase of an order of magnitude. This large gap ariscs because the plyssic--iffomedd loss is unstable as standalone training objective and fails to capture fine- grained solution details without supervised guidance. In contrast, PICore leverages the ahysics-infrrmer loss as a proxy for selecting informative samples, then trains on a small labeled subset with the supervised nRMSE loss, which mitigates the chysics-informan loss instability.
Incom-
There is a tradeoff between efficiency and absolute test accuracy. While PICore offers strong performance and efficiency, one tradeoff is that the absolute test accuracy relative to training on 100% of the data is lower. For example, on the Advection dataset with FNO, the 100% training baseline yields an NRMSE of 2.13 x 10-2, while PICore at 20% yields 3.77 10-2 However, this is an inherent tradeoff for all coreset selection algorithms, Additionally, this is not specific to PICore, as similar reductions in accuracy hold for supervised coreset selection. In practice, one may want to select a higher selection percentage, such as 40%, which would yield higher accuracy (2.69 x 10-2) while still maintaining a competitive efficiency gain (2.54x).
the selected coreset simply contains less information for training,
6.2 Further Results
In addition to investigating the efficacy of PICore, we also aim answer the following questions: (1) How does PICore compare to existing unsupervised dataset selection methods? (2) How different are subsets selected by PICore compared to those selected by supervised coreset selection?
9

---

Published in Transactions on Machine Learning Research (11/2025)
Table 1: Advection NRMSE at resolution 64
Operttor
Method
20.0%
30.0%
40.0%
60.0%
80.0%
100.0%
Plysic- Inkorme
8.13 1: 0:03 10-+
8.37 1: 0.03 x 10-1
8.73 1: 0.04 10-
8.98 1: 0.02 x 10-1
0.14 10-1
9:26 1: 0.03 10-
(4.72x)
(3.06x)
(2.38x)
(1.59))
(1.19x)
(0.96k)
PNO
Random
3.39 + 0.07 x 10-?
2.89 + 0.00 10-2
2.68 0.02 x 10
2.47 + 0.00 x 10-3
2.37 + 0.04 x 10-2
2.22 + 0.05 x 10-
(5.10x)
(3.92x)
(2.56x)
(1.72x)
(1.28x)
(1.00x)
Active Learning
8.32 1: 0.58 10-2
6:29 1 0,00 x 10-2
4.78 1. 0.27 10
3.51 4 0.16 x 10-2
2.96 1 10-2
2:22 1: 1:0.0 x 10
(6.04x)
(3.28x)
(2.52x)
(1.69)
(1.26x)
(1,00x)
Supervised (graNd)
3.42 + 0.02. x 10-?
2.96 + 0.09 x 10-3
2.64 + 0.03 x 10"
2.42 + 0.00 x 10
2.25 + 0.02 x 10"2
2:22 0.00 x 10"2
(4.70x)
(3.15x)
(2.4x))
(L86x)
(1.26x)
(1.00x)
PlCure (graNd)
3.16 1.110.133 10-2
3.01 1: 0.15 x 10-2
2.09 1 0.05 x 10-2
2.40 0.04 10-2
2:25 0.01 x 10-2
2:22 0:006 10
(5.06x)
(3.27k)
(2:54x)
(1.68)
(1:26x)
(1,00X)
Plosice-tfoomee
8.07 0.05 10-1 (487%)
8.10 + 0.06 x 10-1
8:88 + 0.05 10
8:46 0.18 x 10-1
0.00
0.37 10-1
9:25 0.30 10
UNO
Random
x 10-1
1.50
(3.21x)
0.01 10-1
1.44 0.007 x 10
(246x)
1.42 0.12 10-1
(1.65)
0.112 10-1
(1.23%)
(0.98x)
0.25 10-2
(5.08x)
(3.35x)
(2.55x)
(1.70x)
(1.28x)
(1.00))
Active Learning
0:00 10
1.39
0.05 10"1
0.73 10
7.49
10-2
0.05 10"2
7.27 7.27::0.28 10"R
(5.05xc
(3.32x)
(2.52x)
(1.68%)
(1.26%)
(1.00x)
Supervised (gradmateh)
1.55 0.02 10-
1488
0.01 10-1
1.42 0.02 10-2
1.17 + 0.14 10
9.00 +:223 x 10-2
7:27 0.28 10"2
(188x)
(3.23k)
(2.47x)
(L.07))
(1.25%)
(1.00x)
PICure (graimasch)
1 1.55 1 0.01 10"
1.47
0.00 10
1:43 4: 0.008 10"
1.25 1: 0.00 x 10
B.06 1:00 10
7.27 9:2.6 10-2
(6.07x)
(3.34x)
(2:53x)
(1.69x)
(1.26x)
(1.00x)
Table 2: Burgers NRMSE at resolution 64
Operator
Method
20.0%
30.0%%
10.0%
60.0%
80.0%
100,0%
Physles-Ifformed
5.26 0.12 x 10-1
4.55 0.06 x 10"1
10
4.23 + 0.03 x 10-1
0.03 10-1
4.12= 10-1
(5,36x)
(3.45x)
(2.70x)
(L.00)
(L.34x)
(1.07x)
FNO
Random
0.08 10-2
1.18
10-2
0.21 10-3
5.82 0:21 x 10-3
4.75
1.13 10-3
0.10 10-3
(5.07+)
(3.31x)
(2.56x)
(1.72x
(1.28k)
(1.00x)
Active Learning
2.52 10
4.57
10 (3:25x)
0.57 10
2,06 1 0.07 10
1.118 10
0.100 10
(5.03xk
(2.52x)
(1.68)
(1.26x)
(1.00))
Supervised (pradmatch)
0.16 10-2
1.12
0.090 10
7.68
0.29 10-3
5.24
10-3
4:13
0.08 10-3
3.95
1.95+00113 10
(3.28x)
(2.5xx)
(2.11x)
(1.55)
(1.22x)
(1.00x)
PICore (el2u)
1.11 4 0.06 10
1.12
0.07 10
8.07 0.33 10-3
5.49 1 0.08 x 10
4.07 1 10
3.95 2.10 X 10
(5.05k)
(3.30x)
(2.53x)
(1.68x)
(1:26x)
(1.00<)
Inaysic-Ilforuee
1822:0007 x 10-1
1.56 = 0.00 10
4,48 1 0.06 10
4.61 :: 0.03 x 10"
4.53 1 0.02 10"
0.65 10"1
UNO
Random
2.92 + 0.06 x 10-2
(5.28x)
2.55 = 0.05 x 10-2
(3.46x)
2.25 0.05 10-2
(2.66x)
1.80 + 0.03 x 10-2
(1.77x)
1.58 + 0.01 10-2
(1.33x)
1.49
=.04 x 10-2
(1.06x)
(4.99x)
(3.34x)
(2.55x)
(L.70x)
(1.27x)
(1.00x)
Active Learning
5.42 0.2000000 x 10-2
4.12 = 0.11 x 10
3.62 1 0.20 10
3.03 1 0.23 x 10"?
2.51 i 0.13 x 10
1.49: 0.00 10-2
(5.01x)
(3.30x)
(2.51x)
(1.68)
(1.26x)
(1,00x)
Supervised (gaddmathh)
2.53 + +.110 x 10-2
2.42 0.06 x 10-2
2.08 0.05 x 10
1.72 + 0.03 X 10-2
1.54 + 0.00 x 10
1.49=00.04 x: 10-2
(3.77x)
(2.77x)
(2.23x)
(1.59x)
(1,23x)
(1.00x)
PICore (gmNdd
2.84 1 0.05 x 10
2.36 0.05 10
2.08 1: 0.00 x 10
1.72 1 0.05 x 10
1.77 1000 x 10
1.19 0.00 10
(5.05x)
(3.33x)
(2:32x)
(1.6%)
(1.26x)
(1,00x)
6.2.1 Unsupervised Coreset Selection
We compare PICore to three unsupervised coreset selection methods: k-means clustering, cosine similarity, and Herding (Chen et al. 2012). For k-means clustering we use kt = BN clusters, and choose the data points closest to those clusters. For cosine similarity, we evaluate the cosine similarity between all pairs of points, and perform greedy selection to choose the coreset. We report direct comparison of the test NRMSE for both methods in Figures 2 3 and 5 in Section C.1 The results show that PICore consistently matches or outperforms the unsupervised baselines across all tested coreset sizes (20% to 80%) and neural operator architectures (FNO and UNO). For instance, on the Advection dataset at 20% coreset size, PICore with the EL2N algorithm achieves a test. NRMSE of 3.29 x 10-2, outperforming cosine similarity 3.39 x 10-2 and herding 3.46 x 10-2, Similar patterns are observed on the other datasets, indicating that PICore's selection strategy generalizes well across both time-dependent and stationary PDEs compared to other unsupervised coreset selection strategies. We also note that these trends hold across neural operator architectures, with PICore outperforming unsupervised methods with both FNO and UNO architectures. While FNO does consistently outperform UNO across datasets (except Navier Stokes Incompressible), this is due to the architecture differences and not due to PlCore, as shown by the increase in NRMSE for UNO on the non- coreset baseline.
These results suggest that incorporating PDE-specific information can provide benefits over generic unsu- pervised selection methods. While clustering and similarity- approaches provide reasonable coverage of
10

---

Published in Transactions on Machine Learning Research (11/2025)
Table 3: Darcy NRMSE at resolution 64
Operator Method
20.0%
30.0%
40.0%
60.0%
80.0%
100.0%
Plasice Enformel
1.46 0.005 10"
1.7 =: 0.005 1PP
1.47 1 0,002 x 10"
1.47 1: 0.001 10"
1.47
0.002 109
1,48 1 0.001 X 10F
(7-43x)
(4.96x)
(3.75x)
(2.50x)
(1,88x)
(1.11)
PNO
Random
1.34 0.03 x 10-1
1.15 + 0.01 x 10
9.90 + 0.11 x 10-2
7.94 0.04 x 10-2
7.07= +0016
10-2
6.18 + 0.09 x 10-2
(5.00x)
(3.36x)
(2.53x)
(1.09)
(1.27x)
(L.0x))
Active Learning
2.01 2 0.160. x 10-2
1.58 1 0.08 x 10-1
1.25 1 0.06 x 10-1
8.91 0.33 x 10-2
7.19
1.225 10-2
6.18 1 0.09 10-2
(4.98kx)
(331x)
(2.50x)
(1,86x)
(1.26x)
(1,00x)
Supervised (el22n)
28 0.01 x 10-3
1.07 + 0.007 x 10-1
9.43 + 0.00 x 10-2
7.83 + 0.18 x 10-2
6.59 + 0.09 x 10-2
6.18 0.00 x 10
(1.98x)
(1.76x)
(1.58x)
(1.38x)
(1.14))
(1.00x)
PiCare (e12m)
1.25 0.02 10-1
1.12+
0.02 10
9.44 + 0.12 10
7.77
10
0.18 10-2
6:18 0.09 10-2
(5.00x)
(3.32x)
(2.50x)
(1.67x)
(1.26x)
(1.00))
Physies-Tffoemed
0.002 10P
0.003 10P
0.000 10
0.001 10P
1.42 0.002
1.43 0.003 10F
(7.05x)
(4.71x)
(3,55x)
(2.37x)
(1.78%)
(1.16x)
UNO
Randon
1.45 0.012 10
0.03 100
1.10 1: 0.02 10"1
0:23::020 10
10-
7.57 0.13 10-
(8.03x)
(3.37x)
(2.53x)
(1.8k)
(1.27%)
(1.00x)
Active Learning
1. 187 0.14 x 10-2
1:44 0.08 10
1.27 0.06 10-1
1.02 = 0.03 10
8.63 0:199 1012
7.57 + 10
(5.04x)
(3.35x)
(2.52x)
(1,88x)
(1.26x)
(1.00x)
Supervised (gradmmatch)
1.28 + 0.00 x: 10-)
1:44 + 0.01 10"1
9.84 + 0.16 x 10
8.00 - 0.11 10
7.70 0.100
10-2
7.57 + 0.13 x 10-2
(2.20x)
(L.3x)
(1.71x)
(1.38x)
[1.168)
(L.00x)
PICare (graNd)
1.28 1.0.03 10-1
1.12 1 1.0.1 10
9.67 0.36 10-2
8.42 0.14 10-2
7.61 10011 X 10-2
7.57 1.0.13 x 10-2
(3.01x)
(3.33%)
(2.50x)
(1.67x)
(1.25x)
(1.00x)
Table 4: Navier Stokes Incompressible NRMSE at resolution 64
Operatoor
Method
20.0%
30.0%
10.0%
60.0%
80.0%
100.0%
Plysics-tnformed
1.01 + 0.001 x 10"
1.01
+.0002 x 10"
1.02 + 0.002 x 10"
1.02 + 0.002 x 10"
1,03 + 0.002 10P
1.03 - 0.002 109
(09.41x)
(46.30x)
(34.73x)
(23.20x)
(17,43x)
(13.32x)
PNO
Random
2.74 d: 0.45 x 10-1
5.59 0.80 10-2
1.33 0.03 10-2
9.06 0.224 10-3
6.87 4 0.13 x 10-3
5.60 4 1.111 x 10-3 (L.0x)
(5.00x)
(3.3(x)
(2.50x)
(1.07x)
(1.25x)
Active Learning
9.32 6.96 10
2.16 4: 0.68 10
1:27 0.003 x 10
7.86 7.86+0.11 0.14 10
6.24 0.21 10
5.66 + 0.11 10
(5.00x)
(3.33x)
(2.50x)
(1.67x)
(1.25x)
(1.0x)
Supervised (e12m)
9.57 3.87 10-2
0007 10-2
=000 10-2
10-3 7.91::118
6.28 0.16 10-3
5.66 0.11 10-3
(1.05x)
(1.05x)
(1.04x)
(1.03x)
(1.01x)
(1.00x)
PICare (gnNd)
1.12 4: 0.45 x 10-1
181 0.12 10
1.3 +0.05 10-
8.0000223 10
6.34 0,14 10-3
5.66 + 0.11 10-3
(5.00x)
(3.33x)
(2.50x)
(1.67x)
(1.25x)
(1.00x)
Playiic Informed
1.01 a 0.001 10"
1.03 0.000 10"
1.03 a 0.001 10"
1.0 1 0.001 x 10"
1.04 L 0.002 10"
1.0
0.003 10P
(29.42x)
(19.61x)
(14.74x)
(9.86x)
(1.38x)
(5.80x)
UNO
Random
2.72 à 0.04 x 10-2
2.24 0.02 10-2
1.96 0.01 10-2
1.11 a 0.000 10-2
1.38 a 0.004 x 10-R
1.24 A: 0.004 X 10-2 (1.0xx)
(5.02x)
(3.36x)
(2.51x)
(1,67))
(1.25k)
Active Learning
2.91 + 0.06 x 10
2.36 + 0.05 x 10
2.06 0.02 10
1.61 0.00 x 10-2
1.40 + 0.008 x 10-2
1.24 + 0.004 x 10-2
(5.00x)
(3.33x)
(2.50x)
(1667x)
(1.25x]
(1.00))
Supervised (el2n)
2.60 :: 0.00 x 10
2.19 = 0.02 x 10
1.93 0.01 10
1.57 = 0,009 10-2
1.38 1: 0.010 x 10-2
1.24 0.004 x 10
(L.1X)
(1.12x)
(1.10x)
(1,07x)
(1.63)
(1.0x))
PiCare (grddmatch)
2.50
10-2 (5.00x)
2.20 x
1.92
10-3
0.000 10-2
0.007 10
1.24 0.004 10
(3.33x)
(2,50x)
(1.67x)
(1.25x)
(1.00x)
the input space, PICore's physics-informed selection offers more targeted approach that identifies samples where the model violates physical constraints. This results in generally competitive or improved predictive accuracy, though the margin varies across datasets and selection percentages. For best results, practitioners should use EI2N
6.2.2 Convergence of Coreset Selection vs Active Learning
A key distinction between coreset selection and active learning lies in their approach to data selection, which in turn affects their convergence speed. This iterative nature can be suboptimal, as it can lead to selecting redundant data points (Li et al. 2024a).. While some works have shown superior convergence of active learning methods (Haimovich et al. 2024). these are under specific optimizer settings and in easier image classification domains. Our empirical results largely validate this viewpoint, demonstrating that PICore's single-shot selection generally lcads to better subset selection that converges faster than active learning baselines. Figures 7 and g show the training loss convergence of PICore's coreset selection methods compared to the active learning baseline. For both FNO and UNO, the active learning method converges much slower by 2-3x. The difference in loss convergence decreases for more complex datascts such as Navicr Stokes, but this is due to learning a larger FNO-3D / UNO-3D model than due to the subset selection method itself.
11

---

Published in Transactions on Machine Learning Research (11/2025)
Coreset %
FNO
UNO
Cureset %
FNO
UNO
Traiu
Dala
Wacn-up
Trsin
Data
Warm-up
Teain
Data
Warm-up
Train
Warm-up
20.0%
+70.7%
+1.4%
-0.9%
+80.4%
+008%%
-4.6%
20.0%
+70.3%
+10.7%
-0.8%
+74.4%
+6 63
-4.6%
30.0%
+09.7%
+1.2%
-1.4%
+70.7%
+0.75
-1.5%
30.0%
+61.5%
+9.4%
-1.3%
+63.1%
+5.8%
-1.5%
10.0%
+61.3%
+1.1%
-1.8%
+61.7%
+4..6%
+4.5%
40.0%
+54.0%
+8.1%
-1.6%
+57.1%
+5.0%
-4.5%
60.0%
+12.8%
+0.7%%
-2.8%
+13.2%
+0.45
-1.5%
60.0%%
137.7%
15.4%
-2.4%
139.9%
13.3%
-4.5%
80.0%
+24.2%
+0.4%
-3.7%
+24.2%
+-0.2%
80.0%
+21.4%
+2.7%
-3.3%
+22.3%
+1.7%
-4.5%
Table 5: Advection PICore component speedup.
Table 6: Burgers PICore component speedup.
FNO
UNO
Coreset %
FNO
UNO
Train
Data
Warm-
Train
Data
Warmaup
Tran
Data
Warm-
Train
Data
Warm-up
20.0%
-50.2%
4-30.4%
-0.6%
+55.9%
+24.8%
-3.3%
20.0%
45.25
+74.9%%
-0.1%
+12.7%
+67.5%
-0.8%
30.0%
+44 29%
+26 6%%
-0.5%
+19.2%
+21.7%
-3.39
30.0%
+4.35%
+00.59
-0.15
+11.2%
+59.07
-0.8%
40.0%
38.4%
422..9%
-1.2%%
142.7%
18.6%
-3.3%
10.0%
+3.9%
+56.2%
-0.1%
+9.7%
+50.00
-0.5%
60.03
-26.7% 796
+15.2% 25%
-1.8%
+29.7%
+12.4%
-3 39%
60.0%%
+2.7%
+37.49%
-0.2%
+6.7%
+33.7%
-0.8%
40.0%
14.9%
17.6%
-2.4%
416.5%
46.2%
-3.3%
80.0%
11.35
118.7%
-0.2%
13.7%
116.9%
-0.8%
Table 7: Darcy PICore component speedup.
Table 8: Navier Stokes PICore component speedup.
7 Limitations and Future Work
A key limitation of PICore its reliance on a differentiable and known PDE to compute the physics- informed loss, which may not always be available. Practitioners can mitigate this by estimating the governing equations analytically using domain knowledge and trustworthy auxiliary models, or numerically through data-driven surrogates or weak-form formulations that approximate the PDE and its derivatives. Another limitation is PICore's dependence on existing coreset selection algorithms such as CRAIG and AdaCore, which were designed under convexity assumptions and use gradient or Hessian approximations. Their behavior on highly non-convex PDE learning landscapes is not well understood and can degrade performance, especially at low selection ratios. For instance, AdaCore with Hutchinson Hessian estimates applied only to the final layer consistently performs worse than alternative methods. In practice, we recommend GradMatch, EL2N, or graNd, which rely on fewer model-specific assumptions. Future work includes developing coresct selection methods tailored to neural operators by exploiting their inductive biases, and extending PICore to more general geometries to improve generalization while maintaining data efficiency.
8 Conclusion
In this work, we introduced PICore, a physics-informed unsupervised coresct selection framework designed to enhance the data efficiency of neural operator training. By leveraging the physics-informed loss to identify the most informative samples without requiring labeled data, PICore significantly reduces both the compu- tational cost of numerical simulations and the time required for training. Our experiments across four PDE benchnarks demonstrate that PICore achieves competitive accuracy while reducing training costs by up to 78% compared to supervised coreset selection methods. Although PICore inherits some limitations from existing selection methods, we believe its ability to reduce labeling costs and accelerate training makes it a promising tool for large-scale scientific machine learning.
9 Acknowledgments
Satheesh and Balan have been supported in part by the National Science Foundation under grant DMS- 2510216. Satheesh is also partially supported by Daniel Sweet Undergraduate Research Fellowship ($2500). We thank Saptarashmi Bandyopadlyay and Stephen Casper for helpful feedback.
References
Pankaj K Agarwal, Sariel Har-Peled, Kasturi R Varadarajan, et al. Geometric approximation via coresets. Combinatorial and computational geometry, 52(11:1-30, 2005.
12

---

Published in Transactions on Machine Learning Research (11/2025)
Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds, 2020. URL httts:///axxiii wttp:///////b....../. /ttp::///rxx..oorraa////00.. 03671
William H. Beluch, Tim Genewein, Andreas Nurnberger, and Jan M. Kohler. The power of ensembles for active learning in image classification. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1368-9377, 2018. doi: 10.1109 /ttp////P........
Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. Spherical fourier neural operators: Learning stable dynamics the sphere. In International conference on machine learning, pp. 2806-2823 PMLR, 2023.
Xiaofeng Cao and Ivor W. Tsang. Shattering distribution for active learning. IEEE Transactions on Neural Networks and Learning Systems, 33(1):215-225 228, 2022. doi: 10.1109/TNNIIL 2N0.1...00..307766...
Wuyang Chen, Jialin Song, Pu Ren, Shashank Subramanian, Dmitriy Morozov, and Michael W Mahoney, Data-officient operator lcarning via unsupervisod pretraining and in-context learning. Advances Neural Information Processing Systems, 37:6213 6245, 2024.
Yutian Chen, Max Welling, and Alex Smola. Super-samples from kernel hording. arXiv preprint arXiv:1203.3422 2012.
N.J. Cyrus, R.E. Fulton, United States. National Acronautics, Space Administration, and Langley Research Center. Accuracy Study of Finile Difference Melhods. NASA technical note. National Aeronautics and Space Administration, 1968. URL https://books google. con//bootes.dd=z/SESAAaaaaa
Kenneth Eriksson and Claes Johnson. Adaptive finite element methods for parabolic problems iv: Nonlinear problems. SIAM Journal on Numerical Analysis, 32(6):1729 1749, 1995. doi: 10.1137/0732078. URL
https://doi.. orgo 10.117777732078
Bin Gu, Zhou Zhai, Cheng Deng, and Heng Huang. Efficient active learning by querying discriminative and representative samples and fully exploiting unlabeled data. IEEE Transactions Neural Nelworks and Learning Systems, 92(9):4111-444 14122 2021. doi: 10.1 10.1109/TNNLS.
Chengeheng Guo, Bo Zhao, and Yanbing Bai. Deepcore: A comprehensive library for coreset selection in deep learning. In International Conference Database and Expert Systems Applications, pp. 181-195. Springer, 2022.
Animesh Gupta, Irtiza Hasan, Dilip K Prasad, and Deepak к Gupta. Data-efficient training of cnns and transformers with coresets: A stability perspective. arXiv preprint arXim:2303.0200 2023.
Guy Hacohen, Avihu Dekel, and Daphna Weinshall. Active learning on a budget: Opposite strategies suit high and low budgets. CoRR, abs/ 22s/22002,2794, 2022. URL https://arrrt wttp:oa//2/22..00000
Daniel Haimovich, Dima Karamshuk, Fridolin Linder, Niek Tax, and Milan Vojnovic. On the convergence of loss and uncertainty- based active learning algorithms. Advances in Neural Information Processing Systems, 37:122770- 11222777112210,, 2024.
AmirPouya Hemmasian and Amir Barati Farimani. Pretraining a neural operator in lower dimensions. ar preprint arXio:4407.16166 2024.
Neil Houlsby, Ferene Huszár, Zoubin Chahramani, and Máté Lengvel. Bayesian active learning for classifi- cation and preference lcarning, 2011. URL https:///rxiv..rgg wtt./org//bbbb///a//ssss 1144
Claes Johnson. Numericol solution of partiol differential equations by the finite element method. Cambridge University Press, Cambridge, England, January 1988.
Krishnateja Killamsetty, Sivasubramanian Durga, Ganesh Ramakrishnan, Abir De, and Rishabh Tyer. Grad- match: Gradient matching based data subset selection for efficient deep model training. In International Conference Machine Learning, pp. 5464-54774. PMIR, 2021a.
13

---

Published in Transactions on Machine Learning Research (11/2025)
Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, and Rishabh Iyer. Glister: Gen- eralization based data subset selection for efficient and robust learning. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 35, pp. 8110 8118, 2021b.
Krishnateja Killamsetty, Xujiang Zhao, Feng Chen, and Rishabh Tyer. Retrieve: Coreset selection for efficient and robust semi-supervised learning. Advances in neural information processing systems, 34:14488-1445 2021c.
Yeachan Kim and Bonggun Shin. In defense of core-sct: A density-aware corc-sct selection for active learning. In Proceedings of the 28th ACM SIGKDD conference knowledge discovery and dala mining, pp. 804 812, 2022.
Yegon Kim, Hyunsu Kim, Gyeonghoon Ko, and Juho Lee. Active learning with selective time-step acquisition for PDEs. In Forty-second International Conference on Machine Learning, 2025. URL
https://openreviow uff/forrurridoeeefffuHuu
Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1-97, 2023.
Nikola B. Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadnnshhlii, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces, 2021. URL
Randall J. LeVeque. Finite Volume Methods for Hyperbolic Problems. Cambridge Texts in Applied Mathe- matics. Cambridge University Press, 2002.
Dongyuan Li, Zhen Wang, Yankai Chen, Renhe Jiang, Weiping Ding, and Manabu Okumura. A survey on deep active learning: Recent advances and new frontiers. IEEE Transactions on Neural Networks and Learning Systems, 2024a.
Dongyuan Li, Zhen Wang, Yankai Chen, Renhe Jiang, Weiping Ding, and Manabu Okumura. A survey on doep active lcarning: Recent advances and new frontiers. IEEE Transactions on Neural Networks and Learning Systems, 36(4):5879-5899, 2025. doi:
Shibo Li, Xin Yu, Wei Xing, Robert Kirby, Akil Narayan, and Shandian Zhe. Multi-resolution active learning fourier neural operators. In International Conference on Artificial Intelligence and Statistics, pp. 2440- 2448. PMLR, 2024b.
Zongyi Li, Nikola Kovachki, Kamyar Axizadeneshell, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010. 08895, 2020.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations, 2021a. URL
08895
Zongyi Li, Miguel Liu-Schiaffin, Nikola Kovachki, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhat- tacharya, Andrew Stuart, and Anima Anandkumar. Learning dissipative dynamics in chaotic systems. arXiv preprinl XrXim:2106.0666 068988 2021b.
Zongyi Li, Miguel Liu-Schiaffini, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Learning chaotic dynamics in dissipa- tive systems. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Newral Information Processing Systems, 2022. URL https: http://wwwwweeeeeeeerrrrrr/aaaae
Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzade nesheli, and Anima Anandkumar. Plysics-informed neural operator for learning partial differential equa- tions. ACM/JMS Journal of Data Science, 1(3):1 2024c.
14

---

Published in Transactions on Machine Learning Research (11/2025)
Shang Liu and Xiaocheng Li. Understanding uncertainty sampling. arXiv preprint arXit:2307.02713 2023.
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via. deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218 229, March 2021. ISSN 2522-5839. doi: 10.108//42256-001-0000222 URL http: 1008/00384440 0088//4425668888 023-00
Baharan Mirzasolciman, Jeff Bilmes, and Jure Leskovec. Coresets for data-efficient training of machine learning models. In International Conference Machine Learning, pp. 6950-69009 (9000 PMT.R, 2020.
Daniel Musekamp; Marimuthu Kalimuthu, David Holzmüller, Makoto Takamoto, and Mathias Niepert. Active learning for neural PDE solvers. In The Thirteenth International Conference on Learning Repre- sentations, 2025. URL https://openreview httprriwwweeeeeoooeeeeee attt//ooruum.deex422aaaaunn
Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Avizwddeeesheli Pedram Hassanzadeh, Karthik Kashinath, and Animashree Anandkumar. Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators. arXiv preprint arXiv:2202.1120 11214, 2022.
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 20596- 20607. Curran Associates, Inc., 2021. URL htttps://proceeddnng http://ng...euri...cc/.pppe http://pper.ffleepppprrrrrr 200////n|/e5ffff0000038861111111
pdf
Omead Pooladzandi, David Davini, and Baharan Mirzasoleiman. Adaptive second order coresets for data- efficient machine learning. In International Conference on Machine Learning, pp. 17769. PML.R, 2022.
Md Ashiqur Rahman, Zachary E Ross, and Kamyar Azizzadenesheli. U-NO: U-shaped neural operators. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https::/openreviiee net/
forum?id==3oQf9cco..
Amir Hossein Rahmati, Mingzhou Fan, Ruida Zhou, Nathan M. Urban, Byung-Jun Yoon, and Xiaoning Qian. Understanding uncortainty-ban basod active learning under model mismatch, 2024. URL https://
www.:orggaaaaaaaa 13690
Bogdan Raonic, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca. Bartolucci, Rima Alaifari, Siddhartha Mishra, and Emmanuel de Bézenac. Convolutional neural operators for robust and accurate learning of pdcs. Advances in Newral Information Processing Systems, 36:77187-77200, 2023.
Ozan Sener and Silvio Savaresc. Active learning for convolutional neural networks: A corc-sct approach. arXiv preprint arXic: 00489, 2017.
Jae-hun Shim, Kyeongbo Kong, and Suk-Ju Kang. Core-set sampling for efficient neural archilecture search. arXiv preprint arXio:2107.00869, 2021.
Samarth Sinha, Han Zhang, Anirudh Goyal, Yoshua Bengio, Hugo Larochelle, and Augustus Odena. Small- gan: Speeding up gan training using corc-sets. In International Conference on Machine Learning, pp. 9005 9015. PMLR, 2020.
Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Dan MacKinlay, Francesco Alesiani, Dirk Pflüger, and Mathias Niepert. PDEBench: An Extensive Benchmark for Scientific Machine Learning. In 36th Con- ference on Neural Information Processing Systems (NewrIPS 2022) Track on Datasets and Benchmarks, 2022. URL httts://arxiv... org////w.orr/aeooooo abs/ abs/2210...... 22100011202022222222222
Hewei Tang, Qingkai Kong, and Joseph Morris. Multi-fidelity fourier neural operator for fast modeling of large-scale geological carbon storage. Journal of Hydrology, 629: 629:130641, 2024.
15

---

Published in Transactions on Machine Learning Research (11/2025)
Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. arXiv preprint or Xri:22111, 11.3002,,,, 2021.
Kai Wei, Rishabh lyer, and Jeff Bilmes. Submodularity in data subset selection and active learning. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 1954 1963, Lille, France, 07 09 Jul 2015. PMLR. URL https:://proceedings www.mmlr....eee.... wwwpss//3/wweiiieeeeeeeeee html
Max Welling. Herding dynamical weights to learn. In Proceedings of the 26th Annual International Conference on Machine Leurning, ICML '09, pp. 1121 1128, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605585161. doi: 10.1145/1533374.655559
URL lattps://don http:///oooorgg...1111 org//0..144/1
1553374 15535177
Chenxi Wu, Min Zhu, Qinyang Tan, Yadhu Kartha, and Lu Lu. A comprehensive study of non-adaptive and residual- based adaptive sampling for daysics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 403:115671, 2023.
Yazhou Yang and Marco Loog. To actively initialize active learning. Pattern Recognition, 131:108836, 2022.
ISSN 0031-3203. doi: httpss//doo.orr//100.. 10tp://00.0001111111 www.////../...ooo.....oo.o 2022.1. URL https://www.sciemeeeiiin
Zhewei Yao, Peng Xu, Farbod Roosta and Michael W. Mahoney. Inexact non-convex newton-type
methods, 2018. URL https://arxxv.org//aaa wttpgg//bb//10....000 06925
Jaehong Yoon, Divyam Madaan, Eunho Yang, and Sung Ju Hwang. Online coreset selection for rehearsal- based continual learning. arXiv preprint arXiv:2106. 100.:0006 2021.
Xiaoyu Zhang, Juan Zhai, Shiqing Ma, Chao Shen, Tianlin Li, Weipeng Jiang, and Yang Liu. STAFF: Speculative coreset selection for task-specific fine-tuning, In The Thirleenth International Conference on Learning Represenlations, 2025. URL httpp::/ http://pwreereeeee.ee/r/rrrdddda
Guang Zhao, Edward Dougherty, Byung-Jun Yoon, Francis Alexander, and Xiaoning Qian. Efficient active learning for gaussian process classification by error reduction. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, vol- ume 34, pp. 7734-9748 Curran Associates, Inc., 2021. URL /ttp///tps/// http:/ccceennnn..unuuu.sss.. neurips.cc/paper hhtt/p/errrp/////////fffaacedeee
A PDE Datasets
For our experiments, we use several differential equation training sets to evaluate our algorithm. Fach of these is used at input grid resolution of 64. For the Advection, Burgers, and Darcy Flow equations, we generate datasets using code provided by Takamoto et al. (2022). For the Navier-Sookes Incompressible equation dataset, we generate data from Li et al. (2020).
A.1 Advection
We construct our dataset by mumerically solving the linear advection equation on the periodic domain (0, 1): 0,((1, + x), 0, L (0, x E (0, 1),
(13)
The initial condition is defined as a superposition of sinusoidal modes,
27T4
uo( Σ A sin (kyar + (t), k
(14)
L
where each n, drawn uniformly from the range of integers from 8, N the number of waves, and the amplitudes A, E [0,1] and phases o, E (0, 2x) are chosen at random. After assembly of uo(z), we apply with 10% probability each pointwise absolute-value operation multiplication by smooth window function.
16

---

Published in Transactions on Machine Learning Research (11/2025)
A.2 Burger's Equation
We are interested in the one-dimensional viscous Burgers equation on the unit interval with periodic boundary conditions:
+z n2 (1,))) = (t) * E (0, 1), E (0,
(15)
subject to the initial condition
u(0,x) = uo(x), E (0,
(16)
Here y > is a constant diffusion coefficient.
We use the nondimensional Reynolds number
R= TUL
R
where un characteristic velocity scale. In analogy with the Navier Stokes equations, R indicates regime dominated by nonlinear steepening and potential shock formation, whereas R corresponds to diffusion-dominated smooth dynamics.
A.3 Darcy Flow
We obtain the steady-state solution of Darcy's equation the unit square by evolving time-dependent problem until convergence. The larget elliptic problem is
(a(œ) Vu(x)) = f(x). u(x) = 0,
x E (0, 1)? x E 0(0, 1)?
(17)
(18)
where a(x) the spatially varying coefficient and f(x) 3 constant forcing that scales the solution amplitude. Rather than solving equation 17 integrate the parabolic problem
Oeu(x, 6) V (a(x) Vu(I, ()) = 3, x E (0, 8>
(19)
with an appropriate random-field initial condition and homogeneous Dirichlet boundary data. We use the strong form V VaVu) f for the residual as in Li ct al. (202cc).
A.4 Navier-Stokes Equation
We consider the vorticity formulation the periodic domain (0,1)² 1)2:
Oew + u. Vu AE= v-u=0, w(x,0) ~ N(, (
with forcing
f(x) 0.1 [in (m( + x2) + cos (m + x22))
The solution is obtained on a 256 x 256 grid via a Fourier pseudospectral scheme: first, we solve Ae = - in Fourier space to recover the stream function & and velocity u, then compute the nonlinear advection term u Vw in physical space with 2/3-dcaliasing filter, and finally advance in time using Cronk-Nicolson for diffusion coupled with an explicit update for the nonlinear term.
B Coreset Selection Algorithms
In this section, we provide an overview of the coresct selection algorithms used. All implementations are our own, but based on Guo et al. 20022..
17

---

Published in Transactions on Machine Learning Research (11/2025)
Adacore
AdaCore augments CRATG with second curvature that difficull,, high high-influence samples are favoured even when order gradients look similar. In practice estimate only the diagonal of the Hessian with 10 Hutchinson probes per mini-batch, then pre-condition the last-layer gradient V4; by element-wise division. Similarities are computed on these pre-conditioned vectors and the same stochastic greieeggreeyyyyy routine as CRAIG is applied. The extra cost is the time to compute the approximation by deriving multiplications of the Hessian and arbitrary vectors via the Hessian-Free method (Yao et al, 2018), the time of Hutchinson' 's method to find the diagonal, and the time to apply the diagonal to the gradients of the last layer.
EL2N
Our EL2N (Error L2-Norm) coreset selection method follows from the premise that samples that are most worthwhile for the model have the highest losses. EL2N conducts a full training pass, where for each minibatch 2,, calculate the loss without reduction for each individual sample, and calculate the norm for ay's is loss vector. At the end of the epoch, we take the top & minibatches by loss norm and return them with equal weight.
CRAIG
CRAIG (Coresets for Accelerating Incremental Gradient-descent' selects weighted subset of size k whose gradients cover (i.e. represent) all per-example gradients. Let di Vol,(0) E R2 be the gradient for example i. CRAIG finds a near optimal solution to the following problem.
A = arg min (S),
Σ min max 11gm Im
ACV
mES
NEV
so every Is is "covered" by its most similar selected gradient. CRAIG selects the smallest subset such that every example gradient is close (in L2) to at least one gradient in S. We approximate the coverage objective with the stochastic-greedy algorithı applied to the pairwise Euclidean similarity matrix of last- layer gradients. Greedy (or crochastic-greddy: selection gives a evaluations. After S is chosen, CRAIG sets integer weights
approximttii in finite similarity
7 = k arg max Sim = j}{, j je S, mES
so the weighted coreset gradient Ejes ?59r closely matches the full gradient Σ at each optimisation step. In practice the method is applied to last-layer gradients to reduce dimonsionality without degrading the approximation quality.
GradMatch
Let the last-layer per-cxample gradicnts be concatenated as A [g 92 ...l E Rdxx and define the full- batch gradient b = 1.2 Σi Ps. GRADMATCH casts coreset selection as the sparse approximation problem.
min ZER
||A - bll s.t. || < k, x 2 0.
OMP builds the weight vector æ greedily. Starting with residual r = and empty support S: (i) choose the column j* arg maxjes AJ rr; (ii) add j* to S; (iii) refit the coefficients by non-negative least squares ss = arg min Asx - bl + \||=[| (iv) update r = b AsT5. The loop terminates after & selections, giving a coreset S = supp(z) with weights 7/ = Ij.
During training we replace the full loss by the weighted loss EJes / Yj. ensuring the mini-batch gradient of the coreset closely follows the full-batch gradient throughout optimisation.
18

---

Published in Transactions on Machine Learning Research (11/2025)
GraNd
GraNd is similar to EL2N, but simply orders samples by the norm of their individual gradients and keeps the top k. We piggy-back on the same per-sample gradient collection already needed for CRAIG/ CRAIG/GradMattoe Graddatchh but stop after the first backward call. We can rapidly sort these norms on the CPU, and use the selected indices for our coreset.
C Further Results
C.1 Unsupervised Selection Methods Comparison
609
600
0134
ECC
3435
EC0
659
182-006
100.05
637
630
012
011
012
/1F 1
8
Figure 2: (20%~100%) between unsupervised and PICore-basee coreset selection methods using both FNO and UNO architectures.
Test NRMSE the Advection dataset at resolution 64 across varying coreset percentages
112
1400
165
0,0
8.00
601
1000
245
0.00
530
111 181
Figure 3: Test NRMSE on the Burgers dataset at resolution 64 across varying coreset percentages (20%-100%) between unsupervised and PICore-basse coreset selection methods using both FNO and UNO architectures.
19

---

Published in Transactions on Machine Learning Research (11/2025)
230
038
C00
COT
0379
325
500
344
0
020
98
111
118
181
188
Figure 4: Test NRMSE on the Darcy dataset at resolution 64 across varying coreset percentages (20% 100%) between unsupervised and PICore-base based coreset selection methods using both FNO and UNO architectures.
005
B00
0.50
600%
CO
118
/18
111
118
Figure 5: Test NRMSE the Navier Stokes Incompressible dataset at resolution 64 across varying coreset percentages (20% 100%) between unsupervised and PICore-based based coreset selection methods using both FNO and UNO architectures.
20

---

Published in Transactions on Machine Learning Research (11/2025)
C.2 Spatial Orientation of Supervised Coreset Selection and PICore
To better understand the differences between supervised coreset selection and PICore, analyze how well each method covers the input space by computing the average distance from coreset points to their centroid, which serves as a proxy for spread or diversity. We compute this distance with respect to the || norm, where the centroid is the average data point element-wiss wise and the average distance is the average norm between the centroid and the selected data points in the coreset. As shown Figure this distance is nearly identical across datasets and neural operators (FNO and UNO), with overlapping standard crror bars with differences decreasing as the PDE complexity increases (Advection to Navier Stokes). This suggests that PICore selects coresets that as well-distributed as those from supervised methods, despite not using labeled data. The comparable coverage indicates that differences in downstream performance likely arisc from the type of points selected rather than their spatial distribution.
- [=))
(a) Advection
(b) Burgers
(c)
Darcy
(d) Navier Stokes
Figure 6: Average centroid distances across datasets for FNO and UNO.
C.3 PiCore Training Convergence
Advrtee
Derty
Tacder Stokes
Epern
Figure 7: Training convergence of coreset selection v.s. active learning using FNO at 20% selection ratio.
Sagss
Dats
Saric States
Figure 8: Training convergence of coreset selection v.s. active learning using UNO at 20% selection ratio.
21

---

Published in Transactions on Machine Learning Research (11/2025)
C.4
Ablation Study
Table 9: Ablation on Warm Start at resolution 64
Operator
T5
20.0%
30.0%
40.0%
60.0%
80.0%
100.0%
10 epochs
1.64 1. 0.14 x 10-2
1.14 0.10 x 10-3
8.07 1: 0.4.8 x 10-3
5.32 a 0.30 x 10-3
4.17 J. 0.17 x 10-1
3.95 4 0.10 10-3 (1.00x)
(3.23x)
(2.18x)
(2.06x)
(1.52k)
(1.19x)
25 epochs
1,52 0.13 10-2 (3.18x)
1.07
0.08 10- 7.71 +-0.3 10-3
5.33 + 0.14 10-3
4.15 + 0.06 x 10-2
3.95 + 0.10 x 10-3
FNO
(2.44x)
(2.02x)
(1.48%)
(1.17x)
(1.00x)
50 epocks
1,9 0.07
10 (3.10x)
1.12
0,08 x 100 (2.36x)
0.96 x 10
5.51 # 0.17 x 10
4.16 0.11 x 10 (1.13x)
3,95 + 0.10 x
(1:95)
(1.43x)
(1.0xx
100 epocias
1 1: 0.14
10 2 1,06 0.04 (2.21x)
10
8.24 x 10
0,16 x 10
115 x 10
3.95 + 0.10 x 1D"
(2.91x)
(188x)
(1.31))
(1.05)
(1.00x)
10 epocks
2.97 1: 0.04
10 2 2.44 1: 0.06 10 2
2.13 0.04 x 10
1.77 1 0.04 x 10
1.57 = 0.01 x
1.49 i 0.04 x 10
(3.79x)
(2.79x)
(2.25x)
(1.60x)
(1.24x)
(1.00x)
25 epocls
2.99 0.04 10 2 2.51 0.04
10
2.09 1 0.02
10
1.76 1 0.03 x 10
1.57 i 0.04 x 10
1.49 1 0,04 x 1D
UNO
(3.72x)
(2.73x)
(2.19x)
(1.56x)
(1.20x)
(1.00x)
50 epochs
2.89 0.03
10
2,42
0.03 x110 10
2.18 0.05 x 10"
1.73 1 0.03 x 10
1.54 L 0.02 x 10
1.49 1 0.04 x 10
(3.60x)
(2.61x)
(2.11x)
(1.19%)
(1.15x
(1.00x)
100 epecia
3.00 0.08
10
2,53
0.05 10
2.05 1.0.4 10-2
1.73 0.03110.03 x 10
1.55 L 0.02 x 10-2
1.49 1 0.004 x 10
(3.38x)
(2.16x)
(1.96x)
(1.38x)
(1.07))
(1.00x)
The ablation study in Table D examines how different warm start epochs influence the performance of PICore on FNO and UNO. We fix the coreset selection algorithm to EL2N and the dataset to Burgers. Across warm start configurations, extending the number of epochs beyond 10 to 25 50 epochs leads to only marginal gains, and by 100 epochs, the improvements are negligible or even slightly inconsistent within the bounds of standard deviation. For FNO, the 25-epoch variant achieves the lowest errors at lower data fractions (20-40%), suggesting that moderate warm starting may yield slightly better initialization for PICore. UNO, on the other hand, shows stable performance across all pretraining lengths, implying that it benefits less from extended warm starting. Overall, the table suggests that minimal warm starting (around 10 25 epochs) is sufficient for both operator families, while additional finetuning offers little benefit.
C.5
Multi Resoltinn Coreset Selection
Table 10: Burgers NRMSE with Multi Resolution Data
Operator
Method
Algurithm
20.0%
30.0%
40.0%
60.0%
80.0%
100.0%
craig
8.29 =: 1.58 x 10
0.73
10
7.56 = 0.75 x 100
0.78 x 10
7.25
x 10
6.85 0..79 x 10"2
gradeatcch
8.33+ 1.16 x 10-2
8.10
8.10+-093.0
10
7.92 + 0.44 x 10-2
7.64
7.64+0811
10
6.94 6.94+0.55
x 10
6.85
6.55+0779 x 10-2
Supervised
w12n
adacore
9.24
7.61+00.66
10
10-2
7.34
8.73
7.50
10-2
7.55 0,00 x 10-2
8.00
x
7:30 0.0
7.41
10-2
6.94
6.89+0.00
10
10-2
6.85 0.70
6.8 0.79
10-2
gralid
8.04 10-2
1.00 10-2 7.95
7,75 0.88
x 1002
7.45
10
1.25 x 10-2
6.85 1.79
10-2
Ancelenntio
8.00=0000
1.76=000x
0.0000000x
1.60 + +.000
0.24
1.00=0:0x
PNO
craig
7,37
1.66 10
8.09
0 10
7.70 1.20
10-
7.36 +.000 10
7.31 +.0.5 10
6.85 (.779
10
gradnatcc
7.00
1.80
10-2
8.07
10-2
7:70
+ 1.19
10-2
7.54
10-2
7.16+0.55
10-2
6.85 0.70
10-2
PiCure
adacors
a12m
8.56
+ 1.78
10-2
10-2
8.14 1.36
10
10-2
7.49
8.20
- 0.81
1.54
x 10-2
10-2
7:29
7.90
1.03
0.05
10
10-2
6.78
7.34 1.0 10
10
6.85
6.80+00779
10-2
10-2
grald
8.88 1 0.63
x 10-2
8.39
1 0.57
x 10-
8.19
1 1.16
x 10-2
7.56 1 0.61
10-2
7.31 1.0.111111
x 10
6.85 1.79
x 10
Anvelennton
5.08 5.08-0.013 0.00x
3.32 - 0.00x
2.54+00.000
1.70 + 0,00x
1.27+0000
1.00 1.0000.00 0.00x
Table 10 compares Supervised and Physics-Informed Coreset Selection (PICore) across varying data resolu- tions for the Burgers' equation using the FNO operator. In this experiment, the dataset is split into two equal parts: one half is generated at resolution of 32, and the other at resolution of 64, introducing multi-scale variability in the training data. Overall, PICore demonstrates improved stability and often superior accuracy at lower data percentages, particularly in the FNO setting. For instance, under FNO with 20 60% data, PICore achieves consistently lower NRMSE values than supervised methods, especially with the el2n and craig algorithms, indicating better generalization when fewer samples are available. This advantage reflects the integration of physical constraints, which act as a strong inductive bias that helps retain key solution structures even when data is sparse or noisy. The mixedirrsolltion setup further amplifies this effect,
22

---

Published in Transactions on Machine Learning Research (11/2025)
while supervised coresets can struggle to reconcile the differences between coarse (32) and fine (64) grids, PICore leverages physics-informed consistency to align features across scales, resulting in more stable and resolutonn-ivvariang representations of the Burgers' dynamics. However, this setup also introduces greater variability and noise, as discrepancies between resolutions can create inconsistencies in gradient magnitudes and feature smoothness, but these issues are generally inherent to multi-resolution datasets, rather than specific the coresct selection approach itself.
C.6 Zero Shot Super Resolution
Since Neural Operators learn parameters independently the discretization (unlike PINNs), trained neural operators can perform zero-shot super-resolution, which allows for training model at a lower resolution and evaluating at higher resolution. We scale the Advection and Burgers datasets to resolutions of 128 and 256, the Darcy dataset to 128, and the Navier Stokes Incompressible dataset to size 256 in tables 11 16
Operator
Method
Algorithm
20.0%
30.0%
40.0%
60.0%
80.0%
100.0%
craig
7.09 10-2
8.41
10-2
8.0
10-2
8.04 x 10
7.61
10-2
10-2
gradmatch
8.38 x 10
7.87
10
7.85
x 10
7.86 10
7.73 x 10
5.90 x 10
Supervised
adacore
8.45 x
7,85
10
7.99 x 10
T.42
10
7443
x 10
5,90 x
el2n
8.34 x 10
8.05
10
8.10
10
7,88
7.67
10
5.90 10
gralid
8.23 x 10
8.05
10
7.79 x 10
7,60 x
7.26 x 10
5.90 x 10
FNO
craig
8.38 x 10
8.01
7.76 10
7.41 10
7.35 x 10
5,90 x 10
gradnatch
8.85 x 10-2
8.35 10-2
8.15 10-2
7.76 10
7.59 10
x 10
PiCare
adacore
8.82 x 10
8.95
7,57 x 10
7.28 10
8.18
7.27
10
10
7.98
7.03 x 10
10-2
5.90 x 10"2
x 5,90
gralid
8.38
8.15 10
7,86 10
7.72
7.50
10
craig
1.72 x 10
1.66 x 10
1611 10
1.48 x
1.14 10
1.09 x 10
gradmatch
x
10-1
Supervised
edacore
2.25
1.85 10
1.71 10
1,58
1.27 10
1.09 x 10
e12n
1.69 x 10-1
1.65 x 10-1
1,62 10
1.56 10-1
1.25 x 10-1
1.09 x 10-1
graldd
x
10
10
10-1
UNO
craig
1.74 x 10
1.61 10
1,00 10
1,51 10
1,26 10
1.09 x 10
gradwatch
1,70 x 10-1
1.65 x 10-1
1,62 x 10
1.49 10
1.23 x 10
1.09 x 10
adacore
PiCare
e12n
1.60 x 10
1.64 10
1.63 x 10
1,52 10
1.27 x 10
1.09 x 10
gralid
1.70 x 1.64
1.2 10
153 x 10
123
x
1.09 x
Table 11: Test NRMSE on the Advection dataset at resolution 128 across varying coreset percentages (20% 100%) between supervised and JcCore-based coreset selection methods using both FNO and UNO architectures.
23

---

Published in Transactions on Machine Learning Research (11/2025)
Operator
Method
Algorithm
20.0%
30.0%
40.0%
60.0%
80.0%
100.0%
craig
8.55 10-2
9.30 10-2
8:90 10
9:00 10-1
8.64 10
7.07 10-2
gradmatch
9.22 x 10
8.79
10
8.80 10
8.85 x 10
8.76 x 10
7.07 x 10
Supervised
adacore
x
8.43 10
8.47 10-2
x 7.07
e12n
9.18 x
8.98 10
9.04
8.85
8.67
graldd
9.09 x 10
8.95 10
8.76 10
8.59
8.31 x 10
7.07 x 10
FNO
craig
9.13 10
10
8.67 10
8.39
8,38 x 10-
7.07 x 10
gradzatch
9.68 x
9.28 10
9.12 x
8.79
8.63 x
7.07
PICore
adacore
9.40 x 10-
8.37 x 10
8:20 10
8:27 x 10
8.08 10
7.07 x 10
x
9.13 10
8.96 10-2
grali
9.23 x 10
9.06 10-2
8.84 x 10
8.77 10
8.50 10
7.07 x 10
craig
1.86 x
1.80 10
1.76
1.63
1.35 x 10
1.31 x 10
gradmatch
10
Supervised
adacore
2.30 x 10
1.95 10
183 10
1.73
1,46 10
1.31 10
e12n
1.83 x 10-1
1.80 x 10
1.78 x 10
1.71 10
144 10
1.31 10
gralid
10
10
10
UNO
craig
1.87 10
1.71 10
1.75 10
1,77 10
1,45 10
1.31 10
gradmatch
1.83 10-1
1.80 10
1.78 x 10
1.65 10
1,42 10
1.31 10
adacore
166x
10
PiCare
e12n
1.84 x 10
1.79
10-1
1.78
10
1.88 10
1.47 x 10
1.31 x
10
gralid
1.83 x 1.79 10
1.77
1.88 10
1.43
1.31
Table 12: Test NRMSE on the Advection dataset at resolution 256 across varying coreset percentages (00% between supervised and PICore-basod coresct sclection methods using both FNO and UNO architectures.
Operator
Algorithm
20.0%
30.0%
40.0%
60.0%
100.0%
craig
6,83 10
7.32 10
7.20 10
6.90 10
6.14 10
4.57
10
gradmatch
6.56 x 10
6.35 10
6.68 10
6.32 10
6.66 x 10
4.7 x 10
Supervised
8.34
10-2
x
e12c
6,88 x
7.17 10
6.74
6.43
6.88
4.57
gralid
6.72 x 10-2
6.00 x 10
6.47 x 10
6.27 x 10
6.06 x 10
4.57 x 10
FNO
craig
x 10-2
6.38
6.22 10
6.30 x 10-2
4.57 10-2
grainatch
6.40 x 10
6.62 10
6.81 1
6.47 -0
6.15 10
1.57 10
PiCore
adacore
7.39 x 10-2
6.00 10-2
6.50 10
6.38 -0
6.46 x 10-2
4.57 x 10
6.29
grald
6.71 x 10-2
6.58 10-2
10 6.91
6.50 10
6.45 10
4.57 x 10
craig
2,91 10
2.77 10
2.62 x 10
2.33 10
2.19 10
2.16 10
gradzatch
10
Supervised
edacore
4.36 x 10
3.39 10
2.98 10
2:17
10
2 10
2.16 10
e12n
2.94 x 10-2
2,76 10-2
2.62 -0
2.30 -0
2.28 10-2
2.16 10
gralid
3.01 10
10-2
10
-0
2.0 10
2.16 10
UNO
craig
3.18 10-2
2.96 10-2
2.77 x 10
2.38 x 10
2.15 10-2
2.16 10
gradmatch
2.97 x 10
2.78
10
2.89 10
2.40
10
2.77
10
2.16
adacore
4.77
3.68
10
2.49
10
2.16
PlCare
el2n
2.96 x 10
2.77
10
2.62
2.43
2.29 x
2.16 x
10
grald
2.89 x 2.72
2.66
2.49
10
2.29
2.16
Table 13: Test NRMSE on the Burgers dataset at resolution 128 across varying coreset percentages
(20% 100%) between supervised and PICore-based coreset selection methods using both FNO and UNO architectures.
24

---

Published in Transactions on Machine Learning Research (11/2025)
Operator Method
Algorithm
20.0%
30.0%
40.0%
60.0%
80.0%
100.0%
craig
7.02 10-2
7.62 10-2
7.40 10-2
7.12 10-1
6:37 10-2
10
gradmatch
6.72 x 10
6.55 10
6.89 10
6.55 10
6.00 x 10
4,89 10
Supervised
adacore
x
0.55
10
6.52 10-2
e12n
7.04 x
7.38
6.96
6.66
6.32
4,89 10
gralid
6.89 x 10
6:21 10
6.69 10
6,50
6.29 x 10
4.89 x 10
FNO
craig
6.83 x 10
10
6.39 10
6:45
6.63 10-2
1.89 10
gradzatch
6,57 x
6,83 10
7,02
6.70
6,39 x
1,89 10
PICore
adacore
7,50 x 10-
6.76 x 10
6.80 10
6.61 10
6.09 10
4.89 10
x
10
6.62
gralid
6.88 x 10-
6.78 10-2
7.12 10
6.72 10
6.09 10
4,89 10
craig
3.33 x
3.36 x 10
3.26 10
3:06 10
2.96 x 10
2.92 x
gradmatch
10
Supervised
adacore
4.70 x
3.81
3.51 10
3.17 10
3.01
2.92
e12n
3.46 x 10-2
3.33 10
3.27 10
3.11 10
3.06 10-2
2,92 10-2
gralid
10
331x
10
x
10
UNO
craig
3.63 x 10
3.17
10
3.36 10
3.11 10
2,97 10
2,92 10
gradmatch
3.50 10-2
3.35
10
3.34 10
3.14 10
3.08 10-2
2.92 10
adacore
3.12
2.6 10
PiCare
e12n
3.47 x 10
3.30
10
3.29
10
3.16
10
3.00 10
2.02 x 10
gralid
3.41 x 3.34
10
3.34
3.28
x
3.12 x
2.92 x
Table 14: Test NRMSE on the Burgers dataset at resolution 256 across varying coreset percentages
(20%-100%) between supervised and PICore-basod coresct selection methods using both FNO and UNO architectures.
Operator
Algorithm
20.0%
30.0%
40.0%
60.0%
100.0%
craig
1.67 10
1.51 10
1,36 10
1.15 10
1.02 10
8.76 10
gradmatch
1.72 x 10
1.47 10
1,36 x
1.15
1.04
8.76 x 10
Supervised
x
10
10
e12c
1.68 x
1.42 10
1
1.15 10
1
8.76
gralid
1.67 x 10-1
1.46 x 10
1.32 10
1.16 10
1.04 10
8.76 x 10
FNO
craig
1.68 x
10
1.16 10
1.03 10
8.76
grainatch
1.73 x 10
1.48 10
1.38 10
1.16 10
1.03 10
8.76 x 10
PiCore
adacore
1.98 x 10-1
1.58 10-1
1.37 10
1.10 10
1.02 10
8.76 10
10
1,03
grald
1.72 10-1
1.47 10-1
1,35 10
1.13 10
1.03 10-1
8.76 x 10
craig
1,36 10"
1.57 x 10"
1.57 100
1.48 x 10"
1.40 10"
1.72 10"
gradzatch
10
10"
10
10"
Supervised
edacore
1.31 10"
1.29 10"
1.39 10P
1.38 x 10"
1.40 10"
1.72 10"
e12n
1.61 x 10"
1.53
1.68 x 10P
1.53 10P
1.58 10"
1.72 10"
gralid
10"
1.57 10P
10"
10"
1.72 10"
UNO
craig
1.37 x 10"
1.55 10"
1.51 10P
1.59 10'
1.11 x 10"
1.72 10"
gradmatch
1.76 x 10"
1.50 x
1.56 x 108
1.53 x 10P
1.49 10"
1.72 10"
adacore
10"
10"
1.25 10P
1.30 109
1.54 10"
1.72 100
PlCare
el2n
1.67
10"
1.59 109
1,59 x 10P
1.54 x 109
1.52 x 10P
1.22 10"
grali
1.77
1.57
1.59 108
1.55 x 10P
1.32 x 10
1.72
Table 15: Test NRMSE on the Darcy dataset at resolution 128 across varying coreset percentages
(0%%))0%) between supervised and PiCore-based coreset selection methods using both FNO and UNO architectures.
25

---

Published in Transactions on Machine Learning Research (11/2025)
Operator
Algorithm
30.0%
40.0%
craig
2.13 x 10
7.81
10-2
7.24 10
7,33
10
7.05 10-2
8.54 10-2
gradzatch
1.57 x 10
7,36 10
7.24
10
7.56
10
7.49
10
8.54
Supervised
adacore
3.18 x
6.16
10-2
6.56
10
6.48
10-2
8.64
e12n
1.43 10
7.76
10-2
7.38
10
7,93
10
7.84
10
8.54 x 10
grald
1.67 x 10
7.42 10-2
6.78 x 10
7.31
10
6.98 x 10
8.54 10-2
FNO
craig
x
10
10
6.75
10-2
8.54 10
gradmatch
1.70
7,83 10
7.36
10
7.81
10
7,76
10
8.54
adacore
2.84 x 10-1
1.04 10
1 6.38
8.38
10
6.90
10
8.54 10
PICore
7.61
grald
1.53 x 10
7,83 10
7.57
10
7.53
10
10 7,75
8.54 10
craig
5.99 x 10-2
6.79 x 10-2
5.64 x 10
5.56 10
5.52
10
5.16 10-2
gradmatch
10
5.53
Supervised
adacore
6.45 x 10-2
5.98 10-2
5.77 10
6.58 10
6.49
10
5.16 10
el2n
5.93 x 10
5.77 x
5.68 10
5.57 x 10
5.52 10
5.16 10
gralid
UNO
craig
6.07 x 10
5.71 10
5.66
10
5.53 10
5.51 x 10
5.16 x 10
gradmatch
5.97 x
5.76
5.67
5.60
5.55 x
5.16 x
PiCare
adacore
10
5.81
10
5.57
5,50 10
10
e12n
5.96 x 10-2
5.76 10-2
5.68
10
5.58
10
5.03 x 10-2
5.16 x 10
grald
5.96 x
5.77
10
3.63
10
5.58
5.54
10
5.16 x
Table 16: Test NRMSE on the Navier Stokes Incompressible dataset at resolution 256 across varying coreset percentages 200%100%) between supervised and PTCore-based coreset selection methods using both FNO and UNO architectures.
26


Reviews:
[{"review_id": "GEZ05KxtZ4", "reviewer": "Reviewer_ymLV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces PICore, a physics-informed coreset selection framework for training neural operators more efficiently. Instead of requiring all simulation labels up front, PICore uses a PDE residual loss to identify informative samples, reducing both training and labeling costs. It is architecture-agnostic and compatible with multiple coreset algorithms. Experiments on four PDE benchmarks with FNO and UNO show that PICore matches supervised coreset selection in accuracy while reducing total training cost by up to 78% and speeding up training by 5×.", "claims_and_evidence_yn": "Yes", "claims_explanation": "## Strengths:\n\n- Novel combination of physics-informed modeling with coreset selection.\n- General and modular framework, demonstrated across algorithms and architectures.\n- Thorough experiments with clear efficiency and accuracy trade-offs.\n- Strong practical relevance for reducing simulation and compute costs.\n\n## Weaknesses: \n- Dependence on Known Physics: PICore assumes the PDE’s form is known to compute the physics-informed loss. This is a reasonable assumption in scientific problems, but it means the method is less applicable if the governing equations are unknown or too complex to differentiate.\n- Warm-Start Requirement: The method requires a brief unsupervised “warm-up” training of the neural operator (using the physics loss alone) to initialize the model before selection. This adds a small overhead and relies on the model being able to produce a meaningful PDE residual field for each sample. If the warm-start model were very inaccurate, the residuals might not reliably indicate useful samples. IMO the authors mitigate this by using a short warm-up (25 epochs) which seemed sufficient, but this choice might require tuning for different problems. \n- Accuracy Trade-off: Like all coreset methods, PICore inevitably sacrifices some absolute accuracy when using a much-reduced dataset. The authors acknowledge this trade-off. It’s not a flaw per se (just the cost of using less data), but readers should be aware that maximum accuracy is attained only with full data; PICore helps find a sweet spot where large training speedups are obtained at the expense of a small accuracy drop. \n- Inherited Limitations of Coreset Algorithms: PICore’s effectiveness can be capped by the underlying selection algorithm. The paper notes that some methods (e.g. AdaCore with a Hessian approximation) performed suboptimally in this context. PICore itself does not fix fundamental issues of those algorithms; it rather provides a new criterion (physics-loss) for them. This is not a critical flaw, but it suggests that designing selection algorithms specifically for operator learning could further improve results (as the authors mention). \n- Scope of Experiments: While the evaluation is strong, it is focused on uniform grid geometries and a single resolution for each PDE. The paper does not explore multi-resolution scenarios or varying domain geometries. Extending PICore to handle multi-fidelity data or complex geometries (which neural operators are in principle capable of) is left to future work. This is a reasonable limitation for an initial study, but it does slightly narrow the demonstrated scope of the method.", "audience_yn": "Yes", "audience_explanation": "Yes. This work will interest TMLR readers in data-efficient ML, physics-informed learning, and neural PDE solvers. It contributes a novel, general methodology that bridges ML and scientific computing, with practical gains in efficiency and environmental impact. The approach could inspire similar strategies in other domains and is highly relevant to ongoing work in scientific ML.", "requested_changes": "- (Critical) Clarify the implementation of the Active Learning baseline. The paper should more clearly describe how the active learning comparison was conducted in the experiments. Active learning is typically iterative, so it’s important to explain: Did you perform multiple rounds of selection and model retraining, or did you adapt it to a one-shot selection (using uncertainty from a warm-up model)? For example, the text hints at a “loss-as-uncertainty” strategy for active learning – please clarify if the model was first partially trained on a small subset or if the physics-informed warm-up model’s loss was used as a proxy for uncertainty. Currently, readers might be confused how exactly the 20% (or 40%, etc.) of data were chosen in the AL baseline and why it underperforms. A brief methodological note in the experiments section or appendix would resolve this and ensure the active learning baseline is fair and well-understood.\n\n- (Critical) Discuss the assumption of known PDEs and potential limitations. PICore assumes the governing PDE equations are known and differentiable (to compute the physics-informed residual). This is a reasonable assumption for the benchmarks used, but it does limit applicability – for instance, if one only has black-box data or experimental data without a known physics model, PICore wouldn’t directly work. It would strengthen the paper to include a short discussion of this point, perhaps in the conclusion or impact statement. Acknowledge that the method relies on prior knowledge of the physics, and maybe suggest that in domains without known equations, alternative unsupervised criteria would be needed. This will set the appropriate scope for readers and position PICore properly with respect to purely data-driven approaches. \n\n- Comment on Figure 2:\nThis figure is difficult to interpret since all bars look nearly identical, making it hard to see meaningful differences. I suggest either removing it and summarizing the result in text, or re-plotting as differences from the baseline so small variations become visible. In its current form, it adds little clarity.", "broader_impact_concerns": "No major ethical risks. Positive impacts include reduced compute and labeling costs, lowering barriers and carbon footprint.", "additional_comments": "Encouraged to release code for reproducibility!", "extra_scores": {}, "extra_text": {}}, {"review_id": "xE9ZEUNbHc", "reviewer": "Reviewer_xg5b", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "### Summary\n\nThe paper introduces a framework named PICore designed to address two critical bottlenecks in training neural operators for scientific problems: the immense amount of training data required and the high computational cost of generating that labeled data via numerical simulations.  The primary contribution is a novel, unsupervised method for selecting the most valuable training data. Instead of first running expensive simulations to label a massive dataset, PICore uses a \"physics-informed loss\" to score and identify the most informative examples from a large pool of unlabeled data. This loss metric essentially measures how much a model's prediction violates the known physical laws of the system, without needing the ground-truth answer.  Only this small, high-value subset—the \"coreset\"—is then simulated to generate labels. This approach simultaneously slashes the data annotation cost and reduces the final model training time.\n\n### Strengths\n\n- The core idea of using a physics-informed loss as a proxy for sample importance addresses a bottleneck that limits the application of neural operators in many scientific and engineering fields.\n\n### Weaknesses\n\n- The evaluation is restricted to only two neural operator architectures, FNO and UNO. While the authors claim the framework is model-agnostic, its effectiveness has not been demonstrated on other prominent architectures like DeepONet, Graph Neural Operators, or Convolutional Neural Operators, each with distinct properties that might interact differently with the selection process.\n\n- The experiments were conducted using a single input resolution and on uniform geometries. The paper acknowledges that its performance on multi-resolution data or problems with more complex, arbitrary geometries has not yet been evaluated. \n\n- The paper focuses on time savings from reduced simulations but does not discuss the potential training challenges introduced by the physics-informed loss. Such losses can create difficult optimization problems , but the paper does not report on training stability or conduct detailed ablation studies on key parameters, which were chosen arbitrarily.", "claims_and_evidence_yn": "No", "claims_explanation": "A primary concern with the submission revolves around its perceived novelty. While the constituent components—the use of a physics-informed loss and coreset selection algorithms—are indeed well-established schemes in their respective fields, the paper's originality should be viewed through the lens of their unique synthesis. The central contribution is the innovative application of the physics-informed loss as a label-free proxy for sample informativeness, which enables an entirely unsupervised coreset selection process. This specific methodology is non-trivial and directly addresses the dual challenges of expensive data annotation and prolonged training time. This synthesis helps to define the paper's position at the intersection of data-efficient learning and scientific computing.\n\nHowever, this positioning is significantly weakened by the absence of a critical baseline. The analysis is missing a comparison against a neural operator trained exclusively with a physics-informed loss on the full unlabeled dataset, a methodology similar to that of the Physics-Informed Neural Operator (PINO). Such a model would represent a \"zero-simulation-cost\" baseline and is essential for a complete evaluation. Without this comparison, it is difficult to ascertain whether the proposed strategy—expending a small simulation budget on a targeted coreset—is more effective than a purely physics-driven approach that requires no simulation budget at all. The omission of this key baseline makes it challenging to fully validate the paper's claims, suggesting that the motivation, methodology, and overall positioning of the work need to be reconsidered and strengthened.", "audience_yn": "Yes", "audience_explanation": "Particularly, those focused on scientific machine learning and data efficiency would be interested in these findings. The paper offers a novel, practical framework for training neural operators that directly tackles the critical and widespread challenges of expensive data simulation and training time.", "requested_changes": "- The paper's primary claim is that it provides an efficient solution by using a small, targeted simulation budget. A critical missing baseline is a direct comparison against a neural operator trained exclusively with a physics-informed loss on the full unlabeled dataset (i.e., a PINO-like model). This would represent a \"zero-simulation-cost\" baseline, and its inclusion is essential to properly contextualize the trade-offs of the PICore framework and to fully validate the paper's positioning and contributions.\n\n- The submission claims the PICore framework is model-agnostic, but the evaluation is limited to FNO and UNO architectures. To substantially strengthen this claim, the work would benefit from including experiments with other prominent neural operator backbones, such as DeepONet, Graph Neural Operators, or Convolutional Neural Operators, which have different architectural properties.\n\n- The paper focuses on the time saved from reduced simulations but does not fully address the potential increase in training complexity that physics-informed losses can introduce. The work would be better positioned with a more detailed analysis of training stability and convergence. Furthermore, key hyperparameters, such as the number of warm-up epochs, were chosen arbitrarily ; conducting ablation studies on these choices would significantly improve the paper's scientific rigor.\n\n- The authors acknowledge that the current evaluation is confined to a single input resolution and uniform geometries. The paper's claims of generality would be much more convincing if the framework's performance was also demonstrated on problems involving multi-resolution data or more complex, arbitrary geometries, which are common in real-world scientific applications.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "B4GBkGQWa8", "reviewer": "Reviewer_cnne", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper examines coreset selection strategies in the context of neural operators for PDEs. The proposed method, PICore, uses the PDE / PINO loss as the sampling criterion, which allows the method to select inputs without having access to the true labels. The model is warm-started with the PINO-style loss first, and the data is selected in a single shot. The method is combined with five different coreset methods and evaluated on the Advection, Burgers, Darcy, and Navier-Stokes test cases. \n\nStrengths:\n* Extensive investigation of the PINO loss as a sampling criterion. \n* The paper is clearly written and easy to understand.\n\nWeaknesses:\n* Claims are not sufficiently supported by the experiments (see next section), e.g.\n    * Crucial experimental details are missing.\n    * The experiments miss important baselines (random, active learning)\n* Similar data selection techniques in the context of PDEs could be discussed more, e.g., active learning or adaptive sampling for PINNs [1] or other recent papers in the context of neural PDE solvers (e.g., [2]).\n* The paper criticises active learning as leading to slow convergence. However, it is unclear to me if the presented method can still be considered a coreset technique or is actually a pool-based active learning strategy since it tries to reduce labeling effort too. The training speed is also reduced by active learning techniques, after all (as it uses a smaller dataset too).\n\n[1] Wu, C., Zhu, M., Tan, Q., Kartha, Y., & Lu, L. (2023). A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 403, 115671.  \n[2] Kim, Y., Kim, H., Ko, G., & Lee, J. Active Learning with Selective Time-Step Acquisition for PDEs. In Forty-second International Conference on Machine Learning.", "claims_and_evidence_yn": "No", "claims_explanation": "* The main research question was motivated by the stated bad convergence speed of active learning. However, active learning techniques were not tested as a baseline. For example, one could also use only a small subset as an initial training data set, and then sample the rest of the desired training set size with a single iteration of an active learning algorithm. \n* Random selection of x% of the data should be the main baseline, but it is missing in the paper. Right now, it is not possible to evaluate whether the nRMSEs measured at different amounts of training data are better or worse than selecting a random subset of the training data. \n* Missing experimental details (on PDEs, training, model), for example: \n    * Number of total trajectories (train/test)\n    * Number of time steps  \n    * Physical coefficients (e.g. $\\beta$ for Advection, diffusion coefficent for Burgers, ...)\n* The acceleration is unclear to me:\n    *  How exactly was it measured? How much is saved during training, how much by using fewer simulations, and how much time is added by the selection and warm start? \n    *  How can it reduce the time by more than 5 times for selecting 20% of the data? With 5 times fewer data points, we should theoretically save 5 times the number of gradient steps during training (since the number of epochs is fixed) and also 5 times the simulation effort (which I assume is constant per input). Is this due to data loader issues or different batch sizes?\n* No reporting of the confidence intervals or standard errors, even when the results were repeated over multiple seeds.", "audience_yn": "Yes", "audience_explanation": "Yes, reducing training and data selection times would be a valuable contribution to the field of neural PDE surrogates. The proposed method is a novel approach for neural operators. Additionally, traditional coreset selection strategies have been evaluated for neural Operators for the first time.", "requested_changes": "Nesseccary:\n* Address the experimental concerns mentioned above.\n\nMinor:\n* I think that at least a short explanation of the applied coreset selection techniques should be part of the main paper.\n* The sheer amount of tables and numbers in the main paper is overwhelming. I think the paper would profit from a graphical representation, condensed summary statistics, and/or from only using a single coreset algorithm for either supervised or PICore. The main paper lists PICore results for each PDE twice, in comparison to the supervised techniques as well as to the unsupervised ones. \n* Bold font to highlight the best results in all tables \n* The evaluation of the centroid distances is unclear: which centroids are you referring to? How is the distance measured?", "broader_impact_concerns": "None.", "additional_comments": "* In the introduction, convergence speed was mentioned as a drawback of active learning. Did you also take a look at the convergence speed in your experiments?\n* Did you also tune the batch size for the simulator in order to be fair in terms of the evaluation of the simulation effort?", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)