The Tutor-Pupil Augmentation: Enhancing Learning and Interpretability via Input Corrections
Authors: Darya Biparva
Maarten Schoukens
Donatello Materassi
Venue: ICLR 2026 Poster (2026)
Keywords: model augmentation
machine learning for physical sciences
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: State-of-the-art machine learning models often incorporate prior knowledge or structural information about the task or data distribution. In some tasks, such knowledge may arise from first principles or emerge as simplified, learned functions that distill essential aspects of the data distribution. Model augmentation has emerged as a strategy to leverage this structured knowledge by coupling it with an auxiliary model to improve predictive performance, while preserving the interpretability offered by the simpler component. In this work, we present a new augmentation framework called the Tutor-Pupil scheme, which is designed to enhance both performance and interpretability. The Pupil is a fixed model, structurally designed for the core task, while the Tutor is a more flexible model trained to apply minimal input-level corrections to improve the Pupil’s performance on the modified input. This strict separation of roles enables the Tutor not only to compensate for the Pupil’s limitations but also to act as a diagnostic instrument. By examining the Tutor’s targeted interventions, we can identify failure modes, detect regions where the Pupil struggles to generalize, and uncover residual patterns or higher-order structures in the data not captured by the original model.

Published as a conference paper at ICLR 2026
THE TUTOR-PUPIL AUGMENTATION: ENHANCING LEARNING AND INTERPRETABILITY VIA INPUT CORRECTIONS
Darya Biparva
Maarten Schoukens
Department of Electrical and Computer Engineering University of Minnesota Minneapolis, MN 55455
Department of Electrical Engineering
Eindhoven University Technology
Eindhoven, Netherlands m.schoukenssturs
bipar001 www.:001mmmm.mmmmmmmmmmmmmmm edu
Donatello Materassi Department of Electrical and Computer Engineering University of Minnesota Minneapolis, MN 55455 ma mater0133 umn. edu
ABSTRACT
State-of-the-art machine learning models often incorporate prior knowledge or structural information about the task data distribution. In some tasks, such knowl- edge may arise from first principles or emerge as simplified, learned functions that distill essential aspects of the data distribution. Model augmentation has emerged strategy leverage this structured knowledge by coupling it with auxiliary model to improve predictive performance, while preserving the interpretability augmentation
offered by the simpler component. In this work. present framework called the Tutor-Pupil scheme, which is designed to enhance both per- formance and interpretability. The Pupil is fixed model, structurally designed for the core task, while the Tutor input-level corrections to improve the Pupil's performance on the modified input. This strict separation of roles enables the Tutor not only to compensate for the Pupil's limitations but also to act as a diagnostic instrument. By examining the Tutor's 's targeted interventions, we can identify failure modes, detect regions where the Pupil struggles to generalize, and uncover residual patterns or higher-order structures the data not captured by the original model.
more flexible model trained apply minimal
INTRODUCTION
Recent advances in machine learning and artificial intelligence have led to the proliferation of many specialized model architectures, each designed excel particular classes of tasks. For example.
convolutional neural networks (CNNs) (LeCun et al. 1989 Li et al.) 2021 Anwar et al., 2018) have proven highly effective for visual data, leveraging spatial locality and translation invariance to classify and manipulate images. Transformers have revolutionized natural language processing by capturing long-range dependencies through attention mechanisms (Vaswani et al. 2017 Patwardhan al. 2023). Recurrent neural networks (RNNs) (Rumelhart al. | 19855 Jordan 1997) and their variants (Wagas & Humphries] 2024) continue to serve time deme-dependentt modeling tasks, while more paradigms, such liquid neural networks (Hasani al.) 2021) and diffusion models (Sohl- Dickstein et al.) 2015] Croitoru et al.) 2023). expand the landscape of architectures capable of dynamically adapting to new inputs or modeling generative processes. The common thread across these architectures is their ability exploit known patterns and structural relations embedded in domain-specific data, which can be seen as form of a priori information about the task (Hsich et al. 2024 Aditya et al. 2019).

---

Published as a conference paper at ICLR 2026
An alternative strategy for incorporating priori knowledge into machine learning models is offered by Physics-informed neural networks (PINNs) (Raissi 2018) and pyysics-guided neural networks (PGNNs) (Daw et al. (222). Rather than relying solely on architectural design, PINNs and PGNNs inject domain knowledge by enforcing physical constraints, either explicitly through differential equations in the loss function, or implicitly through the use of physsics based features, relationships, or regularization (Karniadakis al., 2021 Willard et al., 2022 These constraints guide the model toward physically plausible solutions even when data is relatively searce or noisy. In both cases, whether through architecture or training objectives, success hinges on the model's ' ability to inter- nalize and utilize structural information about the underlying phenomena (Von Rueden et al. (02T). However, while a carefully chosen architecture or the incorporation of correct first principles in a model can capture substantial structure present in the data, it does not necessarily capture all relevant patterns, especially those that are more subtle, or unanticipated (Krishnapriyan et al. 2021 Karpatne et al. 2024).
To further improve predictive performance and generalization, recent line of research has explored model augmentation, where a primary model is enhanced through the addition of a secondary, auxiliary model (Ghosh et al. 2019). The interconnection between the two models can take various forms, including series, parallel, and feedback configurations, as well as more intricate composite structures (Sun et al. 2020; Götte & Timmermann 2022, Groote et al. 2022, Shah et al., 2022] 2025).. The primary model is typically chosen to reflect the structural characteristics or prior knowle lndwledge suited to the task (for instance, a CNN for image data or dynamical model for time-series), while the auxiliary model keeps more flexible architecture in order to capture secondary patterns or unknown relationships in the data. Indeed, model augmentation has been extensively used in settings where the primary model is derived from scientific knowledge (for example, pyrolysis equilibrium model in (Jiang et al., 2024) and low resolution atmospheric global circulation model in (Arcomano et al.) 2022)) allowing the auxiliary model to compensate for discrepancies between theoretical assumptions and observed data (Sharma & Liu 2022).
Hoekstra al.] [2024] Györök
related approach that of ensemble modeling, where multiple models of comparable capacity are trained and combined to improve robustness and accuracy. In contrast to model augmentation, which establishes structured interaction between primary model (often grounded in priori knowledge) and an auxiliary model, ensembles aggregate diverse learners to mitigate variance and bias, albeit often at the expense of interpretability (Zhou) 2025
Freund| 1999).
The idea also bears resemblance to residual learning (He et al., 2016 Shafiq & Gu 2022). In residual
networks, such as ResNet, the architecture is designed to learn a residual mapping instead of the original transformation, primarily to ease optimization and improve gradient flow in deep models. These residual connections operate in the hidden feature space and serve an optimization purpose rather than a modeling one. By contrast, in model augmentation the primary model is typically fixed based on prior knowledge, and the auxiliary model learns correction in the output space to compensate for discrepancies between theoretical assumptions and observed data.
In this work, we adopt the model augmentation paradigm but shift its focus toward a new goal: interpretability and explainability. Rather than viewing the auxiliary model exclusively as corrective mechanism to boost accuracy, we treat it also as an interpretive tool that enhances transparency and insight. Towards this goal, we introduce specific augmentation scheme, which we call the Tupor-Pupil scheme. In this scheme, the Pupil is the aas--specallized model, chosen to match the structure of the application data, while the Tutor is more flexible architecture that learns to model second-order discrepancies, that is. residual patterns not captured by the Pupil. Crucially, the Tutor is trained to make these corrections minimally by intervening only to the extent necessary to improve the Pupil's 's output.
This separation of roles offers a powerful framework for both improving performance and probing model behavior. Because the Tutor's interventions are designed to be as limited as possible, and since the Pupil remains unchanged during the training of the Tutor model, their structure becomes informative: they expose where and how the Pupil's reasoning fails or needs refinement. By analyzing the Tutor's 's corrections, we can identify specific failure modes, uncover generalization gaps, and explore regions of uncertainty in the input space. Moreover, when the Pupil is grounded in known scientific or domain-specific principles, such as in the case of first-principles models, the Tutor's s corrective patterns may reveal higher-order structures or unmodeled effects. These insights can then be extracted and formalized using symbolic regression (Kronberger et al.] 2024) or model-based explainable AI techniques (Ribeiro et al. 20066). effectively expanding the original model's conceptual
2

---

Published as a conference paper at ICLR 2026
foundation. In this way, the Tutor does not merely enhance performance, but instead becomes bridge between empirical behavior and theoretical understanding.
2 TUTOR AUGMENTATION OF AN INTERPRETABLE PUPIL
To concretely illustrate the Tutor-Pupil architecture, begin with simple yet representative toy example: binary classification task in which the Pupil model is implemented as a decision tree. Decision trees are considered inherently interpretable, but often suffer from limited expressiveness, particularly when faced with complex or noisy decision boundaries. By augmenting this base classifier with more flexible Tutor model, we aim to demonstrate how minimal input-level corrections can substantially improve classification performance while preserving the interpretability of the original decision process. Specifically, consider an elementary binary classification problem, as shown in Figure 1 in which features are denoted by and 22 and data points with label are represented by marker while those with label y decision boundary is shown with a green line.
represented by 'o' markers. The unknown true
3.00
3.00
2.75
2.75
2.50
2.50
2.25
2.25
2.00
2.00
1.75
1.75
1.50
1.50
1.25
1.25
1.00
1.00
1.0
1.5
2.5
3.0
1.0
2.0
2.5
3.0
(b)
Figure 1: (a) represents the ground truth boundaries of binary classification problem. The polygon represents the interpretable function f that performs classification. Data represented in orange and light blue (b) aare data that the augmented model classified them differently than the interpretable model. The minimal change identified by the neural network is depicted with black arrow, and the modified data point + is depicted with black star. To make the figure less crowded, subset of data points are shown in this figure.
To perform this classification, we fit shallow decision tree, which makes predictions based on thresholding values of input features, the data. The decision boundaries induced by such tree partition the feature space into polytopes within which the model's output remains constant. Assume that the decision tree maximizing classification accuracy on the training set has been selected and represented by the function f 22). The decision boundaries defined by Figure [(a) highlighted with a purple stroke.
depicted in
The principal merit of this approach lies in its transparency: each parameter of the model admits complete interpretation in terms of its effect on the decision boundary, rendering the model not just predictive device but an intelligible explanation. Despite its simplicity, the decision tree succeeds in capturing substantial portion of the data distribution. Yet, Figure ((a) shows, the model fails account for certain subtleties the distribution, particularly at its margins.
On the other hand, we could turn to neural networks other more complex machine learning models that have a larger parameter space and achieve better accuracy. However, this would come at the cost of interpretability, as the decision boundary, though potentially more accurate, would be more opaque the contribution of individual parameters becomes less clear. Thus, are presented with

---

Published as a conference paper at ICLR 2026
trade-off between the accuracy afforded by complex models and the intelligibilityy characteristic of simpler ones.
It is precisely this trade-off that motivates the notion of model augmentation. The guiding observation is that simple, interpretable models, though limited in expressive capacity, often succeed in capturing meaningful low-order structure within the data. Rather than discarding such a model because of its limited accuracy, we may retain it as primary model and assign to more complex auxiliary model the task of accounting for the residual error that the primary model cannot account for.
Several augmentation schemes have been proposed. Among them, the parallel configuration, depicted in Figure 2(a) is the most prevalent. In this scheme, auxiliary model is trained to minimally perturb
f
f
9
NN
NN
(b)
Figure 2: (a) Parallel augmentation: The auxiliary model is trained to directly correct the output of the primary model, improving performance but limiting anterpretabilityeeppeeee where the auxiliary model dominates. (b) Tutor-Pupil augmentation: The auxiliary Tutor model applies minimal corrections in the input space, enabling the Pupil to make accurate predictions. This input-level intervention provides more interpretable explanation of the primary model's failure modes and highlights how decision boundaries might need to shift.
in regions
the output of the primary model f. improving overall accuracy while preserving interpretability when possible. While this approach can indeed boost performance, it operates by modifying the output directly. As a result, in regions where the primary model f performs well, the prediction is primarily determined by the interpretable function f. preserving its interpretability. However, in regions where performs poorly, the auxiliary model dominates the prediction. Since this model is typically opaque, interpretability is lost in precisely the regions where the model's reasoning is most in question. Even if the correction is kept small, this scheme only reveals how far the output of the interpretable model must be adjusted, offering little insight into why the model fails or what structure underlies the residuals.
In view, more informative alternative is provided by the Tutor- nutor-Pupil augmentation scheme shown in Figure which we propose and study in this work. this configuration, the auxiliary model, the Tutor, suggests a small perturbation to the input x, such that the Pupil (the primary, musk-ppciallize model) produces the correct output when evaluated at + e. Since the correction e is applied in the input space, it lends itself more readily to human interpretation, offering an intuitive understanding of where the Pupil model's reasoning broke down and how it be amended. In other words, rather than adjusting predictions post hoc, the Tutor helps reveal and compensate for the Pupil's limitations in way that illuminates the decision process, especially in regions where the Pupil "struggles."
We can visualize and illustrate the behavior of this Tutor-Pupil augmentation scheme using our toy binary classification example. The Pupil model, a decision tree, assigns label y 1 to all points that fall within the polytope shown in Figure [(a] However, some points with the true label y lie inside this region and are therefore misclassified. The role of the Tutor, implemented here as a neural network, is to learn small perturbations to the input such that the modified input x + is correctly classified by the Pupil. For misclassified points with y = O, this means nudging the input just outside the decision boundary to achieve f(r + e) O, while keeping minimal. Conversely, for points with y = lying outside the region, the Tutor learns perturbations that shift them inward, correcting the classification to f(x + c) = 1.
This behavior is encouraged through the following loss function used to train the Tutor
L = Lcly, f(x + +)) +X
|| correction magnitude
classification
4

---

Published as a conference paper at ICLR 2026
where [y, [ <)) is the classification loss (e.g. binary cross entropy), and À is regularization parameter that limits the correction magnitude. On average, this augmentation improves test accuracy by about 13%. Implementation details, along with the full set of performance metrics and statistical tests used to assess these changes, are provided in Appendix [
The result of this training is shown in Figure [(b] (to make the figure less crowded, a subset of data points is shown in this figure). Misclassified points (f(x)) o y) are highlighted in light blue for y 0 and orange for y 1. Black arrows indicate the learned perturbation vectors 6, and the resulting modified inputs x + are marked with black stars. Consider, for example. the data point x (2.05, 1.5) with the true label y O. Since lies within the Pupil's decision boundary, we have f(z) = 1. The Tutor proposes a correction e = (- (-0.0082, 0. 0.0960), which yields + = (2.0418, 1.5960) which the Pupil now classifies correctly as f(x+ = 0. This perturbation largely along the æ2-axis, suggesting that the local data distribution deviates from the Pupil's axis-aligned decision boundary, bending inward along that axis. Such directional corrections offer insight into how the decision boundary should deform to better match the true data structure.
In this respect, we would like to emphasize that the Tutor-Pupil scheme enables a form of interpretabil- ity that is inherently more global than traditional, local explanation methods such as counterfactuals (Dai et al. 2022] (ttepin al. 202T). While counterfactual explanations typically focus on how small changes in individual features affect a single prediction, the Tutor is trained across the entire dataset to learn coherent patterns of correction that apply broadly across the input space. Rather than probing the sensitivity of the output to feature-level perturbations, the Tutor acts as a learned, input-level corrective mechanism that identifies systematic discrepancies between the Pupil's is decisions and the true labels. This allows for multi-feature interventions that reflect higher-order structure in the data, offering insights into general failure modes of the Pupil model rather than isolated explanations tied to individual instances.
3 INTERPRETABLE AUGMENTATION OF FIRST PRINCIPLE FUNCTIONS
The Tutor-Pupil augmentation framework described in the previous section is not limited to data- driven interpretable models. It can also be applied to functions derived from first principles, such as those based on physical laws. These models, though principled and interpretable by design, often rely on idealized assumptions that inevitably limit their accuracy when confronted with complexities of real-world phenomena. To illustrate this broader applicability of Tutor-Pupil augmentation, consider the example of the behavior of an ideal gas as modeled from thermodynamic principles. The state of such a system is determined by three macroscopic variables: pressure (P), volume (V), and temperature (T) linked through the well-known equation of state:
n RR
P
where R is the universal gas constant and n is the number of moles of gas (Hill 2013). This relation. theoretically derived from statistical mechanics, captures essential features of gaseous behavior under several key assumptions: particles modeled having infinitesimal radius. interact only via elastic collisions (ensuring energy conservation in accordance with the first law of thermodynamics) and follow Hamiltonian dynamics, which enables the application of Liouville's 's theorem and the H-theorem [Tolman| 1979. Callen] 2006), thereby implying the second law of thermodynamic by ensuring non-negative entropy changes under adiabatic transformations (Chandler] 1987] Thompson 2015). Additionally, no interaction potential is assumed between particles beyond their instantaneous collisions (Tolman) 1979 Chandler 1987). Such a relation serves as a remarkably effective approx- imation for many scenarios, yet. like all idealized models, its predictions may exhibit significant deviation from real-world behavior when its underlying assumptions no longer hold.
To explore this point concretely, we conducted a simulation of spherical particles with finite radius > 0 confined within a two-dimensional box of volume V. All collisions, both between particles and with the container walls, were modeled as perfectly elastic. An illustration of the simulated system is provided in Figure ((a) This setup was designed to offer a microscopic perspective on gas behavior, bridging particle-level dynamics with macroscopic quantities. Indeed, we defined the system's 's temperature as the average kinetic energy of the particles and computed pressure P based on the average momentum exchanged between particles and the container walls (see the supplementary material for full details of the simulation procedure). With these definitions in place, we compared the macroscopic predictions of the ideal gas law to the behavior observed in our microscopic simulation. To visualize the results, we plotted isotherms (Lewis & Randall 2020).
5

---

Published as a conference paper at ICLR 2026
Augmentoc
Ideal gas law
Semulated data
Pooitan larb. urits)
15
Ressure
50
80
15000
32000
50000
Position (arb.
Valume
(a)
(h)
Figure 3: (a) shows a visualization of the simulation performed, where each particle shown with dot and the velocities are shown as the vectors. [b) shows the isotherms of the simulated data, shown in pink, ideal gas law, shown in green, and the augmented model shown in blue. We can see that ideal gas law underestimates the pressure of the gas in small container volumes.
curves of pressure versus volume at constant temperature, that are commonly used to characterize how system responds to compression expansion under thermal equilibrium. Figure 3(b)show the resulting geessure-volume isotherms from the simulation alongside the theoretical curves predicted by the ideal gas law.
While the ideal gas law offers reasonable approximation of pressure at large volumes, observe a systematic underestimation pressure as the volume decreases. Also, this discrepancy becomes more pronounced as the temperature increases. Rather than discarding this well established model, decided treat it as Pupil and introduced Tutor (in the form neural network) learning small data driven input corrections, slight shifts in volume and temperature, which we denote as (cv, Cr) to improve the model predictive performance. Specifically, the Tutor model trained to minimize the following loss function
nR(T + +ET)
C
P
+λ
V V
correction magnitude
prediction loss
The predicted pressure of the Tutor-Pupil model is given by P nR(T+er) Figure (bb tracking almost perfectly the experimental pressure. Since the outputs of the Tutor belong to the space, we can investigate the nature of the applied corrections, that are illustrated in Figure The original temperature and volume represented red dots, while the change is shown as the purple vector. The corrected data points are marked blue. The applied corrections become more significant at smaller volumes. Notably, the Tutor essentially prescribes, for small volumes, further reductions effective volume to improve the accuracy of the Pupil model's predictions.
V+ey and is drawn in
Recognizing this pattern, the physical rationale behind correction becomes clearer: addresses key violated assumption of the ideal gas law, namely, that gas molecules have negligible size and can explore the full container volume without restriction. In reality, when particles have non- negligible radii, their centers cannot come closer to the walls than their own radius, effectively shrinking the accessible volume. This reduction becomes particles), leading to systematic deviations from the predictions of ideal gas law.
pronounced in smaller containers (or with larger
In the scientific literature, an important refinement of the ideal gas law is given by the van der Waals equation (Klein 1974), which introduces corrective terms to account for the finite size particles and the intermolecular forces that become significant under high-density or low-temperature conditions.
The van der Waals relation is typically written as:
p nRT
b)
V2

---

Published as a conference paper at ICLR 2026
Simullted data
1200
Corrected data
1000
8
Tempareture
800
8
8
600-
8
8
400
8
0
200
10000
20000
30000
40000
50000
Volume
Figure 4: The plot shows the volume and temperature corrections EV and ET learned by the Tutor. Larger corrections are observed at smaller volumes, where the assumptions of the ideal gas law break down due to finite particle size. The Tutor learns to systematically reduce the effective volume to account for excluded space.
where the parameters a and b are empirical constants that vary across different substances. The constant accounts for the finite volume occupied by the gas particles themselves, effectively reducing the space available for molecular motion, while captures the effect of intermolecular attractions, which tend to reduce the pressure exerted on the walls.
Interestingly, the data-driven Tutor-Pupil framework naturally leads to analogous insights. As shown in the simulation results, the Tutor learns to correct the predictions of the ideal gas law in regimes where its assumptions break down, especially at small volumes. These corrections manifest as volume reductions, reminiscent of the role played by the b term in the van der Waals formulation.
Furthermore, if symbolic equation discovery methods (e.g., sparse regression or neural symbolic regression) were applied to the learned corrections provided by the Tutor, one could recover expres- sions structurally similar the van der Waals equation. However, unlike the classical form where and b are constant parameters, the data-driven approach could yield analytical expressions for the volume and temperature corrections Ev and ér, potentially as functions of V, T. This provides richer, more adaptive model that maintains interpretability while being able capture secondary principles to effectively discover models for non-ideal effects.
TUTOR AS AN EXPLANATORY TOOL FOR A NON-INTERPRETABLE PUPIL
In the previous sections, we illustrated how the Tutor-Pupil architecture can enhance both performance and interpretability when the Pupil is an inherently interpretable model, either by design or by virtue of its connection first principles. In those settings, the Tutor acts corrective mechanism that not only improves accuracy but also reveals where and why the Pupil fails, thereby potentially offering additional insights into the underlying data or physical processes.
In this section, we turn our attention to a different scenario. where the Pupil model is not readily interpretable, either due to its internal complexity or the high dimensionality of its imput space. In such cases, the Tutor's ' explanatory role shifts. Rather than providing insights on the data or the physical world, the Tutor becomes a diagnostic tool that helps us understand the inner workings of the Pupil itself. By learning to apply small, targeted input modifications that improve the Pupil's performance, the Tutor effectively highlights what features the Pupil is sensitive and what information it requires to succeed. In this way, the Tutor acts as probe into the Pupil's 's behavior, illuminating its implicit strategies, underlying mechanisms, and points of uncertainty.
To examine this setting in practice, we consider the task of handwritten digit classification using the MNIST dataset (LeCun et al. 1998). While this benchmark is well-studied and many modern models can achieve near-perfect accuracy it, we intentionally select logistic regression classifier the Pupil model. This choice is motivated by several factors: first, logistic regression is a classical and well-understood method, which allows us to validate whether the Tutor's interventions align with known decision boundaries and model sensitivities. Second, despite its simplicity, logistic regression still operates in a high-din sigh-dimensiinal input space (each 28 x 28 image yields 784 features), making it difficult to interpret directly in terms of raw parameters. This setup strikes balance between
7

---

Published as a conference paper at ICLR 2026
complexity and interpretability: it is complex enough to be opaque to human intuition when applied to image data, yet simple enough for us to assess whether the Tutor's learned corrections reflect the underlying structure of the Pupil's 's decision process.
We train the logistic regression Pupil using standard cross-entropy loss and achieve a test accuracy of 91%. While this performance is respectable, falls significantly short of saate-of-the-aee art models on MNIST, and well above the Bayes optimal error rate, which is believed to be below 1% (Mitchell 1997). In order to improve the performance, we introduce a Tutor model trained learn minimal, input-level corrections to increase the accuracy of the Pupil's predictions.
To implement the Tutor-Pupil scheme in this setting, aim to train Tutor that applies minimal modifications to input images so that the Pupil's performance improves on these corrected versions. However, directly learning pixel-wise corrections in the nigh-dimensional image space is both computationally expensive and prone to overfitting. To address this, we adopt a strategy inspired by recent advances in generative modeling, which allow for compact and structured representations of images (Bond-Taylor et al. 2021). In particular, (VAE) Kingma& & Welling 2014), a widely used generative model that maps high-dimensional data to a lower-dimensional latent space and reconstructs the original inputs from these representations. The VAE regularizes the latent space by enforeing Gaussian prior, ensuring smoothness and continuity in the learned encoding. This structure enables us to train the Tutor to operate in the latent space, where modifications are more computationally tractable. Figure shows the architecture we used for this modification. This structure functionally equivalent to the general Tutor philosophy shown in Figure 2(b) where the modified input r' = r + corresponds to the image reconstructed from the perturbed latent code.
utilize a pretrained variational autoencoder
Toput
Recontriited mpor
Moditod upg
statoder
decodaar Beg
sucodke
dreadlr hop
NN
Az
4
5
(a)
(b)
Figure 5: shows the architecture of a variational autoencoder that is trained to encode an image into the latent space and then decode the latent space into reconstructed image. (b) the structure of the Tutor model chosen for the image classification example. The neural network qø is designed to modify the latent variable z minimally.
We can train this neural network to minimize the following loss:
C = (c(u, +\1 DKL (go(>') | go x))
+
|
212
x
classification
consistency
reconstruction Gidelity
where æ is the input image and y its true label. The encoder go maps The transformation network ds produces a perturbation A on the latent vector yielding z', which is decoded by the pretrained decoder hy yield x' (z))..). The logistic regression then predicts on the modified image z'. Importantly, the parameter X controls the degree of change in the latent space, enforcing that z' remains close to z. This constraint encourages the Tutor to make minimal, interpretable modifications while still improving the Pupil's performance.
latent representation z.
The augmented Tutor- model achieves a test accuracy of 98.5% (ys 91% % when only the Pupil model is used), marking significant improvement over the baseline logistic regression model. Tigure@hhghhige several representative examples where the original model fails, but the Tutor-Pupil augmentation yields the correct classification. With a closer examination of these corrected cases, can see that in almost all of these, the original handwriting deviates from the standard forms, either due to incomplete strokes, irregular proportions, or stylistic variations. We also notice that in all the examples, Tutor subtly modifies the input in way that restores the key structural features. For example, the Tutor completes the circular loop in malformed 0 or 8 (as shown in Figure 6(a) and 6(h)). or extends a slight vertical stroke to clarify the poorly formed 4 (Figure ((d)) These adjustments make the image more legible to the Pupil and allow it to produce the correct classification.
8

---

Published as a conference paper at ICLR 2026
22
3
(a) Pupil: 9, Tutor-Pupil:
(b) Pupil: Tutor- Pupor-Pupil:
(c) Pupil: 5, Tutor- Tutor-Pupil: 3
4
S
5
(d) Pupil: 6, Tutor- Pupoo-
(c) Pupil: 3, Tutor-Pupil: 5
(f) Pupil: 4, Tutor-Pupil: 6
7
7
4
8
4
9
(g) Pupil: 9, Tutor-Pupil: 7
(h) Pupil: 4, Tutor-Pupil: 8
(i) Pupil: 4, Tutor-Pupil: 9
Figure 6: Representative examples of misclassified MNIST digits by the Pupil that are successfully corrected by the Tutor. The left image in each pair shows the original input, which the Pupil alone misclassifies. The right image shows the minimally modified version produced by the Tutor, leading to a correct prediction.
many cases, the Tutor enhances or reconstructs specific digit-defining features (such as closing open loops or sharpening strokes), making the digits more recognizable and aligned with canonical representations.
From these observations, we infer that the chosen Pupil, though effective for typical inputs, struggles to generalize when the digit's structure departs from familiar patterns that are most common in the training examples. Interestingly, this insight is consistent with the known limitations of logisitic regression for this task. As a linear classifier operating directly on pixel intensities, logistic regression does not capture spatial structure or compositional regularities in the way convolutional networks do. is therefore more vulnerable distortions, omissions, or stylistic inconsistencies in the data.
This limitation is precisely why we selected logistic regression for this experiment. As structurally simple and well-understood model. it provides controlled setting in which the Tutor's corrections reveal meaningful failure modes. The alignment between these corrections and the known weaknesses of logistic regression offers concrete support for the broader applicability of the Tutor-Pupil framework. Even when the Pupil is not inherently interpretable, the behavior of the Tutor can illuminate how the model processes inputs and where it falls short, offering a practical means of interpreting otherwise opaque systems.
assess whether
Since we offer the Tutor-Pupil architecture also as an interpretability tool, we decided to compare it with SHAP (Lundberg & Lee 2017), given its prevalence, strong theoretical grounding, and widespread adoption in academic research and real-world deployments. SHAP provides a principled way of attributing importance to input features, and for image data this is often visualized as heatmaps where each pixel is weighted according to its relevance for the model's output. In Appendix D we report the SHAP outputs for the logistic regression model on the representative images of Figure 6) both for the correct label and the misclassified label. While SHAP offers a quantitative feature attribution, in all these cases the resulting heatmaps are challenging for human to interpret, especially given the large number of pixels involved. By contrast, the Tutor-Pupil explanation produces corrections that directly modify the input in human-readable way, providing more intuitive account of the model's behavior.
5
OTHER APPLICATIONS AND CONCLUSION
The Tutor-Pupil architecture proposed in this work offers novel framework that bridges the gap between performance and interpretability in machine learning systems. When the Pupil is designed

---

Published as a conference paper at ICLR 2026
to generalize well (whether it is model derived from first principles learned function), the Tutor acts as a secondary layer that improves upon the Pupil's residual errors and exposes aspects of the underlying phenomenon the Pupil is describing. When the Pupil is a complex or opaque model, the Tutor instead serves as explanatory layer for the Pupil itself, revealing patterns in how it responds to imput data and highlighting the features it relies on to succeed. In both cases, the Tutor is not merely a correction mechanism, but an interpretive tool. Unlike many explanation methods such as counterfactual methods (Prado-Romero et al. 2024), additive explanations (Doumard et al. (023), example based methods (Van Der Waa et al.J 2021). that are purely local (e.g. they focus on an individual input), the Tutor is trained across the entire dataset. As such, it captures global patterns in the Pupil's behavior and failure modes, providing a broader understanding of the system.
While the present work trains the Tutor after the Pupil, promising future direction is to develop joint training procedures where the two systems evolve simultaneously. For instance, enforeing form of functional orthogonality between Tutor and Pupil (so that each captures distinct, complementary aspects of the task) may yield both better performance and more disentangled explanations. Moreover, the flexibility the architecture enables other novel uses. The Tutor can be trained adversarially, not to help but to hinder the Pupil, revealing weak spots or brittle dependencies. This adversarial mode could serve as a diagnostic tool, highlighting vulnerabilities or testing robustness. Another valuable application is bias detection: if the Pupil is influenced by confounding factors in the training data, the Tutor may systematically learn undo these influences, thereby surfacing implicit biases spurious correlations.
On the other hand, although interpretability is promoted by encouraging minimal perturbations, the Tutor-Pupil framework remains flexible enough to incorporate additional strategies that further en- hance the clarity of learned connections. Depending on the application, this may include constraining the Tutor to operate on superpixels other human-ale auman--ligned feature decomposition methods (Ribeiro) ct al. 2016). thereby improving the interpretability of the resulting explanations.
REFERENCES
Somak Aditya, Yezhou Yang, and Chitta Baral. Integrating knowledge and reasoning in image understanding. arXiv preprini 1XXix:996.00954. 2019.
Syed Muhammad Anwar, Muhammad Majid, Adnan Qayyum, Muhammad Awais, Majdi Alnowami, and Muhammad Khurram Khan. Medical image analysis using convolutional neural networks: review. Journal of medical systems, 12:1-13, 2018.
Troy Arcomano, Istvan Szunyogh, Alexander Wikner, Jaideep Pathak, Brian Hunt, and Edward Ott. A hybrid approach to atmospheric modeling that combines machine learning with a physics-based numerical model. Journal of Advances in Modeling Earth Systems, (42)):2222MM0027 12, 2022.
Sam Taydor, Adam Leach, Yang Long, and Chris G Willcocks. Deep generative modelling: A comparative review of vaes, gans, normalizing flows, energy-based and autoregressive models.
IEEE transactions on pattern analysis and machine intelligence, 44(11):7327-7347, 2021.
Herbert B Callen. Thermodynamics & an introduction to thermostatistics. John wiley & sons, 2006.
D. Chandler. Introduction to Modern Statistical Mechanics. Oxford University Press, 1987.
Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor lonescu, and Mubarak Shah. Diffusion models in vision: A survey. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(9): 10850-10869, 2023.
Xinyue Dai, Mark T Keane, Laurence Shalloo, Elodie Ruelle, and Ruth MJ Byrne Counterfactual explanations for prediction and diagnosis in xai. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, pp. 215-226, 2022.
Arka Daw, Anuj Karpatne, William D Watkins, Jordan S Read, and Vipin Kumar. Physics-guide guiide neural networks (pgnn): An application in lake temperature modeling. In Knowledge guided machine learning, pp. 353-372. Chapman and Hall/CRC, 2022.
Emmanuel Doumard, Julien Aligon, Elodie Escriva, Jean- caan-Btptiste Excoffier, Paul Monsarrat, and Chantal Soulé-Dupuy. A quantitative approach for the comparison of additive local explanation methods. Information Systems, 10:1122,,, 2023.
10

---

Published as a conference paper at ICLR 2026
Yoav Freund. A short introduction to boosting. Joumna-Jaaaanse Society For Artificial Intelligence, 14(771-780): 1612. 1999.
Debanjan Ghosh, Emma Hermonat, Prashant Mhaskar, Spencer Snowling, and Rajeev Goel. Hybrid modeling approach integrating frrst-principles models with subspace identification. Industrial & Engineering Chemistry Research, 58(30):
2019.
Sicarda-Samanteee Götte and Julia Timmermann. Composed physics-and and data-driven system iden- tification for non-autonomous systems in control engineering. In Proc. of the 3rd International Conference on Artificial Intelligence. Robotics and Control, pp. 67-76, 2022.
Wannes De Groote et al. Neural network augmented physics models for systems with partially unknown dynamics: Application to slider-crank mechanism. IEEE/ASME Transactions on Mecha- fronics, 27:103- 14, 2 2022. ISSN 1941014X. doi: 10.1 10.1109/MMECH.2011.300
Bendegúz M Györök, Jan H Hoekstra, Johan Kon, Tamás Péni, Maarten Schoukens, and Roland Tóth. Orthogonal projection based regularization for efficient model augmentation. arXiv preprint arXiv:2501.05842 2025.
Ramin Hasani, Mathias Lechner, Alexander Amini, Daniela Rus, and Radu Grosu. Liquid time- constant networks. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 7657-7666, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770-778, 2016.
Terrell L Hill. Statistical mechanics: principles and selected applications. Courier Corporation, 2013.
Jan H Hoekstra, Chris Verhoek, Roland Tóth, and Maarten Schoukens. Learning-based model augmentation with lfrs. arXiv preprint arXiv:2404.0/ 009001............ 2024.
Han Hsieh, Ru-Xiu Hsiao, Katalin Ferenc, Anthony Mathelier, Rebekka Burkholz, Chien-Yr Yu Chen, Geir Kjetil Sandve, Tatiana Belova, and Marieke Lydia Kuijjer. Cavachon: a hierarchical variational autoencoder to integrate multi-modal single-cell data. arXiv preprint arXiv:2405. 2865. 2024.
Peng Jiang, Chenhan Wang, Jing Fan, Tuo Ji, Liwen Mu, Xiaohua Lu, and Jiahua Zhu. Hybrid residual modelling biomass pyrolysis. Chemical Engineering Science, 293: 10096, 2024. iSSN 0009-2509. doi: https://doi.org/II 0.tg/11.1100 101 ceijcces.222.. 100224.110099 20096.
Michael I Jordan. Serial order: A parallel distributed processing approach. In Advances in psychology, volume 121, pp. 471 195. Elsevier, 1997.
George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422-440, 2021.
Anuj Karpatne, Xiaowei Jia, and Vipin Kumar. Knowledge-guided machine learning: Current trends and future prospects. arXiv preprint arXiv:2403.. 155999 2024.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In Proceedings the 2nd International Conference on Learning Representations (ICLR), 2014. htttps://aat
the
org/ bbg//bbs/ie 1312. 6114
Martin J Klein. The historical origins of the van der waals equation. Physica, 73(1):28 47, 1974.
Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Char- acterizing possible failure modes in physics-informed neural networks. Advances in neural information processing systems, 34:26548-26564 2021.
Gabriel Kronberger, Bogdan Burlacu, Michael Kommenda, Stephan M Winkler, and Michael Affen- zeller. Symbolic Regression. CRC Press, 2024.
Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, Richard Howard, Wayne Hubbard. and Lawrence Jackel. Handwritten digit recognition with a back proo-prrpggttton network. Advances in neural information processing systems, 2, 1989.

---

Published as a conference paper at ICLR 2026
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11)22278-22224 1998.
Gilbert Newton Lewis and Merle Randall. Thermodynamics. Courier Dover Publications, 2020.
Zewen Li, Fan Liu, Wenjie Yang, Shouheng Peng, and Jun Zhou. A survey of convolutional neural networks: analysis. applications. and prospects. IEEE transactions on neural networks and learning systems, 33(12):6999-7019 2021.
Scott M Lundberg and Su-In Lee. unified approach interpreting model predictions. Advances in neural information processing systems, 30, 2017.
Tom M. Mitchell. Machine Learning. McGraw-Hill, New York, 1997.
Narendra Patwardhan, Stefamo Marrone, and Carlo Sansone. Transformers in the real world: A survey on nlp applications. Information, 14(4):242, 2023.
Mario Alfonso Prado-Romero, Bardh Prenkaj, Giovanni Stilo, and Fosca Giannotti. survey on graph counterfactual explanations: definitions, methods, evaluation, and research challenges. ACM Computing Surveys, 56(7):1-37, 2024.
2024.
Maziar Raissi. Deep hidden physics models: Deep learning of nonlinear partial differential equations. Journal of Machine Learning Research, 19(25):1-24 2018.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
David E Rumelhart, Geoffrey Hinton, Ronald Williams, al. Learning internal representations by error propagation, 1985.
Muhammad Shafiq and Zhaoquan Gu. Deep residual learning for image recognition: A survey. Applied sciences, 12 12(18):977,, 2022.
Parth Shah et al. Deep neural network-based hybrid modeling and experimental validation for an industry-scale fermentation process: Identification of time-varying dependencies among parameters. Chemical Engineering Journal, 441:135643, 2022. ISSN 1385-8947. doi: 10.1016/J /111///.CE.200000 202. 135643.
Niket Sharma and YA Liu. A hybrid science-guided machine learning approach for modeling chemical processes: A review. AIChE Journal, 68(5), 2022.
Jascha Sohl- Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pmlr, 2015.
pp.
Ilia Stepin, Jose M Alonso, Alejandro Catala, and Martín Pereira-Fariñla. A survey of contrastive and counterfactual explanation generation methods for explainable artificial intelligence. leee Access, 9:11974-1200 12001, 2021.
Bei Sun et al. A comprehensive hybrid first principles/machine learning modeling framework for complex industrial processes. Journal of Process Control, 86:30-43, 2020.
Colin J Thompson. Mathematical statistical mechanics. Princeton University Press, 2015.
Richard Chace Tolman. The principles of statistical mechanics. Courier Corporation, 1979.
Jasper Van Der Waa, Flisabeth Nieuwburg, Anita Cremers, and Mark Neerincx. Evaluating xai: A comparison of rule-based and example-based explanations. Artificial intelligence, 291:103404, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
12

---

Published as a conference paper at ICLR 2026
Laura Von Rueden, Sebastian Mayer, Katharina Beckh, Bogdan Georgiev, Sven Giesselbach, Raoul Heese, Birgit Kirsch, Julius Pfrommer, Annika Pick, Rajkumar Ramamurthy, et al. Informed machine learning- a taxonomy and survey of integrating prior knowledge into learning systems. IEEE Transactions on Knowledge and Data Engineering, 35(1):614-633, 2021.
Muhammad Waqas and Usa Wannasingha Humphries. A critical review of rnn and Istm variants in hydrological time series predictions. Methodsx, pp. 102946, 2024.
Jared Willard, Xiaowei Jia, Shaoming Xu, Michael Steinbach, and Vipin Kumar. Integrating scientific knowledge with machine learning for engineering and environmental systems. ACM Computing Surveys, 55(4):1-37, 2022.
Zhi-Huu Hua Zhou. Ensemble methods: foundations and algorithms. CRC press, 2025.
13

---

Published as a conference paper at ICLR 2026
A IMPLEMENTATION DETAILS AND PERFORMANCE METRICS FOR SECTION 2
In order to implement the Tutor- Pupor-ppplllo augmentation as discussed section we choose a shallow decision tree as our pupil model and fit it to the data. As mentioned previously. in order train a Tutor neural network that will produce the minimal modifications e, we use the following loss function:
L Lcly, fax + +)) +X
|
classification
correction magnitude
To optimize the weights of the Tutor, must backpropagate the gradient of the overall loss. Since this loss includes the classification loss from the Pupil, the gradient must flow through the Pupil's function. However, derivative of the loss with respect the parameters of the Tutor. To enable this implementation, we approximate the decision tree with differentiable function, defined as follows:
the Pupil's weights are frozen, this gradient is used solely to compute the
8 = ok x (æ1 lower.,)),
02 = a (k: x (- 1 + upper,,)) )),
0g a alk: x (x2 lowerz2)))
04 al x (-22 + upperx2))
f() 0, 020g04.
where o()) is the sigmoid function function. The parameters lower upperays lower a2 and upperag are determined by fitting the model to the data. This function works similarly to a de- cision tree since it checks if the features 21 and Iy lie within the range [lowerz, upper and [lowerr upper>>] respectively. The parameter & determines how sharp the edges and corners of these boundaries are enforced by sharpening the sigmoid plot. the data point inside the polygon, 01. 02 0z, 0₂ are all equal to 1, if any of them is 0, label O is assigned instead (f(x: 0). The polygon fitted this data shown Figure [(a) with a purple stroke. The parameters found by fitting the model to the data arc:lower,, 1.2, upper 2.87, lower = 1.43, upper.> = 2.36, k 110.0.
3.00
3.00
2.75
2.75
2.50
2.50
2.25
2.25
2.00
2.00
1.75
1.75
1.50
1.50
1.25
1.25
1.00
1.00
1.5
2.0
2.5
3.0
1.0
1.5
2.0
2.5
3.0
(a)
(b)
Figure 7: (a) represents the ground truth boundaries of binary classification problem. The polygon represents the interpretable function f that performs classification. Data represented in orange and light blue in (b) are data that the augmented model classified them differently than the interpretable model. The minimal change identified by the neural network is depicted with black arrow and the modified data point x + depicted with a black star.
Now f(x) is a differentiable function and we can implement the Tutor. Both the dataset and the complete code are included in the attached ZIP file.
14

---

Published as a conference paper at ICLR 2026
For this toy example, conduct variability analysis to better understand if the augmentation of the decision tree with the neural network truely leads to better performance metrics.
To evaluate variability, we conducted independent runs of 5-fold cross-validation (a total of 30 training procedures) for the simple model. The best-performing model (based validation loss) from each run was selected as the Pupil in the Tutor-Pupil framework. We then applied the same procedure for training the augmented model. The accuracy, precision, and Loss Per Sample (L.PS) are reported below (mean and std).
Table I: Accuracy and precision (mean and standard deviation) across conditions.
Condition
Accuracy (mean)
Accuracy (std)
Precision (mean)
Precision (std)
Tree training
0,8624
0.0131
0.8459
0.0139
Tree validation
0.8583
0.0290
0.8385
0.0306
Tree test
0,8588
0.0254
0,8413
0,0415
Augmented training
0.9768
0.0464
0.9655
0.0716
Augmented validation
0.9754
0.0485
0.9610
0.0795
Augmented test
0.9705
0.0532
0.9512
0.0859
Table 2: LPS values (mean and standard deviation) across conditions.
Condition Tree training
LPS (mean)
LPS (std)
0.4217
0.0296
Tree validation
0.4586
0.1088
Tree test
0.4446
0.0965
Augmented training
0.8436
0.0965
Augmented validation
0.9739
2.5082
Augmented test
1.0350
2.6602
We also evaluated whether the observed improvement in test accuracy for the augmented model is statistically significant. To this end, we performed one-sided Welch's t-test with the following hypotheses:
Ho Maree Paugmented
H1 Puree < Paugmentedd
We the significance level at p 0.05. The resulting one-sided p-value 2.648 10-11 leading us to reject the null hypothesis and conclude that the augmented model achieves a statistically significant improvement in test accuracy.
B SIMULATION PROCEDURE FOR SECTION 3
This section describes the simulation procedure for spherical particles of finite radius r 0 confined within wwo-dimensional square box of area V. A brief but detailed description is given in this section, however the code, given in a zip file, contains mode detailed comments.
Given the box dimension (assuming a square box so that V 6²), the number of particles n, and the particle radius r, we initialize the positions x and velocities v of all particles at random, while ensuring that two particles overlap at the start.
At time t O, the simulation begins by identifying the earliest collision. This determined by computing the first time at which the distance between the centers of any two particles becomes less than 2r. Collisions with the box walls are also considered in this step.
Once the earliest collision is detected, we compute the corresponding collision time At and identify the collision pair, which may be either two particles or particle and wall. The simulation time is then advanced to + At, and particle positions are updated according +- + vAt.
The velocities the colliding pair are then updated. For purticl--particle collisions, the update reflects conservation of momentum and energy; for particle-wai collisions, the relevant component of the velocity vector reflected.
This process is repeated iteratively until the total simulation time exceeds a prescribed maximum duration T Tmax-
15

---

Published as a conference paper at ICLR 2026
B.1 MEASURING PRESSURE AND TEMPERATURE
To connect macroscopic thermodynamic quantities to microscopic properties the simulated system, we begin with fundamental result of the kinetic theory of gases, according to which, the average kinetic energy of particle is given by:
1d K.T. 2
E
Here na denotes the number of degrees of freedom (equal to 2 for a monoatomic particle confined to two dimensions), and Kù is Boltzmann's 's constant. Independently, the kinetic energy may also be expressed in terms of the mass and average velocity of a particle:
m
E
(3)
2 averaage
where m the particle mass, and Vaverage is the average velocity of particles. By equating these two expressions and solving for T', we obtain:
T nake Vaverage"
Thus, just by appropriately rescaling the temperature by scalar constant, with no loss of generality, we can define it directly in terms of the average kinetic energy of the particles, which is computed as the mean of the squared norms of their velocity vectors.
The pressure may be computed by invoking the kinetic theory of gases in its wo-dimensional formulation, which relates macroscopic force to microscopic momentum exchanges.
F
P
(5)
Here, F denotes the total force exerted by the particles on the walls of square container of width This force arises from the cumulative effect of momentum changes due collisions with the boundaries. According to Newtonian mechanics, force is defined as the rate of change of momentum:
F Ap
(6)
At
where Ap is the total change in momentum due to wart collisions, defined as
Ap Apil.
with each Ap; representing the momentum change from individual collision. The sum is taken over all collisions during the simulation, and the absolute value ensures that all impulses, regardless of direction, contribute positively to the total momentum transfer. Since momentum is given by mu, and changes in momentum arise from velocity reversals upon wall impact, we compute each Ap; as m/ Avy/, where Au; is the change in the velocity component perpendicular to the wall. Summing these over all collisions yields the total momentum transfer. The resulting average force is then given by
Ap
F
(8)
T Tiax
where T Tmax is the total simulation time. The measured pressure thus given by:
Ap
P
Tru
(9)
4b
B.2 TUTOR IMPLEMENTATION
The simulations were performed for 15 different volume settings and 8 different temperature settings, resulting in total of 120 data points.
The Pupil model is based on the ideal gas law, which estimates pressure using the equation P: nRT This model is implemented PyTorch.
Next, we construct the Tutor model, also in PyTorch. It takes temperature T and volume V as input and outputs correction vector (rv, The model consists of two hidden layers and uses
16

---

Published as a conference paper at ICLR 2026
final activation function of Tanh () to ensure that the corrections remain bounded and small in magnitude.
However, because the simulated values of V and T lie in the ranges V E [13550.0, 88492.0] and T [60.0, 1224.0J, respectively, we scale the output of the Tanh () function accordingly. We then create an augmented model as illustrated in Figure 2(b)
C IMPLEMENTATION DETAILS FOR SECTION
In this section, we briefly go over the implementation of the Tutor-Pupil framework discussed in Section 4 The Pupil is logistic regression model, implemented in PyTorch as single linear layer. is trained using the cross-entropy loss and optimized with Adam for 10 epochs, achieving a test accuracy of 91%.
To construct the Tutor, we employ variational autoencoder (VAE) that operates in a compact latent space rather than directly in the input space. While pretrained VAEs are available through libraries such Py PyTocc and TensorFlow, they are typically designed for high-resolution or complex images, which makes them suboptimal for our case. Our data consists solely of handwritten digits with resolution of 28 x 28 pixels, so we opt to train a smaller VAE tailored to this setting.
The encoder of VAE comprises three convolutional layers followed by linear layer that outputs the mean 14 and the log- variance ( of the latent distribution. The latent space is set to have dimension 32. The decoder consists of a linear layer followed by three transposed convolutional layers. During training, an imput image passed through the encoder, latent variable 2 is sampled from the distribution (M,, o²) using the reparameterization trick, and the decoder reconstructs the image from
The VAE is trained for 50 epochs using the Adam optimizer. The loss function used is the standard variational autoencoder loss, which combines a reconstruction term with a regularization term enforcing the approximate posterior match a standard normal prior. Formally, the loss function is given by
LVAE (x) = (=(a)) [log po(=)=)] - DKL (90(2(a)) || p(z)),
(10)
where go (2(2)) is the encoder (approximate posterior), pe (x]22) is the decoder (likelihood), and p(2) = NO, 7) is the prior over the latent variables.
Since VAES known to smooth image features, we evaluated the effect of applying VAE encoding and decoding to the images without involving the Tutor. Under this setting, the logistic regression model achieved an improved accuracy of 91.7%.
The Tutor is then implemented to apply minimal change the latent variable a such minimze the loss given in equation equation] [
This network consists of four sequential linear layers. The complete augmented model is constructed by combining the Pupil (logistic regression model), the Tutor network, and the pretrained VAE.
After initialization, the weights of both the VAE and the Pupil are frozen. When an image is input to the augmented model, the encoder of the VAE produces the mean Y and the varianiee log(o?). The mean μ is then fed to the Tutor network to produce z' in order to eventaully produce via the decoder.
The reconstructed input z' is subsequently passed the Pupil model, which outputs a predicted label
The Tutor network is trained for 40 epochs using the Adam optimizer to minimize the loss defined in Equation equation]] [] The hyperparameters ê and X2 are selected based on empirical tuning. The final test accuracy of the augmented model reaches 98. 37%. This outcome demonstrates substantial improvement over the baseline performance obtained by applying the Pupil model directly to the original input x.
We note that this performance depends critically on the quality of the latent space representation. In particular, replacing the VAE with a simple autoencoder would significantly reduce the likelihood that the full augmented model will perform well. This is because VAEs employ the reparamcterizztion trick, which enables sampling of the latent variable via
a 6, where A/0 T), in
17

---

Published as a conference paper at ICLR 2026
way that permits gradient propagation during backpropagation. This mechanism ensures that small changes in the input & produce small and consistent changes in the latent representation 2.
As result, the latent space learned by the VAE is smooth, continuous, and structured. Furthermore, because the latent variables are regularized toward standard Gaussian distribution, interpolations between latent vectors tend to remain within regions the decoder has been trained reconstruct accurately. This property enables the generation of semantically meaningful outputs across the latent space, In contrast, traditional autoencoders, which lack such regularization, often produce unstruc- tured latent spaces where interpolated points may correspond to implausible or out-of-distribution inputs.
D COMPARISON WITH SHAPLEY VALUES FOR SECTION 4
In this section we provide comparison between the explanation given by the application of the Tutor-Pupil architecture for the logistic regression model applied the MNIST dataset and the explanations obtained with SHAP, a widely used method explainable AI,
The results are reported in Figure g for some representative MNIST examples corresponding to cases of misclassification by the Pupil acting alone. Each subfigure is organized the original image (top left), its VAE-based reconstruction (top right) obtained with the help of the Tutor, and two SHAP heatmaps corresponding predicted by the Pupil alone (bottom row). The intent is to highlight the difference between traditional feature-attribution explanations and the more direct, human-readale Tutor. While SHAP offers pixel-level relevance scores, the resulting heatmaps are often difficult to interpret: they indicate where the Pupil focuses in the image, but they do not provide a clear sense of how the model is reasoning or why it fails. For instance, in misclassified cases, the Pupil's heatmap of attribution scores remains diffuse and not easily translatable into an intuitive explanation of the classification error. By contrast, the Tutor-Pupil approach explicitly modifies the input in minimal and targeted way, effectively "adding" the features that the Pupil requires reach the correct label. These corrections are directly visual and semantically meaningful (for example, sharpening the loop of "0" extending the top bar of a "7") and thereby offer much clearer view into the Pupil's 's internal logic and failure modes. In this sense, the Tutor does not merely localize important pixels but provides a global, structured intervention that is accessible to human interpretation, complementing and often surpassing the insights obtainable from standard attribution methods like SHAP.
3x2 grid, with
the correct label (middle row) and to the label
corrections provided by the
18

---

Published as a conference paper at ICLR 2026
O
22
33
(a) Pupil: 9, Tutor-Pupil: 0
(b) Pupil: 7, Tutor-Pupil: 2
(c) Pupil: 5, Tutor-Pupil: 3
44
9
S 5
(d) Pupil: 6, Tutor-Pupil: 4
(e) Pupil: Tutor-Pupil: 5
(f) Pupil: 4, Tutor-Pupil:
7 7
4
6
4
9
(g) Pupil: 9, Tutor-Pupil: 7
(b) Pupil: 4, Tutor- Pupil: 8
(i) Pupil: 4, Tutor-Pupil. 9
Figure 8: Each subfigure shows 3x2 grid with original image (top left), decoded image (top right), SHAP explanations for correct label (middle row), and SHAP explanations for label output by the Pupil (bottom row).
19


Reviews:
[{"review_id": "ZIT9bqr5H8", "reviewer": "Reviewer_SNr4", "summary": "The paper suggests a new form of interpretability approach without compromising expressivity. It is a common practice to use a simple interpretable model as a primary model and use a second model for handling the residual error for performance reasons. But interpretability is usually lost in the complexity of the second model. \nThis paper proposes an alternate. They use the second model instead to generate input perturbations such that the primary model is more accurate on the perturbed input. Assuming input perturbations are interpretable, the paper argues that their approach improves performance without hurting interpretability.", "questions": "1. If the pupil model is complex, what's stopping the tutor from selecting meaningless perturbations? I.e., how is the interpretability of perturbations enforced?   \n2. The requirement of latent space and pupil model inference with modified inputs could compromise performance due to train-test distribution for pupil model and other reasons. When do you expect far worser performance (than a non-interpretable base model) with your approach?              \n3. Please elaborate your differences from counterfactual explanations.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "* Clean presentation. I enjoyed reading the paper.  \n* Neat conceptual difference. Modeling input perturbation instead of residual output error is a clean conceptual differential from earlier work. \n* In the case of MNIST modeled with logistic regression, improving accuracy through augmentation and output correction could not have led to the explanations the paper demonstrated in Figure 6. MNIST dataset, although simple, supports their claim of improved accuracy and interpretability with their approach.", "weaknesses": "* Empirical validation. The validation in the paper looks preliminary. It requires validation with far more complex datasets to be taken seriously. For instance, CheXpert [1] or some WILDS [2] datasets.   \n* The paper assumes input edits are model-able and interpretable. The interpretability aspect is only assumed without validation. \n\nReferences   \n[1] https://www.nature.com/articles/s42256-021-00338-7\n[2] https://wilds.stanford.edu/datasets/", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4EL0iEYSXj", "reviewer": "Reviewer_Z1yU", "summary": "This paper introduces the Tutor–Pupil augmentation framework, a general approach to improving both model performance and interpretability through minimal input-level corrections. The Pupil is a fixed, interpretable or task-specialized model (e.g., decision tree, physical law, logistic regression), while the Tutor is a flexible neural network trained to apply small corrections to the inputs such that the Pupil produces more accurate outputs. This setup enables the Tutor not only to enhance predictions but also to diagnose failure modes of the Pupil by revealing where and how input perturbations correct errors. The framework is demonstrated on three diverse cases: (1) augmenting a decision tree for a toy binary classification problem, (2) refining the ideal gas law to account for non-ideal behaviors—discovering van der Waals-like corrections, and (3) improving handwritten digit classification via a VAE-based latent-space Tutor acting on a logistic regression Pupil, which also provides interpretable visual corrections. The results show notable gains in accuracy and interpretability across tasks, with meaningful parallels to symbolic regression and explainable AI.", "questions": "How sensitive are the Tutor’s corrections to hyperparameters such as λ (correction magnitude regularization)?\n\nCould the framework handle non-differentiable Pupils (e.g., rule-based systems) at scale?\n\nDoes the learned correction vector generalize across data distributions or must it be retrained for each Pupil or dataset?\n\nHow can one quantify interpretability improvements beyond visual inspection (e.g., through user studies or explanation fidelity metrics)?\n\nCould the Tutor–Pupil setup be extended to adversarial tutoring, where the Tutor exposes brittleness or bias in the Pupil rather than assisting it?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is conceptually novel and elegant, offering a unified and interpretable augmentation scheme applicable across interpretable and black-box models. Its theoretical framing—training a Tutor to apply minimal, semantically meaningful corrections—is both intuitive and powerful. The work’s breadth, spanning interpretable (decision trees), physics-based (ideal gas law), and data-driven (MNIST classification) settings, convincingly demonstrates generality. The analyses are rigorous, supported by visualizations. The MNIST experiment is particularly compelling: the Tutor’s subtle adjustments (e.g., closing loops or clarifying strokes) both enhance performance ( and produce human-readable explanations that outperform conventional attribution maps. The idea of deriving physically meaningful corrections from learned input perturbations is especially original and promising for scientific ML applications.", "weaknesses": "In the MNIST setting, the performance jump (91%→98.5%) could partly result from the use of a VAE-trained latent representation rather than purely from the Tutor’s corrective effect. \n\nAdditionally, the paper could better differentiate its contributions from related ideas like counterfactual explanations, residual learning, and gradient-based input attribution methods. \n\nFinally, the experiments, while creative, are small-scale; a larger empirical evaluation would strengthen the claims of robustness and general utility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MSYazkXUn7", "reviewer": "Reviewer_YX96", "summary": "In this paper, the auxiliary model in the model augmentation framework is used to boost prediction accuracy and interpretability. In a Tutor-Pupil scheme, the Pupil is used to learn according to the domain-specific features, while the Tutor adding a small perturbation to the input of Pupil, is used to uncover the specific failure modes and regions of uncertainty with the Pupil's predictions. In physics-informed models, higher-order global and structural information in the data space and decision boundaries is revealed leading to better modeling of the observations and the explanation of the shortcomings of the reference dynamical processes.", "questions": "Have the authors checked the robustness of the Pupil-Tutor framework? \n\nFamously, [Goodfellow'14] showed that the addition of a small but unstructured noise to the input leads for the change in the class, but not a visible difference in the image itself. In comparison addition of the noise in the latent space would likely translate to a visually significant changes in the image after decoding, but may still not be interpretable as the encoder compresses the image through entangling the meaningful features. As the results of the MNIST experiment are not very conclusive to me, can the authors explain how the Tutor perturbations are interpretable and different from a random cohesive structured blob?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is well written and easy to follow. The authors provide a sound a good background for the problem of interpretability-complexity trade-off of the modern architectures used in ML.\n\nThe authors propose a Pupil-Tutor framework to boost the performance and interpretability of simple architecture or numerical models. This method works for physical models and can be used to modify them according to the observations for better understanding of the underlying processes. \n\nThe authors provide other examples in the image datasets where the Tutor improves the prediction performance of a non-interpretable using this architecture.", "weaknesses": "The scope of interpretability of the Tutor is limited to our understanding of the perturbations in the data space. The results are constrained to synthetic 2d data, 3 variable time independent physical system, which are too simple to understand the potential of the proposed architecture, and logistic regression on MNIST dataset which in I believe had vague inconclusive results on diagnostics and interpretability of the failure modes of a non-interpretable model.\n\nThis might be due to the nature of the Pupil perturbations that are performed in data space and therefore the interpretability is left to human understanding of the discernible features in the data space, which is another complex task. While portrayed as a tool for interpretability, I believe this framework is yet ineffective and cannot improve interpretability as well as e.g. [Sarvmaili'24], where a set of representative samples are produced which can be used to understand the main modes of failure from training data. The authors don't report any evaluation metrics on the interpretability of the predictions especially for high dimensional datasets. A more comprehensive study can be performed e.g. with 3dshapes dataset where the main features are clearly discernible in the data space, or applying known transformations to the MNIST dataset and training the Tutor to undo said transformations could lead to more conclusive results.\n\n[Sarvmaili'25] Data-centric Prediction Explanation via Kernelized Stein Discrepancy, Sarvmaili, Mahtab and Sajjad, Hassan and Wu, Ga, ICLR 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E9IohE0sTy", "reviewer": "Reviewer_4yEq", "summary": "This paper addresses the long-standing \"performance-interpretability trade-off\" in machine learning—where simple, interpretable models (e.g., decision trees, physics-based formulas) lack expressive power, while complex black-box models (e.g., neural networks) sacrifice transparency. It proposes a novel Tutor-Pupil augmentation framework to resolve this trade-off by leveraging \"minimal input-space corrections\" rather than output adjustments, enabling both performance gains and enhanced interpretability.", "questions": "see weakness", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Unlike prior work that corrects outputs (e.g., residual networks, ensemble stacking), this paper corrects inputs, preserving the Pupil’s interpretability.\n\n- The paper’s originality lies in redefining the paradigm of model augmentation, removing limitations of prior work, and creating novel links between data-driven learning and theoretical insight—all of which challenge long-standing practices in interpretable AI.\n\n- The paper does not limit the Tutor-Pupil framework to a single task type but adapts it to three distinct domains—a creative extension that proves its generality.", "weaknesses": "- The paper strictly adopts a \"train Pupil first, then train Tutor\" serial paradigm (Pupil parameters are frozen during Tutor training but fails to explore joint training—a critical gap that limits the framework’s ability to fully leverage synergies between the two models and may amplify Pupil’s inherent flaws.\n\n- Novelty Gap: “Input-space correction” is not new. Position the paper as “systematic, global counterfactuals for interpretable models” rather than a brand-new paradigm and provide a taxonomy table that shows how Tutor-Pupil differs from (i) local counterfactuals, (ii) adversarial examples, (iii) data-augmentation policies on objectives, constraints, and evaluation metrics.\n\n- The paper validates the framework exclusively with interpretable Pupils (decision trees, logistic regression, ideal gas law but fails to test black-box Pupils (e.g., ResNet, Transformer)—a critical gap, as many real-world systems rely on complex models that need interpretive tools (e.g., medical image classifiers using CNNs).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)