Missingness Bias Calibration in Feature Attribution Explanations
Authors: Shailesh Sridhar
Anton Xue
Eric Wong
Venue: ICLR 2026 Poster (2026)
Keywords: explainability
feature
attribution
calibration
missingness
bias
medical
medicine
LLMs
Machine Learning
Primary area: interpretability and explainable AI
TL;DR: Model calibration can fix missingness bias in feature attribution explanations

Abstract: Popular explanation methods often produce unreliable feature importance scores due to missingness bias, a systematic distortion that arises when models are probed with ablated, out-of-distribution inputs.
Existing solutions treat this as a deep representational flaw that requires expensive retraining or architectural modifications.
In this work, we challenge this assumption and show that missingness bias can be effectively treated as a superficial artifact of the model's output space.
We introduce MCal, a lightweight post-hoc method that corrects this bias by fine-tuning a simple linear head on the outputs of a frozen base model.
Surprisingly, we find this simple correction consistently reduces missingness bias and is competitive with, or even outperforms, prior heavyweight approaches across diverse medical benchmarks spanning vision, language, and tabular domains.

MISSINGNESS BIAS CALIBRATION IN FEATURE ATTRIBUTION EXPLANATIONS
Shailesh Sridhar
University of Pennsylvania thai2403@seas www.aa.supeeen..eem
Anton Xue
University of Texas at Austin anton. xue@utexas edu
Eric Wong
University of Pennsylvania exwongêseas. upenn. edu
ABSTRACT
Popular explanation methods often produce unreliable feature importance scores due to missingness bias, a systematic distortion that arises when models are probed with ablated, resentational flaw that requires expensive retraining or architectural modifications. In this work, we challenge this assumption and show that missingness bias be effectively treated superficial artifact of the model's output space. We intro- duce MCal, lightweight post-hoc method that corrects this bias by fine-tuning a simple linear head on the outputs of frozen base model. Surprisingly, we find this simple correction consistently reduces missingness bias and competitive with, or even outperforms, prior heavyweight approaches across diverse medical benchmarks spanning vision, language, and tabular domains.
inputs. Existing solutions treat this deep rep-
INTRODUCTION
As black-box deep learning systems are increasingly deployed in high-stakes settings such as medicine, finance, and law, there is increasing demand for reliable and trustworthy model expla- nations. A common approach for explaining model predictions is to use feature attribution methods, which assign importance scores input features based on their influence methods, such as LIME (Ribeiro et al., 2016) and SHAP (Lundberg & Leel 2017).. estimate these scores by perturbing the input, typically by ablating selected features and measuring the change in prediction. Because true feature removal often infeasible (e.g., one cannot physically delete im- age pixels or omit words from tokenized sequences), attribution methods approximate removal by substituting the selected features with default or placeholder values, such as black pixels or special tokens (Ancona al. 2017: Sundararajan al. 2017).
the output. Popular
These substitutions often result in out-of-distribution inputs that deviate significantly from the model's training data, inducing a systematic distortion in predictions known as missingness bias (Hase et al. [202]] Hooker et al.] [2019] Jain et al.) (022). Such bias can severely undermine the reliability of explanations. As illustrated in Figure a classifier that accurately detects a brain tumor from clean inputs fails to do so when irrelevant features are masked, demonstrating how seemingly innocuous ablations can corrupt model behavior. Since perturbation-based attributions are derived directly from these corrupted predictions, their reliability is fundamentally compromised, leading to inconsistent feature importance scores (Duan et al. 2024 Goldwasser & Hooker 2024 Hooker let al. 2019). This also opens the door to adversarial manipulation: malicious actors can exploit this vulnerability to design deceptive models that obscure their use of sensitive attributes such as race or
gender (Joe al. 2022
Koyuneu et al. 2024 Slack et al. 2020).

---

Tumor /
Tumor V
Healthy X
Figure 1: Removing irrelevant features can cause a misdiagnosis. A fine-tuned ViT (Dosovitskiy al. 2020) correctly predicts "tumor" the clean image (left) and subset of the relevant features (middle). However, masking irrelevant features flips the prediction to "healthy", despite the tumor remaining visible (right). For visualization, gray stripes denote zero-valued pixels, and images are contrast-booste booste...
variety of mitigation strategies have been proposed to address missingness bias. Replacement- based methods aim to reduce distributional shift by imputing masked features with more realistic
content (Agarwal & Nguyen [2000] Chang et al. 2018 Kim et al., 20200 Sturmfels et al.) 2020
Training-based methods fine-tune retrain the model better handle ablations (Hase al. 2021
Hooker et al. 2019] Park et al. 2024 Rong et al. 2022) while prchitecture-basee approaches embed
robustness directly into the model via structural design changes (Balasubramamian Feizi 2023
Jain et al. 2022).
However, these strategies are often impractical. Replacement-basee methods are usually specialized to specific domains (e.g., text (Kim et al.) (0200) or might require training model-specine imputa- tions (Chang et al.] 2018). On the other hand, training-based
based solutions require intensive engineering modifications require deep understanding of model internals. Moreover, it is also increasingly common that models are complete black boxes, such when interacting with API-based L.LM providers.
and computing resources, while architecture-base
In this work, we question whether such complex interventions are necessary. We investigate simple yet surprisingly powerful strategy for mitigating missingness bias: finetuning linear head on the outputs of a frozen base model. This approach, which we call MCal, is lightweight, model-agnostic, and post-hoc. It is significantly cheaper in implementation effort than training-basse -based methods, does not require model-specific adaptations like architecture- based and replacement-baae based methods, and only needs access the model's 's output logits. In the following. we summarize the development of MCal and our contributions.
New Perspective on Missingness Bias. We find that missingness bias, a problem often treated as deep representational flaw, can be effectively mitigated with simple post-hoc correction in the model's output space. This finding suggests the bias is often superficial artifact, challenging the prevailing assumption that expensive retraining architectural modifications are necessary.
Lightweight Method with Theoretical Guarantees. We formalize this approach as MCal, lightweight calibrator that highly efficient to optimize (Section 3). Furthermore, our simple formulation provides theoretical guarantees of convergence to a globally optimal solution, ensuring a level of stability and reproducibility rare for deep learning interventions.
A Strong and Practical Baseline. We demonstrate MCal's effectiveness across diverse models and data modalities, where often competitive with heavyweight approaches (Section 4 This establishes strong and practical baseline that can be immediately adopted by researchers and prac- titioners to improve the reliability of their explanations.
2 UNDERSTANDING MISSINGNESS BIAS
Perturbation bassd feature attribution methods like LIME (Ribeiro et al. 2016) and SHAP (Lund- berg & Lee 2017) evaluate models inputs with ablated features, typically replaced by fixed
2

---

Clean
60%
Predictions are balanced when inputs are clean
Inputs
40%
20%
Healthy
Tumor
80%
Ablated
60%
Predictions skew "healthy" when tumors are visible
Inputs
40%
20%
Healhy
Figure 2: Feature ablations induce class distribution shifts. Masking non-critical regions skews predictions towards the "healthy" class, when tumors remain visible. This effect, known as missingness bias, causes the model to misclassify inputs that retain relevant features, and undermines the reliability of feature attribution explanations.
baseline values (e.g., zero-vectors or mean-pixel values). However, because these synthetic inputs often fall outside the model's training distribution, they can induce systematic prediction distortions, phenomenon known missingness bias. This section provides background on this bias and its consequences for explanation reliability.
2.1 PATHOLOGY: SYMPTOMS AND MEASUREMENTS
The effects of missingness bias are not merely statistical curiosities; they manifest tangible fail- ures that undermine the reliability of explanation methods.
Systematic Skew in Predictions. The most direct failure mode of missingness bias is a systematic skew in model predictions (Jain et al., 2022). As illustrated in Figure 2 the model's accuracy degradation is not random but systematic: it develops a consistent bias towards one class (in this case, "healthy") even when the core evidence for the correct class remains visible. This failure mode is particularly pernicious, persisting even when we selectively avoid masking the central image patches most likely contain the tumor.
Unreliable Feature Attributions. Another consequence of this degraded accuracy is that any fea- ture attributions derived from the model fundamentally unreliable. If model's predictions incorrect on ablated inputs, the importance scores computed from these predictions cannot be trusted reflect the model's true reasoning. Empirical findings support this; for instance, Jain et al. (2022) show that feature importance scores from models with high missingness bias fail standard robust- ness tests such as top-k removal. Prior work has also shown that minor changes the ablation process can yield vastly different explanations, suggesting they reflect perturbation artifacts rather than genuine model logic (Hooker et al. 2019).
Quantifying Missingness Bias. Many feature attribution methods operate under the assumption that feature ablation is a neutral act of intervention intended to simulate the removal of informa- tion (Sturmfels al. 2020 (Sundararajan this expected neutrality. the resulting shift in its aggregate predictive distribution serves direct measure of missingness bias. This shift is typically quantified as the distribution shift between the class frequencies on the clean data distribution D versus the ablated data distribution DD' (Balasub-
al.) 2017). When a model's behavior deviates from
ramanian Feizi 2023 Jain et 2022
MissingnessBias(f) DKL. E. '''D
Class(f(z'))
E Class(((z))) a>D
where D' is the distribution of inputs where each feature i.i.d. ablated with some given probability, and let (lass(f(z)) be the one-hot vector representation of the class predicted by f on The above can then understood as a measure information-theomm biased ablations, supposing only knowledge of its behavior clean inputs. In particular, JJiin al.
"surprise" when f is evaluated on un-
3

---

Ablated Input
MCal (Calibrated Classifier)
Ro
Healthy: 70% x
Healthy: 40% Tumor: 60% /
Tumor: 30%
Figure 3: MCal corrects class distribution shifts induced by input ablations. The model initially predicts "healthy" from the ablated input. MCal applies learned transformation Rg adjust the output probabilities, thereby restoring alignment with expected class distributions. This calibration method is model-agnostia requiring only the classifier's output probabilities of each class.
specifically introduces this to measure missingness bias, rather than of adjacent phenomena, such as prediction sensitivity with respect to top-k feature selections Hase et al. 2021
2022
2.2
THE CHALLENGE OF MITIGATION
variety of strategies have been proposed address missingness bias, which can be broadly cate- gorized as follows:
Replacement-baseee These methods aim to make ablated inputs appear more in nn-distrbbttin. Beyond simple values (e.g., zero and mean-valued (Hase al., (001]), more complex variants include marginalization, which averages outputs over plausible replacement values (Chirkoval et al. 2023; Frye et al.] 2020: Haug et al.] 2021 Kim et al., 2020, Vo al. [2024), random noising (Rong et al. 2002),, and generative modeling, which uses secondary model to in-paint realistic content (Agarwal & Nguyen 2020; Chang et al. 2018). However, these approaches are often complex and can introduce their own artifacts.
Training-based, This approach treats feature ablations as a form of data augmentation. Methods like ROAR (Hooker et al.] 2019) and GOAR (Park et al.) 2024) retrain fine-tune the model on masked inputs to align its train and test distributions. Although effective at building robust representations, this strategy is computationally expensive and only possible when the model can be modified.
Architecture-bass based. These methods embed robustness directly into the model's design. For ex- ample, modified vision transformers (Dosovitskiy et al. 20200] Jain et al. 2022) and CNNs (Bal- asubramanian & Feizi| (023) can be altered to use dedicated mask tokens explicitly suppress the influence of ablated regions. However, these changes are often non-trivial, architecture- specifie, and not generalizable.
While often effective, the high cost and complexity of these methods make them impractical for many modern use cases, especially those involving large-scale, pre-trained foundation models. Fur- thermore, such approaches are entirely infeasible when working with API-based models that do permit retraining or architectural changes. This gap highlights the need for practical, lightweight, and model aggnosic approach mitigating missingness bias that we introduce next.
3 MCAL: A LIGHTWEIGHT CALIBRATOR FOR MISSINGNESS BIAS
Having established the pathology of missingness bias and the practical limitations of existing heavy- weight solutions, we now introduce our method. We propose MCal, lightweight, post-hoc correc- tion that is surprisingly effective at mitigating missingness bias.
3.1 ARCHITECTURE AND OPTIMIZATION
The calibration process is illustrated in Figure 3 A base classifier f R² - R first processes an input x to output the raw logits fax) calibrator Rs Tm - Tm then transforms the raw logits into the calibrated logits Ro(z). Specifically, we implement this as an affine transform:
RØ(=) Wz +
4

---

where the calibrator is parametrized by 8 == (W,b), with W Rmxm and R". To fit the cali- brator, we use a standard cross cross-entropp entropy objective that aligns the calibrated prediction on an ablated input with the base model's prediction on the clean input:
(())
E (m
CrossEntropy (Rarrrooppo[[(((aaaaaaa (x') Class(/(<))):
where (I,x') ~ 'D are samples of a clean input x and its ablated version x', and (Iass( denotes the one-hot prediction the clean input.
Our approach is deliberately minimalist, prioritizing efficiency without compromising performance. We apply standard cross-entropy objective, identical that used in heavyweight retraining meth- ods (Hooker et al. 2019), but only to a lightweight matrix -scring calibrator (Guo et al. (017)) This design highly efficient, with orders of magnitude fewer parameters (m² + m) than fine-tuning or even parameter-efficiee methods like LoRA (Hu et al. (022). Our experiments in Section 4 con- firm that this minimalist approach in fact, sufficient to yield competitive performance with more caaneerrinn-inenniii approaches like retraining the model or architecture modifications. Further-
more, this simple design also comes with strong theoretical guarantees on its optimization process, which we detail next.
3.2 THEORETICAL GUARANTEES AND GEOMETRIC INTERPRETATION
Our affine parametrization of Ro means that standard gradient-based optimization will provably converge to an optimal solution, which we formalize as follows.
Theorem 3.1 (Guaranteed Optimal Convergence). The MCal objective [(0) is convex in 0.
Proof. The function [()) is convex in 0, as it is composition of the convex cross-entropy loss and an affine transformation. Because local minimums are also global minimums for convex functions, standard gradient-base based optimization (e.g., SGD, Adam) will converge an optimal solution.
The importance of this guarantee twofold. First, it ensures reproducibility and stability: the optimization process is guaranteed to converge to the same optimal solution, reducing the need for extensive hyperparameter sweeps or random seed searches. Sccond, it provides a strong assurance of quality, guaranteeing that the resulting calibrator is a globally optimal affine correction for the given data.
Geometric Interpretation. MCal also has a clear geometric interpretation, visualized in Figure 4 The uncalibrated outputs form biased point clouds on the probability simplex, with the Class A cluster pulled towards the Class B vertex, leading systematic misclassification. MCal learns an optimal affine transformation in the logit space that rotates, scales, and shifts these distributions.
This untangles the clouds and pushes them towards their correct vertices. Theorem 3.1 guarantees that this correction is globally optimal for our parametrization.
3.3 IMPLEMENTATION CONSIDERATIONS
Conditioning on Ablation Rates. Our experience shows that the severity of missingness bias is strongly correlated with the fraction of features that are ablated. To account for this, we recommend using an "ensemble" of specialized calibrators, each one fit for specific ablation rate (e.g.. 10%, 20%, etc.). At inference time, we apply the calibrator that was trained for the ablation rate closest to that of the input. We study the advantage of this ensemble in Section
Integration with Explainers. As a post-hoc wrapper, MCal compatible with any perturbation- based explanation method. The calibrated model, f. can be used as a drop-in replacement for the original model, f, in any existing explainability pipeline. The resulting feature attributions are then generated from a model that has been explicitly corrected for the missingness bias induced by the explanation method's perturbation strategy.
5

---

Class C
Class
MCal
Class A
Class B
Class A
Class B
Uncalibrated Accuracy: 59.33%
Calibrated Accuracy; 93.00%
Figure 4: Geometric intuition of MCal a synthetic dataset. Missingness bias causes the uncal- ibrated outputs to shift. For instance, the Class A cluster (blue circles) is pulled towards the Class vertex, leading to systematic misclassification and low accuracy. MCal applies an optimal affine transformation to the uncalibrated outputs, correcting the shift and improving accuracy.
Training Set Size and Overfitting. Dense paramctrizations of W risk overfitting when the num- ber of parameters exceeds the number of training samples (Guo et al. (017). which can occur when there are many classes. In such cases, the training loss may go to zero while test performance does improve. We recommend two strategies to mitigate overfitting. First, one may consider adding regularization term to the objective. Second, one may also consider sparse parametrizations, such as taking W to be a diagonal matrix (also known as "vector-scaling", which would reduce the total parameter count (m).
EXPERIMENTS
We now present experiments validate the impact of missingness bias in explainability, well as the ability of MCal to mitigate it. Moreover, we demonstrate that MCal repeatedly outperforms more expensive baselines, such as full retraining and architecture modifications, Additional details are given in Appendix
Models, Datasets, and Compute. We evaluate on diverse set of medical benchmarks that span vision (Brain MRI (Nickparvar) 2021), Chest X-ray (CheXpert) (Irvin et al., 2019), and Breast Can- cer Histopathology (BreakHis) (Spanhol et al. 2015). ), language (MedQA (Jin et al. 2021). MedM- CQA (Pl et al. 2022)) and tabular domains (PhysioNet (Haug al. 2021 Breast Cancer (Wolberg et al. 1993). Cardiotocography (CTG) (Campos & Bernardes] 2000)). We respectively evaluate on these domains with ViT-B16 (Dosovitskiy et al.) (020), Llama-3. 1-8---3----B--Ittreuc (AT@ Meta (024)) and XGBoost (Chen & Guestrin 2016),, which are trained using standard methods. For compute, we had access to a machine with four NVIDIA H100 NVL GPUs.
Input Ablations and Calibration. We say that an input x R" has ablation rate p //n l of its features are ablated. To evaluate on a tractable range of p. we use (0/16, 1/16,..11......... 15/16] for vision, p (0/10, 1/10,... 9/10} for language, and p E 1/10,,
9/10} for tabular, where recall that we recover the clean input at p = O. For imputations, we use zero-valued (black) patches for vision, we replace whitespace-separater words with the special string UNKWORDS for language, and we perform mean imputation for tabular data. For vision specifically, we select patches to ablate, regardless of their original values (e.g., some MRI images already have black patches). Following discussion from Section 3.3 the unconditioned calibrator where each feature was uniformly ablated with probability 1/2, whereas the conditioned ensemble has calibrator fit at each value of p. All calibrators optimized using Adam (Kingma & Ba 2014 with a learning rate of 10-3 for 5000 steps.
fit inputs
Do calibrated models lead to better explanations? Missingness bias is known to skew the explanation quality of feature attribution methods (Jain et al. 2022). To that end, we sider how two representative methods. LIME (Ribeiro et al. (016) and SHAP (Lundberg Lee 2017), perform calibrated uncalibrated models. These methods output ranking of each in-
Question 1:

---

Sufficiency
MCal
Sufficiency
10-
Base MCal
Sensitivity
10
Sensitivity
10
Base MCal
Base
2
4
8
10
12
4
6
8
10
2
4
6
10
12
2
10 12
Top-K
SHAP Top-K
LIME Top-K
SHAP Top-K
Figure 5: Calibrated models have better explanations. Compared to uncalibrated bascline model (Base), LIME and SHAP explanattns on MCal-calibrated models have more accurate feature importance scores (sufficiency 4). In addition, calibrated models also robust to feature ablations (sensitivity 4). Results are shown for the MRI dataset using an unconditioned calibrator.
10a
10
Cond
10-1
Miss. Blas
10-2
Uncond
Miss. Bias
10-2
Miss. Bias
10-2
10-3
10-1
10
10-4
Cond
Cond
10-1
Uncond
10
Uncond
8/16
2/10 4/10 6/10 8/10
4/10 6/10 a/10
MRI Ablation %
MedQA Ablation
PhysioNet Ablation %
Figure 6: Conditioning on ablation rate improves MCal. Fitting an ensemble of calibrators at discretized set of ablation rates help reduce the overall missingness rate, compared to using single unconditioned calibrator. (Left) MRI, (Middle) MedQA, (Right) PhysioNet.
put feature's importance the model, which we evaluate using the standard sufficiency metric (Hase et al. 2021), detailed in Appendix [.]] Informally, sufficiency measures whether the features iden- tified as important are enough on their own to maintain the model's original prediction confidence (lower values indicate higher quality ranking). We report results in Figure 5
Question 2: How does calibration affect model robustness? It is generally desirable for models be robust to feature perturbations, this can improve generalization and reduce the risk of ad- versarial behaviors. To that end, we measure the robustness of the underlying model to the removal (ablation) features via the sensitivity metric, detailed in Figure We show our results in Ap- pendix A.I which shows that the model is not overly dependent on its top-k features for prediction.
Question 3: What is the impact of conditioning on feature ablation fractions? Rather than fitting a single calibrator, we observe that using an ensemble of calibrators, each conditioned upon a single fraction (ablation rate), can improve performance. We compare the performance of this con- ditioning in Figure 6 where we observe an improvement in performance over an unconditioned calibrator. This is expected, as a model's missingness bias known to vary with the ablation rate (Hooker et al.] [0199] Jain et al. 2022). and an ensemble thereby allows each calibrator to specialize to their respective rates.
Question 4: How does MCal compare to the baselines? We compare MCal each of the fol- lowing prior approaches, which have all been employed previous work to combat the problem of out-of-distribution inputs.
Base: This the unmodified, uncalibrated classifier that as a reference baseline.
Replacement-based (Replace): Our implementation of replacement-based mitigation is in- spired from Hase et al. (202T). particular, for vision, we use the channel-wise mean pixel value of the clean dataset (Carter et al.) 2021). For language, drop tokens from the sequence so that the ablated token sequence is shorter in length than the clean one (Hase et al. 202[). For tabular, perform mean imputation.
7

---

Dataset
Base
Replace
Retrain
Arch
TempCal
PlattCal MCal
Brain MRI
1.18-1
e-
6.70
140e-1
1.16e1
1.27e-1 7.43e-3
Vision
CheXpert
e.70e-1
9.70e-2
e-67e-2
1.50-11
1.65-11
2.02e-1
8-82
BreakHis
e-
4.20e-1
2.19e-2
1.54-1
1.80-1
1.66e-1
4.29e-3
MedQA
e-
1.50e-1
1.70e-
2.68e-2
e-
9.48e-2
9.41e-
Language
MedMCQA
e-
e-59
1.52e-1
1.40-1
7
1.13e-1 e-0
PhysioNet
1.17e-1
1.20e-1
5.59e-3
8.1e-2
1.17e-1
1.19e-1
e-01
Tabular
Breast Cancer
e-02e-1
1.4e-1
e-3
2.13e-
1.02e-1
e-9 e-20
1.08
CTG
1-06e11
e- 6-61e-3 e-85e1 1.06e-1
e-
Table 1: MCal is an effective and lightweight way reduce missingness bias. It repeatedly outperforms more computationally expensive baselines, such as retraining and architecture modifi- cation. We report the KL divergence- based metric in Equation (T).
1.0
Accuracy
0.8
Accuracy
0.8
Accuracy
0.6
0.6
0.8
Cond
Cond
Cond
Uncond
Uncond
0.6
Uncond
Base
Base
Base
0.0
0/16
4/16
8/16
12/16
0/10 2/10 4/10 6/10 8/10 MedQA Ablation
0/10 2/10 4/10 6/10 8/10 PhysiaNet Ablation %
MRI Ablation
Figure 7: MCal does not harm classifier accuracy. Across different ablation levels, the accuracy of uncalibrated vs. calibrated classifiers is comparable. This also holds when the input clean (ablation fraction zero). (Left) MRI, (Middle) MedQA, (Right) PhysioNet.
Training-based approaches (Retrain): Models fine-tuned ablated inputs, where each feature (patch, token) is uniformly ablated with probability 1/2.
Architectural-based (Arch): We perform a non-trivial modification of ViT to accept attention masks as in Jain et al. (2022). For models with architectural support for missing features, use those: c.g., attention masking in Llama-3 and native support for NaN in XGBoost.
Standard calibration (TempCal, PlattCal): We additionally consider existing calibration- based methods from literature, particularly temperature (TempCal) and vector-scaling Platt cal- ibration (PlattCal), as described in Guo et al. (2017).
We report in Table T the average of values from the ensemble of conditioned calibrators. We found that MCal is often superior even to computationally and engincering intensive basclines, such as model retraining and ViT architecture modifications. observe that MCal outperforms both temperature and Platt calibration. Replacement based methods have inconsistent performance, which aligns with known observations on their sensitivity to impu- tation values. Finally, we note that architecture-natiin support for missing features may, in fact, exacerbate missingness bias, as seen in XGBoost on the Breast Cancer and CTG datasets.
support of our earlier claims, we also
Question 5: How does MCal affect classifier accuracy? MCal fundamentally alters a pretrained base classifier f into f. which then deployed to downstream applications. Importantly, the aceu- racy of must remain high, even when it is optimized on ablated images Equation (3). We show in Figure 7 that this is indeed the case: we compare the uncalibrated base model against both ab- lation rate- conditiidd and unconditioned calibrators. We observe that both forms of calibration improve classifier accuracy all ablation rates, where recall that the clean image is obtained at an ablation rate of zero. Aligning with earlier findings, we see that the conditioned calibrator outperforms the unconditioned calibrator.
8

---

5 RELATED WORK
Missingness Bias in Explainability. Missingness bias (Jain et al. 2022) denotes the systematic distortions that arise when attribution methods "remove" features via ablations, e-g., with black pixels, zero-valued embeddings, or special [MASK] tokens. Such ablated inputs are often out-of- distribution with respect to the model's training distribution, which can result in erratic predictions, inflated confidences, and unstable feature importance scores (Hooker et al.) [219 Vo et al.) 2024).
In particular, importance scores can vary drastically with the chosen replacement technique (Haug et al., 2021 Sturmfels et al.] (020) and even be exploited adversarially (Slack et al.] 2020). Consequently, feature-based explanations commonly reflect ablation artifacts rather than genuine model reasoning (Hase et al. 2021), which risks eroding trust in high-stakes settings. In addition to the methods described earlier in Section 2.2 there are several benchmarks related to missingness
bias (Duan al. 2024
Hesse et al. 2023 Liu 2021).
Calibration Methods. A calibration method post-hoe rescales the logits probabilities of model prediction without modifying the underlying model weights. Classic techniques include bin- ning (Zadrozny & Elkan 2001). Platt scaling (Platt al. 1999), and temperature scaling (Guo al.) 2017). This is often used to improve and calibrate model predictions under input distribution shift, such in autonomous driving (Tomani al., 202T). healthcare (Shashikumar et al. 2023 and LLMs (Kumar et al. 2022). To our knowledge. however, calibration for missingness bias is novel.
Robust and Reliable Explanations. There is much interest in the development of robust expla- nations for machine learning models. Notable efforts include the development of benchmarks for explanations, particularly feature attribution methods (Adebayo et al.] 2018 [202]] Agarwal et al.
2022
Dinu et al., 2020 Duan et al.
2024
Havaldar al. [2025] Jin et al. [222]] Kindermans al.
2019 Nauta et al. 2023 Rong et al.
2022 Zhou et al. (022). There is also interest in formally cer-
tifying explanations (Bassan & Katz
2023 |assan et al.) 2025a b Jin et al. 2025 Lin et al. [0033
Xue et al., 2023] You et al. 2025).
Other efforts, such as this work, involve adapting classifiers to
be more robust to input ablations in feature attributions.
6 DISCUSSION, FUTURE DIRECTIONS, AND CONCLUSION
Calibration Design. While other calibrator parametrizations are viable, any non-convex parametrization of the objective risks losing guarantees of optimality convergence. In turn, this risks introducing undesirable behavior, such as sensitivity to the initialization of calibrator parame- ters. Additionally, observe that the measure of missingness bias (Equation (T) is different than the calibrator optimization objective. This is because the missingness bias measure is not differentiable due to the one-hot Class function, which motivated us to search for reasonable alternatives, e.g., the standard cross-entropy objective in classification. While it would be interesting to explore, for instance, differentiable relaxations of Equation I we leave this to future work.
Missingness Bias and Data Variability. Visual medical datasets, e.g., Chest X-rays, often exhibit lower variability than image datasets like ImageNet (Deng et al., 2009). Despite this, missingness bias can persist in models trained on datasets such as ImageNet (Jain et al. 2022). However, it is not known how missingness bias varies as function of both dataset variability and model architecture.
Beyond Explainability. Missingness bias is a fundamental risk when evaluating feature subsets on a model that is not explicitly designed to handle missing data. While we are primarily motivated by challenges in explainability, this work has broader applications. In vision. model evaluation with masked images a standard practice. In language modeling, a token's 's embedding is often dependent its position, meaning that ablations are senttiiveee whether via the attention mask, subsetting the input sequence, or replacement with special [MASK] tokens.
Limitations. MCal requires access to a collection of clean and ablated prediction logits, which may not always be available, such for some API-based LLMs. Even then, gradient-based opti- mization is only guaranteed to converge to global optimality under certain parameterizations of the calibrator. Overfitting is also a potential risk, particularly in settings with large number of possible

---

classes (e.g., language model's vocabulary size), in which case regularization is warranted. Fur- thermore, MCal is only intended to mitigate missingness bias, and other forms of bias in the model and data may still be propagated. While our experiments show that linear corrections suffice to mitigate missingness bias, and hence our use of the description "superficial", it may be the case that for missingness bias in certain model classes, it is harder mitigate in this manner.
the logits
Future Work. One direction investigate the theoretical guarantees and empirical performance of different calibrator parametrizations such as one-layer feedforward network instead of an affine transform. Another extension is broaden our study on the performance of calibrated classifiers in explainability, such with respect to the explanation methods and metrics surveyed in Section [3] It would be interesting to explore methods for mitigating missingness bias when prediction logits are not available, common restriction for API-based large language models. Additionally, the idea of calibration may also extended to other instances of domain shift and out-of-distribution inputs, which are prevalent throughout Machine Learning literature
Conclusion. Missingness bias threatens the reliability of popular explanation methods and tech- niques, a problem magnified by the increasing impracticality of existing angineering-intessie solu- tions. To overcome this, we introduce MCal, lightweight calibration method that requires only collection of clean and ablated prediction pairs. We demonstrate that a simple, affine parametriza- tion of the calibrator offers strong theoretical guarantees while achieving empirical performance that often outperforms more expensive baselines. In summary, MCal is an efficient, calibration scheme that improves the reliability of popular feature- based explanation methods.
Ethics Statement. This work presents a method for improving the reliability feature-based explanation methods. Our intended audience includes researchers and practitioners interested in explainability. While there may be potential for misuse, we do not believe that the contents this paper warrant concern.
Reproducibility Statement. All code and experiments for this paper available at:
REFERENCES
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
Julius Adebayo, Michael Muelly, Harold Abelson, and Been Kim. Post hoc explanations may be ineffective for detecting unknown spurious correlation. In International conference on learning representations, 2022.
Chirag Agarwal and Anh Nguyen. Explaining image classifiers by removing input features using generative models. arXiv preprint arXiv:1910.04256. 2020.
Chirag Agarwal, Satyapriya Krishna, Eshika Saxena, Martin Pawelczyk, Nari Johnson, Isha Puri, Marinka Zitnik, and Himabindu Lakkaraju. Openxai: Towards transparent evaluation of model explanations. Advances in neural information processing systems, 15:45784-1 2022.
Ae@ Llama model card, 2024. URL blob/main/MoDSJI CARD .dd
Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. Towards better understanding of gradient- based attribution methods for deep neural networks. arXiv preprint 171 2017.
Sriram Balasubramanian and Soheil Feizi. Towards improved input masking for convolutional neu- ral networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1855-1865, 2023.
10

---

Shahaf Bassan and Guy Katz. Towards formal xai: formally approximate minimal explanations of neural networks. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pp. 187-207. Springer, 2023.
Shahaf Bassan, Guy Amir, Meirav Zehavi, and Guy Katz. What makes an ensemble (un) inter- pretable? arXiv e-prints, pp. arXiv-2506, 2025a.
Shahaf Bassan, Yizhak Yisrael Elboher, Tobias Ladner, Matthias Althoff, and Guy Katz. Ex- plaining, fast and slow: Abstraction and refinement of provable explanations. arXiv preprint 085::25066.0805, 2025b.
D. Campos and J. Bernardes. Cardiotocography. UCI Machine Learning Repository, 2000. DOI: https//ddb.oog//0224443/11111
Brandon Carter, Siddhartha Jain, Jonas W Mueller, and David Gifford. Overinterpretation reveals image classification model pathologies. Advances in Neural Information Processing Systems, 34; 15395-15407, 2021.
Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image clas- sifiers by counterfactual generation. arXiv preprint arXiv: 080244 2018
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
Nadezhda Chirkova, Germán Kruszewski, Jos Rozen, and Mare Dymetman. Should you marginalize over possible tokenizations? arXiv preprint arXiv:2306. 17757, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Jonathan Dinu, Jeffrey Bigham, and Zico Kolter. Challenging common interpretability assump- tions in feature attribution explanations. arXiv preprint arXiv:2012.02744, 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint vrXiv:2010.11929 2020.
Jiarui Duan, Haoling Li, Haofei Zhang, Hao Jiang, Menggi Xue, Li Sun, Mingli Song, and Jie Song. On the evaluation consistency of autribution-base explanations. In European Conference Computer Vision, pp. 206 224. Springer, 2024.
Christopher Frye, Damien de Mijolla, Tom Begley, Laurence Cowton, Megan Stanley, and Ilya Feige. Shapley explainability on the data manifold. arXiv preprint arXiv:2006.01272 2020.
Jeremy Goldwasser and Giles Hooker. Provably stable feature rankings with shap and lime. arXiv preprint arXiv:2401.15800, 2024.
Chuan Guo, Gcoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
Peter Hase, Harry Xie, and Mohit Bansal. The out-of -iit--i-ittitttiinn problem in explainability and search methods for feature importance explanations. Advances in neural information processing systems, 34:3650-3666, 2021.
Johannes Haug, Stefan Züm, Peter El-Jiz, and Gjergji Kasneci. On baselines for local feature attri- butions. arXiv preprint 1xi::2101000900 2021.
Shreya Havaldar, Helen Jin, Chaehyeon Kim, Anton Xue, Weiqiu You, Mareo Gatti, Bhuvnesh Jain, Helen Qu, Daniel A Hashimoto, Amin Madani, et al. T-fix: Text-based explanations with features interpretable to experts. arXiv preprint (0l::2551.040770 2025.

---

Robin Hesse, Simone Schaub-Meyer and Stefan Roth. Funnybirds: A synthetic vision dataset for a part-based analysis of explainable ai methods. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 3981-3991, 2023.
Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for interpretabil- ity methods in deep neural networks. Advances in neural information processing systems, 32, 2019.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3 2022.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-40 system card. arXiv preprint arXiv:2410.21276, 2024.
Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-llcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 590-597, 2019.
Saachi Jain, Hadi Salman, Eric Wong, Pengchuan Zhang, Vibhav Vineet, Sai Vemprala, and Alek- sander Madry. Missingness bias in model debugging. arXiv preprint arXiv:2204.000 08945, 2022.
Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What dis- does this patient have? large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):442,, 2021.
Helen Jin, Shreya Havaldar, Chaehyeon Kim, Anton Xue, Weiqiu You, Helen Qu, Marco Gatti, Daniel A Hashimoto, Bhuvnesh Jain, Amin Madani, al. The fix benchmark: Extracting features interpretable to experts. arXiv preprint arXiv:2409. 13684, 2024.
Helen Jin, Anton Xue, Weiqiu You, Surbhi Goel, and Eric Wong. Probabilistic stability guarantees for feature attributions. arXiv preprint arXiv:2504. 13777 2025.
Byunggill Joe, Yonghyeon Park, Jihun Hamm, Insik Shin, Jiyeon Lee, et al. Exploiting missing value patterns for backdoor attack machine learning models of electronic health records: Development and validation study. JMIR Medical Informatics, 10(8):e38440, 2022.
Siwon Kim, Jihun Yi, Eunji Kim, and Sungroh Yoon. Interpretation of nlp models through input marginalization. In Proceedings of the 2020 Conference Empirical Methods in Natural Lan- guage Processing (EMNLP), pp. 3154-3167, 2020.
Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T Schütt, Sven Dähne, Dumitru Erhan, and Been Kim. The (un) reliability of saliency methods. Explaimable AI: Interpreting, explaining and visualizing deep learning, pp. 267-280, 2019.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Deniz Koyuncu, Alex Gittens, Bülent Yener, and Moti Yung. Exploiting the data gap: Utilizing non- ignorable missingness to manipulate model learning. arXiv preprint arXiv:2409.04407. 2024.
Ananya Kumar, Tengyu Ma, Percy Liang, and Aditi Raghunathan. Calibrated ensembles can mit- igate accuracy tradeoffs under distribution shift. In Uncertainty in Artificial Intelligence, pp. 1041-1051. PMLR, 2022.
Chris Lin, Ian Covert, and Su-In Lee. On the robustness of removal-based feature attributions.
Advances in Neural Information Processing Systems, 36:79613-79661 2023.
Yang Liu, Sujay Khandagale, Colin White, and Willie Neiswanger. Synthetic benchmarks for sci- entific research in explainable machine learning. arXiv preprint arXiv:2106.12543 2021.
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
12

---

Meike Nauta, Jan Trienes, Shreyasi Pathak, Elisa Nguyen, Michelle Peters, Yasmin Schmitt, Jörg Schlötterer, Maurice Van Keulen, and Christin Seifert. From anecdotal evidence to quantitative evaluation methods: A systematic review evaluating explainable ai. ACM Computing Surveys, 55(13s):1-42, 2023.
Msoud Nickparvar. Brain tumor mri dataset, 2021. URL 2645886
Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmeqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on health, inference, and learning, pp. 248-2648 PMLR, 2022.
Yong-Hyun Park, Junghoon Seo, Bomseok Park, Seongsu Lee, and Junghyo Jo. Geometric remove-and-retrain (goar): Coordinate-invarie explainable ai assessment. arXiv preprint arXiv:2407.12401, 2024.
John Platt al. Probabilistic outputs for support vector machines and comparisons regularized likelihood methods. Advances in large margin classifiers, 10(3):61-74, 1999.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Yao Rong, Tobias Leemann, Vadim Borisov, Gjergji Kasneci, and Enkelejda Kasneci. A consistent and efficient evaluation strategy for attribution methods. arXiv preprint arXiv:2202.00449, 2022.
Supreeth P Shashikumar, Fatemeh Amrollahi, and Shamim Nemati. Unsupervised detection and correction of model calibration shift test-time. In 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 1-4. IEEE, 2023.
Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pp. 180-186, 186, 2020.
Fabio A Spanhol, Luiz S Oliveira, Caroline Petitjean, and Laurent Heutte. A dataset for breast cancer histopathological image classification. leee transactions on biomedical engineering, 63 17):1455-1462, 2015.
Pascal Sturmfels, Scott Lundberg, and Su-In Lee. Visualizing the impact of feature attribution baselines. Distill, 5(1):e22, 2020.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference machine learning, pp. 3319-3328. PMLR, 2017.
Christian Tomani, Sebastian Gruber, Muhammed Ebrar Erdem, Daniel Cremers, and Florian Buet- tner. Post-hoc uncertainty calibration for domain drift scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 10124-10132, 2021.
Tuan L Vo, Thu Nguyen, Luis M Lopez-Ramos, Hugo L Hammer, Michael A Riegler, and Pal Halvorsen. Explainability of machine learning models under missing data. arXiv preprint arXiv:2407.00411, 2024.
William Wolberg, Olvi Mangasarian, Nick Street, and Street. Breast Cancer Wisconsin (Diag- nostic). UCI Machine Learning Repository, 1993. DOI: https:/Abi.ogg//2233/555CD
Anton Xue, Rajeev Alur, and Eric Wong. Stability guarantees for feature attributions with mul- tiplicative smoothing. Advances in Neural Information Processing Systems, 36:62388-6244 2023.
Weigiu You, Anton Xue. Shreya Havaldar, Delip Rao, Helen Jin, Chris Callison-Burch, and Eric Wong. Probabilistic soundness guarantees in Ilm reasoning chains. arXiv preprint 1224:2507..1248.. 2025.
13

---

Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Proceedings of the Eighteenth International Conference on Machine Learning. pp. 609-616, 2001.
Yilun Zhou, Serena Booth, Marco Tulio Ribeiro. and Julie Shah. Do feature attribution methods correctly attribute features? In Proceedings of the AAAI conference on artificial intelligence, volume 36, pp. 0023-9633, 2022.
14

---

A ADDITIONAL EXPERIMENTS AND DETAILS
We present our experimental setup here, along with any additional experiments and relevant details.
Compute.
We had access to server with four NVIDIA H100 NVL GPUs.
A.1 METRICS FOR FEATURE ATTRIBUTIONS
Given an imput x IR and classifier R" + R" feature attribution explanation method returns a vector a E R" where ag denotes the importance of feature Iyz. We now discuss some metries for evaluating the quality of the feature attribution a.
Sufficiency. One way to assess the quality of attribution a is by using only its top-k-selected features for prediction. Let (T, a:) E R be the version of x where its top-k features are selected, ranked by Equivalently, Top a)) is the version of where its bottom n features are ablated. Then, the top-k sufficiency metric (Hase et al. 2021) is defined as:
Sufficiency(J: a, k) = f()), - /TTopp (x)
(4)
where ý = arg max, (x) is the predicted class. In this formulation, lower sufficiency is prefer- able, as it indicates that the selected features can more reliably attain the confidence associated with clean input's 's prediction.
Sensitivity. Conversely, one also assess how much omitting the top-k features would affect prediction. Analogously to the sufficiency metric, let (x, a) denote the version x where the bottom k features are selected; i.e., the top k features are ablated. The top-k sensitivity metric, also called comprehensiveness (Hase et al. 2021). is defined as:
Sensitivity x. k)) = f (x), (())),
(5)
higher score indicates the features were critical the prediction, whereas lower score suggests that the model is more robust (i.e., less sensitive) to their inclusion.
A.2 MCAL TRAINING DYNAMICS
Here, investigate the training dynamics and performance of MCal as the training set size varies. We show the results in Figure & In general, as the training dataset size increases, test-time accuracy increases. When n is small, however, the problem is over-parametrized, meaning that the training loss continues to decrease without significantly improving test-time accuracy.
A.3 MCAL VS. RETRAIN
We note an interesting phenomenon in Table where MCal often outperforms retrain-based meth- ods. At first glance, this is surprising because the model's representational capacity should exceed that of MCal. We suspect that this may be due to our use of conditioned MCal to train an ensemble of calibrator heads, which may at times be sufficient to overcome this gap in representational power. Indeed, we observe that unconditioned MCal, in which only a single calibrator head is used across all missingness levels, typically underperforms retraining.
In attempting to diagnose this observation with more intensive training runs (25K steps, 10-3 learn- ing rate and 50 ablations per data point) we observe that while retrain can outperform MCal in certain situations, its unconditioned variant may also do so.
We suspect this may be due to the differences between the formulations of missingness bias in Equa- tion ( and the optimization objective in Equation (3). A more in-depth analysis would be warranted in future work. We show these trends in Figure with a focus on tabular models given their ease of retraining.
15

---

10a
m-5
mm5
2x
005
m-10
1-
m-
25
Loss
10-1
m=100
0w50
Loss
10"
5=50
1=000
Loss
10"
0-50
1-00
n>250
2=25
25
6-520
5-100
60
100
10
n=1002
1=0000
--34
n-2500
10
f=2500
n=5002
fw3900
3
2000
4000
2000
4000
2000
4000
MRI Train Steps
MedQA Train Steps
PhysioNet Train Steps
Test Accuracy
Accuracy
Test Test Accuracy
0.90
0.5
0.70
0.85
Test Test.Accuu
0.4
0.65
0.80
0.3
0.2
0.60
10
10c
10a
10a
102
10a
10a
10a
10*
MRI Train Samples
MedQA Train Samples
PhysioNet Train Samples
100
10
Base
Base MCal
10-1
Base
Miss. Bias
MCal
Miss. Bias
10-1
Miss. Blas
1
10-2
10-3
10-3
10
10a
10a
10a
10
102
10a
10a
10a
10a
MRI Train Samples
MedQA Train Samples
PhysioNet Train Samples
Train Time (s)
4.4
Train Time (9)
4.4
Train Time (s)
4.3
M
4.2
4.2
4.0
4.2
4.1
10a
Train Samples
MedQA Train Samples
PhysioNet Train Samples
Figure 8: MCal training dynamics and performance on different dataset sizes. For different
amounts (n) of clean-ablated input pairs, show the training loss curves and test-set accuracies. Each training run consisted of 5000 iterations, and all runs finished in < 5 seconds.
Breast Cancer
CTG
PhysioNet
10a
Miss. Blas
Miss. Bla8
10-1
Miss. Blas
Bane
Base
Base
Uncond
10
Uncond
10-4
Uncand
10-5
Cand
Cond
Cond
10-4
Ablation Fraction
Ablation Fraction
Ablation Fraction
Figure 9: MCal can outperform retrain due its specialization Fitting an ensemble of calibrators at discretized set of ablation rates can even outperform retraining the model at times (Left) Breast Cancer, (Middle) CTG, (Right) PhysioNet.
A.4 CASE STUDY: INTEGRATION WITH API-BASED MODELS
Current-day machine learning workflows increasingly dependent closed-weight API-based models. Recognizing this, we perform a case study demonstrating how MCal can be extended to such settings, assuming only access output logits.
16

---

Beseline (Uncalibrated) LIME Explanation for "A"
year-si
fellbaar
nor-llsoodh
Smoroma
undergoos
chemethe woy
deveelose
aupropuct
saln
and
herralafia.
Which
Tollwing
Becy
ropasisk
sadie
sturg hogulve
Neguave
hougral
Posiiee
Strang Ore
Normalized LIME Importance
Unconditioned (Calibrated] Explanation
year-ald
nar-Hlonged
Smohome
chamethe
dencloga
sipmpuck
herrat-rts fla
Whizh
the
Falowing
PORT
responiile
3D
soblett
sytrpic
Store
Regative
heurral
Pesilue
Negstvv
Bive
Normalized
Figure 10: For selected example question, MCal results in different feature importances, for example medically relevant features/terms such "hematuria" gain importance in the calibrated heatmap. The model task is, for the above question, to choose between the Options: A: Cyclophos- phamide, B: Cisplatin, C: Mesna, D: Bleomycin.
Uncied Wiu (Calovinec
Icelvited Intebel
Dfferace
P4b
Figure 11: Direct comparison of LIME-derived feature importance values, with (a) Uncalibrated, (b) Calibrated, and (c) demonstrating the difference between the two, i.e., (b) (a).
In particular, we demonstrate how GPT-4o-mini (Hurst et al. 2024) can be calibrated on selected MedQA instances in Figure I0 and Figure [T] The calibrated model redistributes feature impor- tance in potentially meaningful ways, elevating diagnostic symptoms like "hematuria" hallmark sign of cyclophosphamide toxicity) and "pain", while reducing dominance of the anatomical term "suprapubic". This rebalancing suggests calibration may produce more dinically-aligne and faith- ful explanations, though domain expert validation and further faithfulness testing are needed.
17


Reviews:
[{"review_id": "MxYtBLRXoh", "reviewer": "Reviewer_7kkY", "summary": "The objective of this paper is to enhance explanation quality by resolving the missingness bias, which is a consequence of distribution shifts caused by input manipulations during the explanation procedure. The motivation is clearly stated, and the proposed solution, while intuitive, is reasonable. However, the discussion throughout the paper consistently mixes the pursuit of better explanation quality with model robustness, which are two distinct topics.", "questions": "- The validity of the conclusion from the observations is somewhat questionable. For example, Figure 5 claims that “calibrated models have better explanations”; however, two versions generally show the same perturbation tendencies. The differences in magnitude appear to be more relevant to the changed model behavior due to the additional dense layer. Additionally, it is noteworthy that the two tested models have different functionalities due to the appended dense layer in one version. It is unclear how the results from different explanations on different models can be effectively compared.\n\n- The results presented by the middle chart of Figure 7 appear suspicious. Why is the accuracy a constant with increasing ablation fraction for Conditioned MCal? An ablation rate of up to 90% is very likely to remove all informative features in an input. How does the model manage to maintain the accuracy without “seeing” anything relevant?\n\n- This is more about a question that is relevant to the previous point. The example in Figure 2 motivates a correction of model outcomes when the relevant region is not masked, but what should be the target label if all relevant regions are masked out? Enforcing the output of disease on a manipulated input without indications is arguably another form of “skewed” prediction, particularly when the model originally predicts healthy correctly. I think the argument of this paper only builds upon one side of the coin and leaves the other side insufficiently discussed, and the relevant difficulty unsolved.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The proposed method is straightforward and well-presented, but the analyses and discussions on its impacts lack depth.", "weaknesses": "- In fact, if taking a data augmentation perspective, the proposal of this work is not brand new.\n\n- While the experiments focus on elaborating how MCal corrects the missingness bias, the results fail to show how the calibration on model outcomes affects explanation quality. There is no qualitative example that illustrates the differences, nor quantitative assessments following standard regimes that effectively compare explainers under different settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eenGBRPgUN", "reviewer": "Reviewer_Tchr", "summary": "The paper proposes a calibration technique to mitigate the missingness bias problem encountered when finding feature importance scores. The technique proposed is lightweight and requires access only to logits. They evaluate their technique on various medical data across vision, language, and tabular modalities.", "questions": "Based on the argument in the paper that missingness bias can cause class distributional shift, couldn’t the ablation of seemingly unrelated or spurious features, which the model might rely o,n contribute to this distributional shift?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1) The proposed calibration technique is lightweight, not requiring retraining, access to weights, or architectural modifications of the original model.\n2) The proposed technique can be well adapted to common perturbation-based explainability techniques.\n3)The paper is generally well written, easy to follow, and the motivation is well presented.", "weaknesses": "1)In baseline comparisons, for the replace and retrain categories, the paper uses basic techniques instead of comparing against more advanced techniques in those categories. For example, in the retrain category, some of the recent techniques for imputation, like ROAD, GOAR, are not compared against. Since these are relevant methods to tackle the missingness bias, it would be better to compare them. \n2)The paper claims the missingness bias is a superficial artifact, while the original model embeddings might still be facing this issue.\n3)Some of the crucial details of the experiments are missing. For Figure 7, it isn’t clear if the accuracy is computed using ablated or clean input.\n4)The details of input ablation rates for the baselines in Table 1 are not mentioned, while the proposed method reports an average of calibrators conditioned on various ablation rates.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZcWN1its11", "reviewer": "Reviewer_3Csh", "summary": "This paper introduces a straightforward plugin to enhance explanation quality, which is clearly described and compatible with perturbation-based feature attribution methods. The authors propose to append the to-be-explained model with an additional dense layer, and fine-tune the added layer with the expected manipulations on the input samples while keeping the major part of the model frozen. The proposed solution is intuition-guided and empirically evaluated under small-scale settings, and the experimental results have an emphasis on reporting model performances rather than the intended explanation quality improvement.", "questions": "- See the first point in Weakness.\n- Could the authors better explain the objective of this paper? What are the takeaways of the stability test on the calibrated models? How do they support the claim of improved explanation quality?\n- Do the explanations generated on the calibrated model still faithfully reflect the behavior of the original model?\n- Figure 5 presents the only results that are relevant to explanation quality. Could the authors elaborate on the interpretation of the sensitivity metrics reported there? Given the focus is on explanation quality, the sensitivity measure should reflect the effectiveness of an explainer in identifying the most relevant features. Excluding them should lead to a larger drop in prediction confidence (as stated in line 726); thus, a higher sensitivity score should indicate more effective explanations. In this sense, the results are in favor of explanations from uncalibrated models, which contradicts the interpretation presented in the paper. Also, I do not follow the argument related to model robustness and its connection to the evaluation of explanation quality.\n- Could the authors clarify the different training settings for retrain and MCal? Particularly, why is the retraining scheme fitted to a different ablation distribution than MCal? MCal was fine-tuned exactly for the stratified mask sampling that is exactly used for the validation, which has a balanced distribution of ablation fractions, whereas the retraining scheme only sees the results of uniformly ablated inputs, lacking observations of rich and rare feature presences (e.g., ablation fractions of 2/16 and 14/16 are rarely possible to be sampled). I’m surprised and, in fact, skeptical about the conclusion due to the misaligned ablation strategies.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- This paper is well-motivated and self-contained, with a clear description of the proposed method.\n- The described plugin is flexible and can be appended to arbitrary models regardless of input modalities or model architectures.\n- The empirical results demonstrate the capability of MCal in aligning the predictions on manipulated inputs with the standard model outcomes, supporting the method design.", "weaknesses": "- The discussion on the central motivation is insufficient. Particularly, line 120 states that “masking non-critical regions …”, yet it is arguable that considering certain regions as “critical” already injects human inductive biases. If the model truly learned to look at a meaningless/wrong region, would this additional “correction” for the missingness bias consequently cover up its problematic behavior?\n- While the objective of the paper is clearly stated, the line of discussion deviates from how the added component will enhance explanation quality, instead leaning towards the effectiveness in resolving missingness bias. The modification, which can be considered a form of data augmentation, is intended to facilitate the derivation of more faithful explanations; however, the designed experiments focus more on the impact of this augmentation on model stability and do not sufficiently demonstrate how the modification leads to better explanations.\n- In fact, this paper consistently mixes the discussion on feature attributions with model robustness. Although related in some ways, they are different subjects, where the missingness bias is a particular challenge that feature attribution methods are facing, and robustness is an inherent property of a model. A properly designed explainer should be able to faithfully reflect model behaviors, no matter whether it is robust or not. Mixing these two topics loses the concentration on explainability, and particularly disconnects the latter part of the discussion in this paper from its stated motivation.\n- The faithfulness of the derived explanations becomes questionable given the added dense layer. While adapting the tested model to the disturbed manifold, it arguably also changes the model behavior as a whole.\n- Some experimental settings and result interpretations are questionable; see questions for more details.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QyVmP1hYNS", "reviewer": "Reviewer_ojnE", "summary": "This paper introduces a novel method called MCal, which aims to mitigate \"missingness bias\" in model explanations. Missingness bias occurs when input features are ablated or removed in explanation methods, leading to biased predictions that undermine the reliability of feature importance scores. The authors propose MCal, a lightweight, post-hoc calibration technique that corrects missingness bias by fine-tuning a linear head on the outputs of a frozen base model. The method is model-agnostic, cost-effective, and offers strong theoretical guarantees. The results demonstrate that MCal outperforms more complex, computationally expensive methods across a variety of domains, including medical image classification and natural language processing tasks.", "questions": "1. Could you provide a detailed case study or experiment in a real-world application, such as medical diagnostics where MCal has been implemented to solve a specific problem?\n\n2. In situations with large class spaces (e.g., language models), how do you prevent overfitting during calibration?\n\n3. How does the size of the training dataset affect the performance of MCal? What was the specific training time of your experiments? How do you balance the data size and the computational cost?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Practicality and Efficiency: MCal is lightweight and does not require extensive computational resources, making it more accessible than existing solutions that require retraining or architectural adjustments. The use of a linear transformation and a cross-entropy loss function ensures minimal overhead.\n2. Theoretical Guarantees: The paper offers strong theoretical guarantees of convergence for the proposed method, ensuring that the calibrator will consistently find an optimal solution. This gives confidence in the reproducibility and stability of the approach.\n3. Empirical Validation: The authors provide extensive experimental results demonstrating MCal's effectiveness across diverse medical domains (vision, language, and tabular). It outperforms other methods such as retraining and architecture-based solutions, making it a compelling baseline for addressing missingness bias.", "weaknesses": "1. **Lack of Real-World Application Scenarios**: While the method is demonstrated on several benchmarks, the paper lacks concrete real-world use cases where MCal is applied to solve specific problems. Providing experimental results from a more tangible application would strengthen the paper’s argument and demonstrate the method's practical value in more complex scenarios.\n\n\n2. **Lack of Discussion on the Training Process**: The authors mention that the calibration process requires training a new parameter. However, if the training dataset is too small, the model may overfit, while larger datasets may increase computational costs. The paper could benefit from a discussion and experimental analysis of how the choice of training data size affects the performance of MCal and its scalability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GiHQvDH13N", "reviewer": "Reviewer_3ZQX", "summary": "The paper presents MCal, a method that addresses the missingness or out-of-distribution (OOD) bias inherent in perturbation-based XAI methods. Unlike alternative approaches, MCal does not require computationally expensive retraining or architectural modifications. The paper demonstrates quantitatively that MCal produces improved explanations, provide comparative analysis against baseline methods, and verify that the approach preserves overall model accuracy.", "questions": "See Weaknesses W1 - W3.\n\nComment: An error.txt file remains in the code repository (maintaining anonymity).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "- The proposed solution is elegantly simple yet well-motivated and thoroughly evaluated, though this simplicity could raise questions about the scope of the contribution.\n- The method exhibits modality- and architecture-agnostic properties, being largely model-agnostic depending if the logins can be accessed.\n- The manuscript is very well-structured and clearly written, facilitating reader comprehension through effective organization.\n- The paper provides a pip-installable code repository, enhancing practical accessibility and reproducibility.", "weaknesses": "**W1:** Medical images, particularly in radiology and histopathology, exhibit relatively low inter-patient visual variability, making OOD examples straightforward to generate. But how is this for natural images? As these are more diverse, I suspect less sensitivity to such missingness biases through the perturbations.\n\n**W2:** The evaluation relies exclusively on two metrics that are known to potentially introduce bias. The addition of qualitative visualizations comparing saliency maps before and after debiasing would significantly enhance reader understanding and provide practical insight into the method's real-world performance.\n\n**W3:** Given the method's claimed model-agnostic nature, evaluation or at minimum code implementation with closed-source API-based models would strengthen the work. Such demonstrations, contingent on API access to model logits, would underscore the method's relevance for the broader research community that relies predominantly on closed-source models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)