Reliably detecting model failures in deployment without labels
Authors: Viet Nguyen
Changjian Shui
Vijay Giri
Siddharth Arya
Amol Verma
Fahad Razak
Rahul Krishnan
Venue: NeurIPS 2025 poster ()
Keywords: ML for Healthcare
Guardrails for AI
AI Safety
Deterioration Monitoring
Trustworthy ML
Primary area: machine_learning_for_sciences
TL;DR: D-PDDM provably monitors model deterioration requiring no training data during deployment, and performs well in real-worlds datasets.

Abstract: The distribution of data changes over time; models  operating in dynamic environments need retraining. But knowing when to retrain, without access to labels, is an open challenge since some, but not all shifts degrade model performance. This paper formalizes and addresses the problem of post-deployment deterioration (PDD) monitoring. We propose D3M, a practical and efficient monitoring algorithm based on the disagreement of predictive models, achieving low false positive rates under non-deteriorating shifts and provides sample complexity bounds for high true positive rates under deteriorating shifts. Empirical results on both standard benchmark and a real-world large-scale internal medicine dataset demonstrate the effectiveness of the framework and highlight its viability as an alert mechanism for high-stakes machine learning pipelines.

Reliably Detecting Model Failures in Deployment Without Labels
Viet Nguyen 1.3., Changjian Shui?.f, Vijay Giri Siddarth Aryal Amol Verma Fahad Razak1.5, Rahul G. Krishnan 1.3
University of Toronto University of Ottawa Vector Institute University of Pennsylvania Unity Health Toronto
Abstract
The distribution of data changes over time; models operating in dynamic environ- ments need retraining. But knowing when retrain, without access labels, is an open challenge since some, but not all shifts degrade model performance. This pa- per formalizes and addresses the problem of post-deployment deterioration (PDD) monitoring. We propose D3M, practical and efficient monitoring algorithm based on the disagreement of predictive models, achieving low false positive rates under non-deteriorating shifts and provides sample complexity bounds for high true positive rates under deteriorating shifts. Empirical results on both standard benchmark and real-world large-scale internal medicine dataset demonstrate the effectiveness the framework and highlight its viability as an alert mechanism for high-stakes machine learning pipelines.
Introduction
Performance guarantees of conventional machine learning (ML) models hinge on the belief that the distribution of data with which these models train is identical to the distribution on which they are deployed [1, 3]. many real-world scenarios such as healthcare, however, this assumption fails due to distribution shift during model deployment. Benchmarks such as WILDS [4] and WILD- Time [5] have encouraged machine learning researchers to study and better understand how data shifts influence predictive systems. Yet, the number of tools at practitioner's disposal for building predictive models far exceeds those to monitor model failures. There need to create guardrails that self-detect and alerí end-users to critical changes in the model when its performance drops below acceptable thresholds [6, 7].
We define post-deployment deterioration (PDD) as the scenario where a trained ML model underper- forms on distributionally shifted deployment query with respect its validation performance. PDD presents a distinct set of systemic challenges stemming from considerations over the feasibility of deployment real-world ML pipelines. Predominant is the scarcity of labels during deployment: for many downstream tasks such as in healthcare, labels are expensive to obtain [8] or require human intervention [9]. Due to deployed models predicting events temporally extended in the future [10, 11], labels might even be unavailable. Another is the robustness of the monitoring system: it should flag critical changes in model deterioration early, using few samples, while remaining resilient to non-deteriorating changes to minimize unnecessary interruptions of service among other practical considerations.
To address these challenges, we conceive a set of desiderata for any algorithm monitoring PDD, targeting their practicality and effectiveness as plug-ins to ML pipelines. To address the scarcity of
1 Equal contribution, correspondence to Viet Nguyen: viet@cs. toronto.
39th Conference on Neural Information Processing Systems (NeurIPS 2025).

---

labels, PDD monitoring algorithms should operate unlabeled data from the test distribution to ascertain potential deterioration of the deployed model. Further, PDD monitoring algorithms should not depend on training data during deployment, as continuous (even indefinite) access to sensitive or personally identifiable training data might violate certain regulations protecting the privacy of data subjects [12]. An algorithm satisfying this desideratum is scalable, as it only audits model's input stream during monitoring with minimum data storage and regulatory concerns. Finally, PDD monitoring mechanisms should be robust to flagging -ondeeteriogating changes and effective even when samples from the deployment distribution are scarce.
When it comes designing monitoring protocols that satisfy the above desiderata, recent related works only partially attend to individual desiderata. The literature on distribution shifts while achieving strong empirical performance unlabeled deployment data [13, 14, 15], false positives when the distribution shift is non-deteriorating. The model disagreement framework [16, 17, 18, 19, 20] emerges as competitive setup for monitoring with downstream performance considerations via the tracking of disagreement statistics, while foregoing explicit distribution shift computations. However, shift-based and disagreement-base based monitoring methods all depend on the presence of training data deploymetn, and do not provide any guarantees on robustness against false positives in the monitoring of non-deteriorating shifts.
robust to
In this work, we answer all desiderata for PDD monitoring via the disagreement framework by proposing Disagreement- Driven Deterioration Monitoring (D3M). Our contributions are follows:
Answering desiderata. D3M is a novel algorithm operating in the label-free deployment setting (I), requiring no training data during monitoring (2), and is provably robust in flagging deteriorating shifts as well as resilient to flagging non-deteriorating shifts (3). comparison the satisfaction of the PDD desiderata of our method with other related work in the literature is in Table 1.
Practical and scalable. D3M is model agnostic so long as the base model's s feature extractor can be optimized via gradient descent. This flexibility allows D3M to monitor various modalities of high-dimensional data. Unlike previous work, D3M avoids the retraining or finetuning of the base model via posterior sampling, crucial for the efficient monitoring of large models. Finally, owing to the decoupling of the algorithmic protocol into two distinct stages, D3M is efficiently scalable the size of the training dataset, critical consideration on the feasibility of its application onto current ML pipelines that is not enjoyed by standard baselines.
Empirical validation. We showcase experimental results on various shift scenarios in the UCI Heart Disease dataset [21], CIFAR-10/10.1 [2), Camelyon17 (WILDS) [4], and the GEneral Medicine INpatient Initiative (GEMINI) dataset [22, 23] to demonstrate its effectiveness in monitoring models of various modalities. Our method effectively detects deployment- deterioration with low false positive rates (FPR) when shifts are non deteriorating, and achieves competitive true positive rates (TPR) when shifts are deteriorating compared to standard baselines. In particular, we discuss how results on the internal medicine dataset suggest D3M to be well-suited for integration into real-world clinical monitoring pipelines.
Provable algorithm. Under certain assumptions about the underlying distribution changes, D3M provably monitors model deterioration when a deteriorating shift is present. In the presence of non-deteriorating shifts, D3M provably resists detection, thereby achieving low false positive rates.
Table 1: Comparisons between related work. Training data-free: whether post-deployment monitor- ing requires training data; Deteriorating: whether the method provably monitors the deteriorating shift; Non-deteriorating: whether the method is provably robust in the non-deteriorating shift; Disagreement: whether the method is based on the disagreement framework.
Training data-free
Deteriorating Non-deteriorating
Disagreement
Yu and Aizawa, 2019 [16]
X
Liu et. al., 2020 [13]
X
Jiang ct. al., 2021 [18]
X
X
Zhao et. al., 2022 [14]
X
X
Rosenfeld and Garg. 2023 [20]
X
X
Ginsberg et. al., 2023 [19]
X
X
D3M (ours)
2

---

2 Background and Algorithm
2.1 Problem setup
Assume base model f is supervisedly trained to classify inputs X into finite discrete classes V = (1....,C) from training examples D = {xr Ji where tuples (2+,, Vi)i==in P for all i E [n). For a joint distribution P over X' x ) let Ps denote its marginal distribution over X. We are interested in designing mechanism such that upon seeing collection of unlabeled inputs {ar fi Iam sampled from a deployment distribution Qz, the mechanism flags model deterioration if Qx * Ps and / underperforms on Q2 without being able to observe labels for Q... On the other hand, Qx + P. while f remains performant on Q.. the mechanism should resist flagging. Achieving so ensures that our mechanism only flags deployment-tie time changes that are truly deteriorating.
How can we monitor ML models for deployment deterioration due to distribution shift without indiseriminately flagging any changes in the data?
We require a computable quantity Φ. independent of labels, whose value statistically differs in- distribution (ID) and out-of-distribution (OOD) if and only if model deterioration occurs. Monitoring, then, regresses to recording baseline values for & evaluated on ID held-out validation samples. Then, upon collecting unsupervised deployment samples from unknown distribution, the monitoring mechanism computes and compares it the recorded baseline values, and finally outputs verdict.
Leveraging insights from [16, 24, 19], the framework of model disagreement possesses this property under certain assumptions about the underlying distribution change (see Appendix A). We say that two models h, and ht2 disagree on an input æ X if h1 (x) j (ar). In particular, models exhibit greater predictive disagreement on unsupervised samples that lead to model deterioration, compared to in-distribution (ID) samples. This is observed through the increased entropy-basse discrepancy between classification heads in [16]. or maximum disagreement between models in the same ensemble in [24] and [19], as signal for detecting deployment deterioration. Maximizing classification head discrepancy for OOD detection [16] is efficient for monitoring at deployment time, requiring only one forward pass to compute a verdict. However, this trades off classification accuracy as this training procedure alters the original trained decision boundaries. On the other hand, computing model disagreement between ensembles [19] requires finetuning a potentially large network to collect ID and deployment memettmmnt-timeeeeee disagreement statistics 6. In addition, ID training data is required at deployment, further limiting the scalability of such framework.
2.2 Overview of D3M
To avoid needing the original training set at deployment, we replace finetuning with Bayesian approach that models posterior predictive distribution (PPD) over logits. This yields a distribution over decision boundaries that remains faithful to ID behavior. By comparing samples from the PPD to the mean prediction, we approximate maximum disagreement without retraining or access training data. As the PPD is usually intractable, we instead model it with variational distribution, easily optimizable using standard methods.
Sampling disagreement statistics Φ in this way yields a reference distribution & of ID maximum disagreement rates. At deployment, we compute the same statistic & and flag model deterioration when & exceeds high quantile of &. This enables unsupervised. training-free monitoring. Our Deterioration Monitoring (D3M) follows three key steps: Train,
method Disagreementt Calibrate. and Deploy.
1. (Train) Base model training. Firstly, neural feature extractor FE. X - Rd coupled with Variational Bayesian Last Layers [25] VBLL Rd -> ((RC)) parametrized by E θ are trained on supervised ID data D ~ pd output posteriors over logits corresponding C' classes. For an input x E X' and its ground-truth label E ):
3 FE (z)
go (x) = VBLL,
We define our base model the variational posterior predictive distribution (PPD) given by integrating with respect to qo:
Plyl D") E (-1x) [softmax(2))
3

---

Train
Calibrate
21 O OK
D ~ P 22 o
Dr
P
Bootstrapped
y
Sample Dr
2k:
X
X3
Φ max DisRate(k) kE(K]
VBLL
1,...
Deploy Dm Q
Deteriorating Shift
Quantile (+)
if / Q Quantile (
No Deteriorating Shift
otherwise.
0
ØT
Figure 1: Overview of D3M. (1) Train: a feature extractor (FE) and a Variational Bayesian Last Layer (VBLL) are trained to model posterior predictive distribution (PPD) over class logits. (2) Calibrate: disagreement statistics are computed by bootstrapping held-out ID datasets, sampling from the learned posteriors, and comparing sampled predictions to the base model's outputs to collect a set of maximum disagreement rates Φ For illustrative purposes, agreements and disagreements between ÿ(3) and ý are colored green and orange, respectively. (3) Deploy: at deployment, D3M monitors the model on incoming unlabeled data by computing the maximum disagreement rate o and flags deteriorating shift if á 2 Quantile)-a())
where predictions are assigned by computing the argmax of the PPD. Classification training is performed by maximizing the ELBO with a standard Gaussian prior p(=):
LELBO (0; x. y) (=22) [log softmax(2))) KT. (=](= [|](=)]
(Calibrate) Training of max disagreement rates with respect the base model. are to gather disagreement statistics é into & computed ID samples for deployment time compar- ison. For rounds t E [T], on a held-out ID collection, we bootstrap an unsupervised ID dataset D {z, 1m pm and acquire posteriors over logits (ge(-lar (.r1),c qa (-1æ2). (=(-(amm)) Instead, pseudolabels 7i are assigned by the base model using the mean predictive distribution:
Ui argmax (z
(softmax(2:)y)-
Vx; E D
y=
We draw K samples from the variational posteriors qo (e(+));) in parallel using vectorized sampling. apply a temperature-scaled softmax, and sample class labels from the resulting categorical distribution:
(k)
go(-12z) = P (k)
softmax
Vi E [m]. A [K]
(k)
Categorical
P
For each posterior sample & [K] and their corresponding predictions (0) ..... Fm the disagree-
(k)
ment rate of is with respect to the base model predictions (71. Fm) is calculated as:
X
DisRate(k)
e E [K]
Finally, for the bootstrapped dataset D the maximum disagreement rate is given by:
de == max DisRatock) kE[K]

---

After T rounds, store our collection of ID maximum disagreement rates +b = {or L E [T]]-
3. (Deploy) Deployment monitoring of the base model. Our base model now ready to be deployed in test environment and outputs predictions for an unsupervised input stream. To monitor it for deployment deterioration, we periodically gather inputs into a insupervised deployment dataset Di ~ Qt and compute its maximum disagreement rate & as previous. D3M outputs 1 if é 2 Quantile (+) else O for a desired significance level a.
The entire protocol is summarized in Figure 1. Assume that the deployment distribution is the same as the training distribution. In this case, we expect that I Quantile (+) with probability a, thus having immediate control over the false positive rate (FPR) of D3M. When there is, however, distribution shift between training and deployment distributions, under certain assumptions about the underlying shift and ground truth labeling. we show that D3M provably flags deteriorating changes, i.. high true positive rate (TPR) is achieved, while being resistant to flagging non-deteriorating changes, i.e. changes in the input distribution that do not result in the base model underperforming. We defer the presentation and discussion of our theoretical analysis to Appendix A.
2.3 Algorithmic insights
D3M is free from training data. D3M is designed to monitor deployment time input streams without requiring the training data of its base model. This is important advantage shared by [16]. however not enjoyed by other baselines in Table 1, whose computation of é statistics require maintaining agreement on training data. When the neural feature extractor FE is millions or billions of parameters, trained with a comparably large dataset, it becomes infeasible to store the training data within edge compute nodes in order to run the monitoring mechanism.
No tradeoff with prediction accuracy. Furthermore, D3M does not trade-off prediction accuracy as the mean prediction model from the PPD is not modified during the construction of & nor its deployment (Appendix C.1). Therefore, ID generalization theories remain applicable [26, 27, 28, 29, 30, 31]. In addition, recording a collection of II) maximum disagreement rates reinforces the robustness of the method. While single instances of maximum disagreement rate are subject to noise
in the choice of the held-out validation set, the sampling of posterior logits 2, (R) and the number of drawn samples K, a collection & of maximum disagreement rates coupled with a quantile test softens the effect of noise by leveraging large sample statistics.
Less diverse samples compared to its fully Bayesian counterpart. Harrison et. al., 2024 [25] suggests that one may apply a variational Bayesian treatment to the last layer only, saving on computational costs of forward and backward propagating through Bayesian neural network, and enjoy largely the same uncertainty estimation and OOD detection performance. However, the usage of VBLL in D3M is unorthodox with respect the original work: by sampling posterior weights from a Bayesian model, we are hoping to land on a set of weights such that the resulting model strongly disagree with the base model. VBLL allows more efficient sampling compared to fully Bayesian network, though lacking diversity in the sampled predictions, as the variability comes from the last layer only. This results in K sampled logits mostly agreeing with the mean prediction. To improve the diversity of sampling, we employ (1) temperature scaling and (2) the sampling of predictions from the categorical distribution for the computation of maximum disagreement rather than argmax labeling. We defer the study of this tradeoff between fully Bayesian and VBLL to Appendix C.5.
2.4 Implementing D3M
The composition VBLLg o FEg : + P(RC) is readily trained end-to-end on ID data of size n by maximizing the ELBO. For rounds i E [T]. on a large held-out ID) validation set, we sample datasets DrT of size m & n with replacement for calibration. DI is processed in a single forward pass, producing m independent C-dimensional Gaussian posteriors:
qa (=( = (=)lpoo( diag(o) for = 1,..,,
For each posterior, we draw 2K samples, K for our Monte-Carlo estimation of the mean model and K for maximum disagreement computations. We keep the sampling temperature т, the size of bootstrapped datasets m, and the number of posterior samples K as tunable hyperparameters. Importantly, these should be the exact same for the computation of & and the subsequent deployment-
5

---

time monitoring test, where the exact same computational procedure is employed deployment sample Di A detailed implementation can be found here: https://githinn htt.tttuu..ooomaaaceoommm tei vng/ d3m.
2.5 Theoretical foundation and practical approximation
Indeed, D3M monitors ML models for deteriorating shifts. As formulated, D3M is approximation of Idealized D3M (Algorithms 1 and 2 in Appendix A). Our theory formalizes model deterioration under some conditions into set of incqualities (Definition 2) where Idealized D3M is designed to track their satisfiability. However, this idealized version requires an oracle that can exactly solve the optimization problem of finding the hypothesis in Hp (well-performing models on the training distribution, Appendix A) that maximally disagrees with the base model. In practice, this search is computationally intractable for large function classes. D3M addresses this by replacing the intractable optimization with the described sampling procedure to approximate hypotheses that likely belong to Hp, trading off exact optimization for computational efficiency while maintaining the core disagreement-basse monitoring principle. Our theoretical analysis demonstrates that Idealized D3M provably detects deteriorating shifts while controlling false positive rates (FPR) guarantees that D3M inherits approximately through its posterior sampling strategy.
In fact, our ablation results empirically show that D3M oversamples hypotheses beyond Hp. deviating from theoretical guarantees. Despite this, D3M demonstrates remarkable robustness in practice, achieving strong empirical performance across diverse datasets. This suggests that the variational posterior sampling, while not perfectly constrained to Hy. still captures sufficient disagreement signal to effectively detect deteriorating shifts. We refer the reader to Appendix A for the theoretical setup and analysis, Appendix C.4 for our results and discussion on our oversampling ablation, and Appendix C.5 for a comparison between oversampling VBLL and fully Bayesian neural network.
3 Experiments
In all experiments, at minimum, competitive performance in detecting deteriorating shifts (when present) should be achieved. This would validate D3M's effectiveness in alerting end-users of critical changes in deployment. Results on the vision datasets (CIFAR-10/10.1, Camelyon show that D3M effective in monitoring high-dimensional, structure-rich data, in addition to tabular setups. Finally, we explore D3M in monitoring deteriorating and non-dcteriorating changes in the GEMINI dataset, real-world longitudinal electronic health record dataset to study how well our method aligns with clinically meaningful degradation, bringing forth discussions on mechanism's 's practical utility for trustworthy, now-intervention deployment in healthcare settings.
3.1 Experimental setup
Datasets. (1) The UCI Heart Disease prediction dataset [21]. where each hospital corresponds to a different domain. Here, the distribution shift due to differences in patient populations and data collection practices across hospitals. (2) CIFAR-10/10.1 datasets [32, 33] where shift comes from subtle changes in the dataset creation process. By viewing samples from CIFAR-10 10 as P. we test our models' ability to flag deteriorating shift from samples in Q CIFAR- 10.1. I. (3) The Camelyon17 dataset from the WILDS benchmark [4, 5] a histopathology image dataset for detecting metastases in lymph node slides, where distribution shift arises from variations in slide staining and image acquisition between hospitals contributing the data. (4) The General Medicine Inpatient Initiative (GEMIND) [22, 23] dataset, a comprehensive repository of standardized clinical and administrative data from hospitalizations within general internal medicine. We focus on predicting patient mortality within 14-day horizon, leveraging static and longitudinal clinical features. This task is deemed essential for facilitating timely clinical interventions, optimizing the allocation of healthcare resources, and ultimately striving to improve patient outcomes [34, 35, 36]. Detailed data descriptions can be found in B.3. Full sweeping details and hyperparameter configurations are reported in B.5.
Implementation & Baselines. For tabular data (UCI, GEMINI 1 14-day mortality), in our implementa- tion, FE, corresponds to sequences of affine transformations and nonlinearities with skip-connections. For image datasets (CIFAR-10/1001. Camelyon FE, is either a trained or a finetuned ResNet [37]. In particular, the D3M mechanism is agnostic to the choice of feature extractor, provided that

---

the architecture reflects appropriate inductive biases for the data modality and permits gradient flow through the extracted features.
To demonstrate that our D3M enjoys low FPR deteriorating shifts and high TPR deterio- rating shifts, we compare it against several distribution divergence-based detection methods from the literature: Deep Kernel MMD (MMD-D) [13], H-divergence [14], Black Box Shift Detection (BBSD) [38], Relative Mahalanobis Distance (RMD) [39]. Deep Ensembles [40], CTST [41], Domain Classifier (DC) [1], and Detectron [19]. Details can be found in Appendix B.1.
Evaluations. For all experiments, the significance level a is fixed to 0.10. (2) For UCI Heart Disease, CIFAR-10/10.1, and Camelyon17, where there are known post-deployment deterioration, we evaluate the baselines' and D3M's ability to monitor shift for query sizes (10, 200,20, 50) of the deployment distribution. In doing so, we require that good monitors quickly recognize whether the deployment distribution has deteriorating consequences to the model or not. (3) For the GEMINI dataset, we study the detection rates on temporally-spli sub-datasets and mixtures of subpopulation splits incurring deteriorating changes and report TPR/FPR where appropriate. For D3M, all TPRs reported are achieved while maintaining cause D3M to overfit its reference disagreement distribution & to the held-out validation set, allowing perfect TPR but also maximal FPR.
ID FPR below This is due to the temperature т: increasing can
UCI Heart Disease
CIFAR 10.1
Camelyon 17
10
20
50
10
20
50
10
20
50
BBSD
.13.1033
.22.1.04
461.05
07:1.03
.051.02
.12..033
.161.04
.3811.05
.871.03
Rel. Mahalanohis
.11+.03
.36+.05
.66+.05
.05+-02
.03+-03
.04+-02
.16+.04
.40+.05
.89+-03
Deep Ensemble
.134.03
.32+.05
.642.05
.332.05
.52+.05
.68=.05
.14:.03
.264.04
.82+.04
CTST
.15+.04
.51.05
.98-.01
.03+.02
.04+002
.04+.02
.11+.03
.59+.05
.59+.05
MMD-D
.9:1.03
.121.03
.771.04
.44.00
.10..03
.051:02
.421:05
.62:1.05
.691.00
H-Div
.15+.04
.26+004
.34.05
.02+.01
.05+-02
00++002
.03+-02
.07+-03
.23+-04
Detectron
.2.1.04
.57.1.05
.2.1.04
.37..00
.541.05
.831.04
.971.02
1.0.1.00
.9611.02
D3M (Ours)
.8.1.19
.251.28
.691.33
.401.10
.451.10
.741.12
.891.20
.931.005
.991.02
Table 2: True positive rates (TPR) comparison across datasets and query sizes. As models do experience deterioration, the higher TPR the better. Bold indicates best in column. We report the means and standard deviations of TPRs obtained from 10 independently seeded runs.
3.2 Discussions and Analysis
UCI Heart, CIFAR-10/10.1, and Camelyon17. Model deterioration is present, thus the higher reported TPR the better. Consistently, across the benchmark datasets, we observe that for each query size, D3M enjoys comparable TPR to the top achieving baselines. Results on the Camelyon17 17 experiments at all query sizes highlight D3M's ability detect diagnostically--eeevan deterioration.
High-variance reported TPR and limitations. However, remarkable is the high variance of the TPRs reported on the UCI benchmark. We believe this is a shortcoming of D3M compared to the other baselines in that our results are noisier and less performant. On the one hand, there is merit in high-risk settings to flag critical changes
as possible. But on the other hand, more samples may be required in order to truly ascertain the deteriorating nature of a shift. It is up to users of D3M decide the batching size of collected deployment samples depending on what provides meaningful signal for further analyses or actionable items. This also reflects the user's tolerance for alert fatigue as noisier estimates on smaller deployment samples may get flagged more liberally. We discuss how selecting this deployment size implicitly allows control of the FPR of the model with additional results of D3M on larger query sizes in the above benchmarks in B.6.
Comparisons with Detectron. [19] While D3M and Detectron achieve similar monitoring per- formance both successfully flagging all deteriorating shifts in higher query size scenarios (see 16)-D3M-PT is significantly more scalable and practical for real-world deployment, especially at the edge (Appendix C.2). Unlike Detectron, which requires persistent access to the original training data and gradient-baat based finetuning during deployment, D3M operates in a truly "source-free" fashion post-training: it needs neither storage access to sensitive training samples, nor does it perform computationally expensive or potentially destabilizing finetuning in production. Furthermore, by
7

---

0.925
10-
0,3
AUROC
0.873
FPR05%
0.6
B.D0c
KL-Twv
0,850
Oure
0.825
0.4
0.800
0.775
Boscine perfectance
18111
1882
19111
1012
20111
2012
2181
2112
0.0
18101
1091
3011
301
2101
2012
(a) Non deteriorating shift in GEMINI
(b) Performance monitoring
Figure 2: Performances time evolving shifted test data from GEMINI. (a) Performance drop (bar plot) is small, thus non-deteriorating shift is observed. (b) Time evolving shift monitoring. D3M is robust with small False Positive Rate (FPR) level 0.05.
1,
J8-1ks
KL-De:
0.8
AUROC
0.80
TPRM5%
MMDD
Oura
0.6-
0.75
BBSD
0.70
0.4
DC
0.2
Bacerine performanes
1.0
0.0
6²
18
(a) Deteriorating shifts GEMINI
Deterioration monitoring
Figure 3: Monitoring results on artificially shifted test data from the GEMINI dataset. (a) Perfor- mance drop (bar plot) significant when the degree shift is large (0.0 + 1.0) (b) Results on different monitoring methods, D3M achieves competitive TPRs at level a 0.05.
relying solely forward passes and simple statistics over deployment data, D3M is inherently robust and less susceptible to the risks associated with continual retraining, such as accidental data contamination or privacy leakage.
GEMINI Evaluations. Notably, the GEMINI dataset possesses an inherent temporal covariate shift, as patient demographics, laboratory values, and treatment patterns evolve over time, particularly during the COVID-19 period. While we expect distribution shift to occur, whether it represents performance deterioration or benign drift remains unknown priori, mirroring real-world deploy- ment uncertainty. This makes GEMINI the closest available appreximation to prospective clinical deployment, where D3M selectively flags shifts those that provably deteriorate performance while ignoring benign drift, ensuring intervention occurs exclusively when warranted. As such, GEMINI serves realistic testbed for evaluating both model robustness under distribution shift and D3M's practical utility in settings that closely resemble actual clinical practice.
GEMINI Temporal Shift Results. We train the mean model pre-2018 data and deploy on half- year splits thereafter. shown in Fig.2(a),, there little to no performance drop over time, indicating non-deteriorating temporal shift. Accordingly, D3M resists unnecessary alerts, maintaining low false positive rate compared to baselines (Fig.2(b)).
GEMINI Age Shift Results. For deteriorating shifts, we train on adults aged 18-52 and test on various mixtures of age groups. Fig.3(a) shows a clear performance drop as more disttiibuittnnonnn samples are included. Here, all methods methods-including D3M- SuM-ssccesfuully flag these shifts (Fig.3(b)) with D3M achieving competitive detection across all mixture ratios. This demonstrates that D3M matches the strongest baselines in detecting genuine post-deployment deterioration.

---

Clinical integration and flexibility of monitoring, In real-world clinical settings, the deployment of monitoring systems like D3M must be complemented with clear guidelines for human interven- tion. One practical integration point is within existing model governance frameworks in hospital information systems, where D3M's alerts could be logged and periodically reviewed by clinical data stewards or model governance committees. Therein lies the flexibility of detecting versus agnostic adaptation: monitoring provides a choice on what to do next given a deterioration flag, whether it be retraining, triggering deeper performance audits, shadow deployments, or even adapting. We believe D3M offers tunable layer of oversight that can flexibly support both real-time and retrospective clinical review processes.
Notably, we view the success of D3M on the real-life D3M as representing a paradigmatic shift in the AI in Healthcare space: current MI. models trained on historical data, validated on held-out test sets, and then deployed directly into clinical practice with minimal ongoing monitoring for performance degradation. While existing approaches relying on heuristic thresholds or requiring expensive retraining cycles have been limitedly employed, our method offers healthcare practitioners a principled mechanism to maintain model reliability in production environments. The validation of the D3M monitoring framework on the GEMINI dataset. which exhibits natural temporal shifts including the COVID-19 disruption, demonstrates the practical utility of this approach in real- clinical settings where model deterioration can directly impact patient care decisions.
4 Related Work
Adaptation. Test-time adaptation (TTA) concerns itself more with how one achieves strong de- ployment performance in spite of a critical change, rather than how one would flag this critical change. To this end, we fundamentally believe that monitoring offers an additional level of ficxibility: when a deterioration flag is raised, adaptation is a possible course of action among many others. A few works from this rich body of literature are tangentially related to disagreement- based monitor- ing. [42, 43, 44] all leverage prediction uncertainty discrepancy drive unsupervised domain adaptation. [45], measures and minimizes prediction disagreement at test time, adaptively aligning target-domain features to a domain-invariant feature space, improving performance without requiring access to target domain data during training. In the continual learning literature, [46] also leverages classifier disagreement as a unsupervised proxy of distributional change.
Performance monitoring of ML models & deteriorating shift. Evaluating a model's 's reliability during deployment is crucial for the safety and effectiveness of the machine learning pipeline over time. [47, 48] provided a causal viewpoint wherein the challenge to adapt to diverse scenarios still remain due to a lack of access to the true causal graph. Model disagreement is often used monitoring tool for the model generalization [49, 17, 19, 20J. These works align strongly with our method, though the framing is orthogonal to ours our analysis provides FPR and TPR guarantees whereas they provided sufficient conditions in either ID or other shifts beyond our scope. Several works in the recent literature differentiate shifts in terms of deteriorating or non-deteriorating shifts. [50] studied (deteriorating) shift detection in the continuous monitoring setting using a sequential hypothesis test. Due to the setting being sequential in nature, their method requires true labels from Q immediately after prediction or at the least in a delayed fashion. [51] approached the monitoring problem from a time-continuous anomaly detection perspective, allowing periodic querying of deployment-ting ground truths from experts. Other related empirical works along this literature are [52, 53], and the very applied [54].
Distribution shift detection. Methods to detect distribution shift arise from different perspectives. In covariate shift detection, [41, 13, 14] treated detection as two-sample tests via classifier, Deep Kernel MMD, and H-divergence. For label shift on the other hand, [38, 55] formulated the problem as a convex optimization problem by solving the label distribution ratio a Q P(y). The problem of OOD detection [56, 53] seeks to detect if an individual sample x comes from the training distribution
P (z).. Some previous works [57. 58] also adopted the methods in covariate shift detection and generalization by estimating the density ratio for the identification of OOD samples. Whilst these methods detect shifts, they are constrained by their requirement of training data post-deployment and do not consider the extent to which shifts affect model performance.
Estimating test error with unlabeled data. Another rich body of research is the estimation of (OOD) test error. This technique and its variants are often inspired by domain adaptation theories [59,

---

60, 61, 62], seeking guarantess in the form of err(e Qg) Pr)(J: H.) with A(J H) Su err(h; h: p,) err(h; y-)l- This objective can be alternatively viewed as searching for a critic function h E H maximize performance gap [20, 18]. One could thus provably estimate the upper bound of the test distribution error. These theories also implicitly assume the availability of training data. Further, they assume that test error should be larger than the training error (granted this is often the case for deteriorating OOD), making them sensitive to non deteriorating shifts as well i.e. a high FPR in detection. Our theoretical analysis addresses this gap.
Conclusion and Limitations
Two core limitations to D3M are of note when considering deployment in real-world setting. Firstly, confidence intervals in Table evidences the noisiness of our method compared with other baselines on low query sizes, due to the inherent randomness arising from sampling alternate hypotheses. We do remark, however, that at larger query sizes of 100 and 200, both performance and confidence regions of D3M match those of the strongest baselines (Table 7). We believe that stronger, more general feature representation helps alleviate this issue as well as enhances performance. Empirically, we observe that by borrowing pretrained ResNet features ImageNet in the Camelyon17 experiments, D3M achieves high performance at low noise. Future work may explore how performance and stability vary function of the quality of representation in the base model. Second, our provided theory only provides provable guarantees when labels are assigned by ground-truth function. Future work could leverage stronger analytical tools in statistical learning such as PAC-Bayes to derive tighter bounds when labels are noisy, more closely matching real-life deployment scenarios on mislabeled, even adversarially poisoned labels.
In sum, we study the problem of post-deployment deterioration monitoring of machine learning models in the setting where labels from test distribution are unavailable. We propose threc-stage disagreement-bare
Bayesian monitoring algorithm, D3M, which monitors and detects deteriorating changes in the deployment dataset while being resilient to flagging non-deteriorating changes. Importantly, our method does not require any training data during deployment monitoring, allowing for efficient out-of-the-box deployment in many machine learning pipelines across various domains. While D3M enjoys increased efficiency, larger confidence intervals demand stricter hyperparameter settings (and thus, bigger initial sweeps) to function effectively. On the theory side, under certain assumptions, statistical guarantees are provided for achieving low FPR in non-deteriorating shifts and high TPR deteriorating shift.
Empirically, we validate insights from our theory on various synthetic and real-world vision and healthcare datasets evidencing the effective of D3M. Critically, the GEMINI dataset-our our closest approximation to prospective clinical deployment- DDMM demonstrates the precise selectivity required for real-world viability: it resists flagging during periods of non-eeteii temporal shift while reliably detecting and flagging true performance deterioration. This behavior, validated under realistic conditions with inherent distribution drift, suggests D3M's readiness for practical deployment scenarios where distinguishing actionable degradation from benign evolution is essential. Our work signals step toward the robust, scalable, and efficient deployment of mechanisms to audit and monitor machine learning pipelines in the break of dawn of ubiquitous AI.
10

---

Acknowledgments and Disclosure of Funding
RGK gratefully acknowledges support from the Canada Research Chairs Program (CRC-2022-00049), the Canada CIFAR AI Chairs Program. This research was funded in part by NERF Special Call 2022-00026).. Resources used in preparing this research were provided, part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the wwwrent-partrenne
Award
Vector Institute (www. vector tute. aai/partnan prartnershipp was supported by GEMINI, a research program based out of Unity Health Toronto. The development of GEMINI's data platform has been supported by several funding partners, which can be found listed All analyses, results, conclusions, and opinions
This study
https:/ //gem///mmmm ///mmmmmnn htt//emmiimmmet nimedi cine. , ca/ ca/cc//ppaaa htpppattersse presented in this paper are solely those of the listed authors and are independent from GEMINI's funding sources. No endorsement by GEMINI's funding sources intended, nor should it be inferred. Amol Verma receives salary support from the Temerty Professorship of AI Research and Education in Medicine the University of Toronto. Fahad Razak receives salary support from the Canada Research Chairs program.
References
[1] Stephan Rabanser, Stephan Günnemann, and Zachary Lipton. Failing loudly: An empirical study of methods for detecting dataset shift. Advances in Neural Information Processing Systems, 32, 2019.
[2] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do cifar-10 classifiers generalize cifar-10? arXiv preprint 100.00180,00055, 2018.
[3] Shibani Santurkar, Dimitris Tsipras, and Aleksander Madry. Breeds: Benchmarks for subpopu- lation shift. arXiv preprint SoXiv:2008.04883) 2020.
[4] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: benchmark of on-the-wild distribution shifts. In International conference on machine learning. pages 5637-5664. PMLR, 2021.
[5] Huaxiu Yao, Caroline Choi, Bochuan Cao, Yoonho Lee, Pang Wei Koh, and Chelsea Finn. Wild-time: A benchmark of in-the-wild distribution shift over time. In shirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
[6] Anand R Habib, Anthony L Lin, and Richard W Grant, The epic sepsis model falls short the importance of external validation. JAMA Internal Medicine, 181(8): 2021.
[7] Karina Zadorozhny, Patrick Thoral, Paul Elbers, and Giovanni Cinà. Out-of-distribution detection for medical applications: Guidelines for practical evaluation. In Multimodal Al in healthcare: A paradigm shift in health intelligence, pages 137-153. Springer, 2022.
[8] Narges Razavian, Saul Blecker, Ann Marie Schmidt, Aaron Smith-Mel Somesh Nigam, and David Sontag. Population-level prediction of type 2 diabetes from claims data and analysis of risk factors. Big Data, 3(4):277-287, 2015.
[9] Xiaofeng Liu, Chachwa Yoo. Fangxu Xing, Hyejin Oh, Georges El Fakhri, Je-Won Kang, Jonghye Woo, et al. Deep unsupervised domain adaptation: A review of recent advances and perspectives. APSIPA Transactions on Signal and Information Processing, I1(1), 2022.
[10] Noorhannah Boodhun and Manoj Jayabalan. Risk prediction in life insurance industry using supervised learning algorithms. Complex & Intelligent Systems, 4(2):145- 1544, 2018.
[11] Yuan Zhang, Xi Yang, Julie Ivy, and Min Chi. Time-aware adversarial networks for adapting dis- ease progression modeling. In 2019 IEEE International Conference on Healthcare Informatics (ICHI), pages 1-11. IEEE, 2019.
[12] Rainer Mühlhoff. Predictive privacy: Collective data protection the context of artificial intelligence and big data. Big Data & Society, 10(1):0553911231 166886, 2023.

---

[13] Feng Liu, Wenkai Xu, Jie Lu, Guangquan Zhang, Arthur Gretton, and Daniea J Sutherland. Learning deep kernels for non-parametrie two-sample tests. In International conference on machine learning, pages 6316-6326. PMLR, 2020.
[14] Shengjia Zhao, Abhishek Sinha, Yutong He, Aidan Perreault, Jiaming Song, and Stefano Ermon. Comparing distributions by measuring differences that affect decision making. In International Conference on Learning Representations, 2022.
[15] Zhen-Yu Zhang, Zhiyu Xie, Huaxiu Yao, and Masashi Sugiyama, Test-time adaptation in non- stationary environments via adaptive representation alignment. Advances in Neural Information Processing Systems, 37:94607-94632, 2024.
[16] Qing Yu and Kiyoharu Aizawa. Unsupervised out-of-distrrbution detection by maximum classifier discrepancy. In Proceedings of the IEEE/CVF international conference on computer vision, pages 95189951855 15266992,, 2019.
[17] Ching-Yas Yao Chuang, Antonio Torralba, and Stefanie Jegelka. Estimating generalization under distribution shifts via domain-invarian representations. arXiv preprint arXiv:2007.03511, 2020.
[18] Yiding Jiang. Vaishnavh Nagarajan, Christina Back, and Zico Kolter. Assessing generalization of sgd via disagreement. arXiv preprini arXiv:2106 13799, 2021.
[19] Tom Ginsberg, Zhongyuan Liang, and Rabul G Krishnan. A learning based hypothesis test for harmful covariate shift. The Eleventh International Conference on Learning Representations, 2023.
[20] Elan Rosenfeld and Saurabh Garg. (almost) provable error bounds under distribution shift via disagreement discrepancy. Advances in Neural Information Processing Systems, 36, 2023.
[21] Andras Janosi, William Steinbrunn, Matthias Pfisterer, and Robert Detrano. Heart Disease. UCI Machine Learning Repository, 1989. DOI: https://doi.org// http://bbboorg/110224433445552
[22] Amol A. Verma, Yishan Guo, Janice L. Kwan, Lauren Laaponte-Shaw. Shail Rawal, Terence Tang, Adina Weinerman, Peter Cram, Irfan Dhalla, Stephen W. Hwang, Andreas Laupacis, Muhammad M. Mamdani, Steven Shadowitz, Ross Upshur, Robert J. Reid, and Fahad Razak. Patient characteristics, resource use and outcomes associated with general internal medicine hospital care: the general medicine inpatient initiative (gemini) retrospective cohort study. Canadian Medical Association Open Access Journal, 5(4):E842-E849, 2017.
[23] Amol A Verma, Sachin V Pasricha, Hae Young Jung, Vladyslav Kushnir, Denise YF Mak, Radha Koppula, Yishan Guo, Janice L Kwan, Lauren Lapointe-Shaw Shail Rawal, al. Assessing the quality of clinical and administrative data extracted from hospitals: the general medicine inpatient initiative (gemini) experience. Journal of the American Medical Informatics Association, 28(3):578-555 7777885777777 2021.
[24] Shafi Goldwasser, Adam Tauman Kalai, Yael Tauman Kalai, and Omar Montasser. Beyond perturbations: Learning guarantees with arbitrary adversarial test examples. In Advances in Neural Information Processing Systems, volume 33, pages 16942-16952, 2020.
[25] James Harrison, John Willes, and Jasper Snoek. Variational bayesian last layers. In International Conference on Learning Representations (ICLR), 2024.
[26] Vladimir N Vapnik. An overview of statistical learning theory. IEEE transactions neural networks, 10(5):988-99)) 1999.
[27] Peter L. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3(Nov):463 482, 2002.
[28) Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv: 1.l:::///0.030... 2016.
[29] Vaishnavh Nagarajan and Zico Kolter. Deterministie pac-bayesian generalization bounds for deep networks via generalizing noise-resilience. arXiv preprint 9rXiv:1905.13344 2019.
12

---

[30] Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. arXiv preprint ar12v0/912.022 2019.
[31] Yoonho Lee, Juho Lee, Sung Ju Hwang, Eunho Yang, and Seungjin Choi. Neural complexity measures. Advances in Neural Information Processing Systems, 33:9713-9724, 2020.
[32] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[33] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International conference on machine learning, pages 5389-5400. PMLR, 2019.
[34] Amin Naemi, Thomas Schmidt, Marjan Mansourvar, Mohammad Naghavi-Behzad, Ali Ebrahimi, and Uffe Kock Wiil. Machine learning techniques for mortality prediction in emergency departments: a systematic review. BMJ open, 11(11)=e05263,, 2021.
[35] Diana Barsasella, Karamo Bah, Pratik Mishra, Mohy Uddin, Eshita Dhar, Dewi Lena Suryani, Dedi Setiadi, Imas Masturoh, Ida Sugiarti, Jitendra Jonnagaddala, et machine learning model to predict length of stay and mortality among diabetes and hypertension inpatients. Medicina, 58(11)11568 2022.
(36] Guillem Hernández Guillamet, Ariadna Ning Morancho Pallaruelo, Laura Miró Mezquita, Ramón Miralles, Miquel Angel Mas, María José Ulldemolins Papaseit, Oriol Estrada Cuxart, and Francesc López Seguí. Machine learning model for predicting mortality risk in patients with complex chronic conditions: Retrospective analysis. Online Journal of Public Health Informatics, 15:e52782, 2023.
[37] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings the IEEE conference on computer vision and pattern recognition, pages 770-778. 2016.
[38] Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. Detecting and correcting for label shift with black box predictors. In International conference on machine learning, pages 3122-3130.
PMLR, 2018
[39] Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Laksh- minarayanan. A simple fix to mahalanobis distance for improving near-ood detection. arXiv preprint arx: 09022, 2021.
[40] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's 's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019.
[41] David Lopez-Paz and Maxime Oquab. Revisiting classifier two-sample tests. arXiv preprint arXiv: 1610.0655, 2016.
[42] Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3723-3732, 2018.
[43] Tuan- Hann-Hnng Vu, Himalaya Jain, Maxime Bucher, Matthieu Cord, and Patrick Pérez. Advent: Ad- versarial entropy minimization for domain adaptation in semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2517-2526. 2019.
[44] Dequan Wang. Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint 2xXiv:2006.10728 2020.
[45] Xin Zhang and Ying-Cong Chen. Adaptive domain generalization via online disagreement minimization. IEEE Transactions Image Processing, 32:4247-4258. 2023.
13

---

[46] Fei Ye and Adrian G Bors. Task-free continual learning via online discrepancy distance learning. Advances in Neural Information Processing Systems, 35:23675-23688, 2022.
[47] Jean Feng, Adarsh Subbaswamy, Alexej Gossmann, Harvineet Singh, Berkman Sahiner, Mi-Ok Kim, Gene Anthony Pennello, Nicholas Petrick, Romain Pirracchio, and Fan Xia. Designing monitoring strategies for deployed machine learning algorithms: navigating performativity through a causal lens. In Francesco Locatello and Vanessa Didelez, editors, Proceedings of the Third Conference on Causal Learning and Reasoning, volume 236 of Proceedings of Machine Learning Research, pages 587-608, PMLR, 01-03 Apr 2024.
[48] Jean Feng, Alexej Gossmann, Romain Pirracchio, Nicholas Petrick, Gene A Pennello, and Berkman Sahiner. Is this model reliable for everyone? testing for strong calibration. In Sanjoy Dasgupta, Stephan Mandt, and Yingzhen Li, editors, Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, pages 181-189. PMLR, 02-04 May 2024.
[49] Yiding Jiang, Vaishnavh Nagarajan, Christina Back, and Zico Kolter. Assessing generalization of sgd via disagreement. In International Conference on Learning Representations, 2022.
[50] Aleksandr Podkopaev and Aaditya Ramdas. Tracking the risk of deployed model and detecting harmful distribution shifts. arXiv preprint 10.0677,,, 2021.
[51] Tony Ginart, Martin Jinye Zhang, and James Zou. Midemon: Deployment monitoring for machine learning systems. In International conference artificial intelligence and statistics, pages 3962 3992-3997. PMLR, 2022.
[52] Ziming Wang, Changwu Huang, and Xin Yao. Feature attribution explanation to detect harmful dataset shift. 2023 International Joint Conference on Neural Networks (LJCNN), pages 1-8, 2023.
[53] Vathy M Kamulete. Test for non-negligible adverse shifts. In Uncertainty in Artificial Intelli- gence, pages 959-968. PMLR, 2022.
[54] David Nigenda, Zohar Karnin, Muhammad Bilal Zafar, Raghu Ramesha, Alan Tan, Michele Donini, and Krishnaram Kenthapadi. Amazon sagemaker model monitor: A system for real- time insights into deployed machine learning models. In Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pages 3671-3681, 2022.
[55] Kamyar Azizzadenesheli, Anqi Liu, Fanny Yang. and Animashree Anandkumar, Regularized learning for domain adaptation under label shifts. arXiv preprint arXiv: 1903.0349,, 2019.
[56] Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of- distribution image detection in neural networks. arXiv preprint
2017.
[57] Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distribur detection. Advances in neural information processing systems, 32, 2019.
[58] Warren Morningsta,, Cusuh Ham, Andrew Gallagher, Balaji Lakshminarayanan, Alex Alemi, and Joshua Dillon. Density of states estimation for out of distribution detection. In International Conference on Artificial Intelligence and Statistics, pages 2232-3240. PMLR, 2021.
[59] Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. Advances in neural information processing systems, 19, 2006.
[60] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jen- nifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79:151-175, 2010.
[61] David Acuna, Guojun Zhang, Marc T Law, and Sanja Fidler. f-domain adversarial learning: Theory and algorithms. In International Conference on Machine Learning, pages 66-75. PMLR, 2021.
14

---

[62] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Domain-adversariaa training of neural networks. Journal of machine learning research, 17(59):1-35, 2016.
[63] Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms. Cambridge university press, 2014.
[64] Arthur Asuncion, David Newman, et al. Uci machine learning repository, 2007.
[65] Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, et al. From detection of individual metastases classification of lymph node status at the patient level: the camelyon17 challenge. IEEE Transactions on Medical Imaging. 2018.
[66] Ajnrk-Aréé Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (clus). arXiv preprint urXiv:1511.072881 2015.
[67] Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448-456. pmlr, 2015.
[68] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, arXiv preprint 1rXiv: 2017.
[69] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit- learn: Machine learning python, the Journal of machine Learning research, 12:2825-2830 2011.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the
paper's contributions and scope?
Answer: [Yes]
Justification: Indeed, in the abstract and introduction, we enumerate our contributions (1), (2), (3). The rest of the main paper demonstrates (I) the practical and scalable aspect of our method, (2) provides empirical validation, while (3) the promised theoretical analysis is referred to in the Appendix section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We do acknowledge noisier results in few-shot settings, provide discussions and provide our interpretation the results. As our method seeks to pad the shortcomings of two prior works, we make those comparisons clear in the discussions section as well as the potential tradeoffs.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
16

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
The theoretical setup is presented with full proofs in the supplementary Appendix. We clearly state that theoretical guarantees are provided for particular problem setup which we devise.
Justification:
Guidelines:
The answer NA means that the paper does include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: Yes. Our code is structured as package and is entirely modular. We report hyperparameters and the random seed schedule used reproduce reported results. In addition, we provide boilerplate code in iPython notebooks for users looking to familiarize with our system and even build upon it.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution
specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
17

---

(d) We recognize that reproducibility may be tricky in cases, in which authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Yes, please see the above.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: htti///uiddes/ooeuubniiiiii
/ /nips.
CodeSubmiss wswsnnolliiio.esssoessooo Policy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Yes. This extensively documented the relevant Appendix section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Yes. We provide standard deviations across 10 confidence interval. To the best of our knowledge, this quantity captures our method and the baseline methods' capability to accurately monitor model deterioration at deployment.
Guidelines:
18

---

The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide an Appendix section for the statement on computing resources and detail our cluster usage. As the functioning of our method depends heavily on correct hyperparameters, sweeping details and costs are also discussed.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https://neurrii http///nuripp..p/ cttppcccccccccccccccccccccccccct
httttp////errppppppppppip/////// e/Ethi SeGGuideeiines"
Answer: [Yes]
Justification: To the best of our knowledge, our research has no potential for harm and does not incur any concerns regarding data privacy.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
19

---

Answer: [Yes]
Justification: Yes. We believe the reliable monitoring of MI model deployment will contribute positively to society. Our method, while achieving this in certain conditions, comes at cost from (pre)conputation, albeit much lesser than some of the baselines we compare to. We discuss this openly. Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
To the best of our knowledge, this work poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes. We extensively use the VBLL library by [25] and cite them properly.
Guidelines:
The answer NA means that the paper does not existing assets.
20

---

The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi theode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/mmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
21

---

Usage of the IM dataset (anonymized for now) is IRB-approved. We do do any research with human subjects, though the data is processed EHR dataset which is produced by measurements humans and interventions humans.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
The core method development in this research does involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /anpas:///uuipee// /2//c for what should or should not be described.
22

---

Appendix
A Theoretical Setup and Analysis
A.1 Overview
In our experiments, find that the D3M algorithm is effective in monitoring deteriorating changes in multi-class classification. As a proof of concept, we provide sample complexity analyses for the binary classification, where deteriorating shifts are shifts in the covariate distribution of data. We show that an ideal D3M algorithm can achieve strong TPR/FPR guarantees at desirable significance levels under these assumptions. We discuss the extent which empirical observations from experiments match with the insights revealed by this analysis.
A.2 mest-eeployment Deterioration (PDD) and Disagreement-baze Deterioration (D-PDD)
Dest-Deployment
Consider a function class H. of h - {0, 1} in the binary classification setting. We use 9 H to denote the ground truth labeling function. We denote the marginal distribution w.r.t as Pz (ar) and the joint distribution with the labeling function in the subscript, that is, for a data distribution Pa over the domain X' and any labeling function g(x), we define the joint distribution as P. Pa (ar y) = (x(,g(())) For a f E H. define its generalization error with respect to P. and its corresponding empirical counterpart with respect to sample D
{(11, y:) P. as:
err(J; Pa) (x.y)~>P,
(f(x) # y
Pa)
Σ
((x) - val
Training and deployment distribution. We denote Pa as the training (marginal) distribution, and Q2 as the deployment distribution. We assume sampled batches of data are I.I.D. with respect their underlying distribution Pa and/or Qz. We consider that labeled samples from P, are available before deployment, and m unlabeled samples from Q₂ are collected during the deployment's input stream.
Disagreement. For any two functions f and h in H. we say that they disagree on any point æ X if f(x) * h(x). Given the binary classification setting, we can write the disagreement rate of the function h with f distribution Q2 in terms of error as (h; Q,) or err Qn).
Moving forward, f E H will be understood to mean our base classifier obtained during training on D Pg. while is the ground truth P. and h H denotes auxiliary classifiers in the same hypothesis space, unless otherwise stated. Our goal to study and monitor the following phenomenon:
Definition (Post-deployment deterioration, PDD). Denote and g' as ground truth labeling functions in the training and deployed distributions P, and Qn respectively. We say that PDD has occurred when:
arr(f:Q'') P,)
(I)
Intuitively, Eq. (1) suggests that PDD occurs when model f experiences higher error during deployment than during its training. Due to the unsupervised nature of the deployment dataset, PDD monitoring is difficult for any arbitrary g' A g as we cannot trivially compute empirical errors for the LHS. Though tools from the literature of OOD error estimation may be used, we propose to proxy via a related notion. Def. introduces new and practical concept-model disagreement-baa baaedd PDD equivalent to PDD under specific assumptions.
Definition 2 (Disagreement based PDD (D-PDD)). We say that D-PDD) has occurred when the following holds for some ef < 1: err(h; P3) 65 Ef and err(f: Ps) S ef and err(h; err(h; Py)
3h E H
(2)
D-PDD Def. 2 defined as the situation where there exists an auxiliary model E H achieving equally good performance on P (with a small error ey) but exhibits strong disagreement with f in Q. In this case, the distribution Q is further referred to deteriorating shift. the following lemma, we demonstrate the conditions for the equivalence of PDD and D-PDD.
23

---

Lemma A.1 (Equivalence condition). Assume that the ground truth training and deployment are identical, Lee. = g'. and that TV(P, Q) S K, we have that when erri (.. eer(j, Ph) 2 2(x+ + E), i.e. the disagreement gap large enough, D-PDD and PDD are equivalent.
Proof. To show PDD = D-PDD, assume g = g i.e. identical concepts during training and deployment. Assume our base classifier is well-trained with err(f. Pg) We have that
err(f, Qg) err(f,Pg)
Let our candidate auxiliary function E H be given by h = g- Then, h satisfies all conditions for D-PDD.
We now show that D-PDD = PDD. Assume that there is no concept shift, i.e. the ground truth distribution is identical, g'
We transport the D-PDD condition 3h E H s.t. err(f. Qn)) condition err(f, Q₂) > err(f, P₂) by leveraging the proximity of h to g P and that the total variation between and Q are constrained by K.
err(f, to the general PDD
We observe that for any f. f.g.h E H:
|err(f, P.) - -er(s,, Px)] <6
Indeed, this is true since:
|err(f, P3) - -er(ss Px)] -
S Ep]l(f+g)
Ep [p]1((
where we used Jensen's inequality, and that [1 + 9) + 1s + h [1{g # n)). Let TV(P,Q P) S k for some k > 0. We further observe that for any f.g E H:
[err(f, Qg) - err(f, = +g) pf * g)l
sup [Q(A) P(A)
A
Putting our two observations together yields following decomposition:
|err(f. Qn)) - err(f. Qg)l < err(f. Qn) - err(ferfff Pr)l
+ |err(f. [f. Ph) crr(f, Py) ) + |err(f,, Pg) err(f. Qg)
<
2n
For PDD hold, Qr)(f.Q,) needs be no less than err(f, Ph) + and at most 2x + less than err(J, Qx). Equating yields:
erT Ph) + (fr(f, Qn) 2k - c
err(J, Qn) - err(J, Pa) 2 2(x + c)
prescribing the conditions for which D-PDD implies PDD.
Thus, if an algorithm monitors D-PDD, then under the assumptions of Lemma A.1, the algorithm also monitors post-deployment deterioration (PDD). In fact, D3M approximately monitors D-PDD. To see this, we show that an ideal version of D3M monitors D-PDD in the above subsection.
24

---

A.3 Idealized D3M and D-PDD Monitoring
Tracking D-PDD in finite samples as formulated requires training data during deployment, which counter to the set of desiderata previously established. To circumvent this, we decouple the detection of D-PDI into two Calibrate and Deploy stages. The Calibrate stage finds a subset Pg in Def. 2 as well approximates err(h; Py).
Hp C. H whose elements satisfy conditions while the Deploy stage tracks the satisfaction of the last inequality. In this way, information from the training data is compressed into Hp and the approximation of err(h; Py). Mcanwhile, the approximation of the disagreement threshold err (h, P,) for Hp can be done via its empirical distribution computed during the Calibrate phase. We thus present the idealized versions of the Calibrate and Deploy stages of D3M.
Algorithm 1 Idealized D3M: Calibrate
Require: D ~ Pg. f, e, H
I: Train sub hypothesis space Hp {he H; (h; Pg) S e)
2: Φ x-]
3: for 1.2...... T do
4: DD" P,
5: h argmax eri D Hr
6: append (h: D to +
7: end for
8: return 4, Hp
Calibration in P. Given in-distribution training data D base model f trained on D an error tolerance 6, and the hypothesis class H. we formulate the subset of H achieving the error tolerance, Hp (h E H; err(h: P3) S c). Then, the disagreement distribution rounds, samples pseudo-labeled by (D") is used to train auxiliary models h by maximizing disagreement between and under Hp on D to approximate disp max err(h; Pf). The hE
trained: for T
empirical disagreement rate achieved by h is appended to o. Finally, the pre- pre-training procedure returns and Hpp
Algorithm 2 Idealized D3M: Deploy
Require: Hp, $ f.a Dmm ~ Q2
2: h + argmax err D hE
3: return err(h; DT) (1 a) quantile of
2. Deploy in Q. Given d and Hp. we compute the one-sample approximation of the maximal disagreement with f on Q: disq max err(h; Q,). We say D-PDD happens when disq lies in the top quantile of +.
The idealized algorithms and 2 together track D-PDD in finite samples. Indeed. deployment Q flagged when the last incquality in Def. 2 is detected, while the imposition of the other inequalities are done via formulating the constrained hypothesis space H,p of hypotheses that already satisfy these inequalities and searching over it. Of note is that at deployment time, the original training set D is not required.
D3M approximates Idealized D3M. The primary implementation consideration in the Idealized D3M algorithms is the search over Hp, which for large function classes cannot be done trivially. D3M "Bayesianly" approximates this search by turning the intractable optimization problem into sampling problem. By drawing from (VBLL2 FFg) posterior, we hoping sample hypotheses that belong in Hp with high probability. When viewed this way, the correspondence between D3M and its idealized version above follows. The price of the increased efficiency from avoiding intractable
25

---

searches Hp is thus the sampling noise from D3M which may return hypotheses beyond the constraints Hp-
A.4 Provable Guarantees of Idealized D3M
We present theoretical guarantees for Idealized D3M (Algorithms and 2), which we refer as D3M for remainder this section. Recall that D3M is tracking a sufficient condition of D-PDD. We show that with enough samples, when there is non-deteriorating shift, the algorithm achieves low false positive rates with high probability. Then, we show that with enough samples, when there is deteriorating shift, the algorithm provably succeeds. Finally, discuss pathological cases where monitoring fails irrespective of sample size.
A.4.1 Preliminary quantities
Definition 3 (Deployed classifier emror). This quantifies the generalization ervor of the deployed base classifier f. This is measured on the distribution seen during training Pg.
cf :===erl. :=err(f:p)
Indeed Def. we want the population error be most e,, which results in the constraint for the empirical error in the optimization problems of Algorithm at most Ey Ep. where eo is syper-parameter measure the gap between the empirical and population error.
We also define the VC dimensions of the hypothesis space H. and the subset of interest Hp as:
Hp (h H. err(h; Ps) S cy), dp := VC(Hp), d d: VC(H))
Note that d, < If the base classifier f is well-trained (ey is low), then dp can be much smaller than d i.e. dp < d.
Definition 4 (Ep, Eg maximum error in Hp). The maximum error in Hy for both P and Q using pseudo-labes labels from f defined as:
Ep
max err Py). Eq max err Q,)
(4)
HER,
Note that empirical quantities of these are also the maximum empirical disagreement rates used in Algo. I and Algo. 2. Effectively, the algorithm detects Eq cp > 0 with finite samples.
Definition 5 (g quantifies D-PDD). We define & to quantify the degree of D-PDD. We adopt Def. and define as
$ : max hEHp
(err(h; Q5) err(h, pf)
(5)
Therefore, D3M detects whether 0. Furthermore, is non-negative since f E Hy. Hence, in case of non hon-dettriooratig shift, g =
Note that & E, Ep. It follows that 6, not necessarily true. Therefore Algo. 2, (Eg Cp > 0) is detecting sufficient condition of D-PDD (E > 0).
- { 0, though the reverse implication is
Next, we relate the amount of D-PDD, &, with the amount of distribution shift in the form of TV- distance between P. and Q.. As seen in the Eq. 5, deterioration depends on the complexity of the function class and ef which affects the size of Hp. We capture these factors by introducing a mixture distribution U:
U 1 -=
(6)
Definition 6 (n error gap between Hp and Bayes optimal). For the distribution U, the gap in error between the best classifier h E Hy in the function class and the Bayes optimal classifier is m.
n min crr(h; U) -err(frovee U)
Note that y depends on the shift and complexity of the function class. We relate various definitions introduced in this section as follows.
26

---

Proposition A.2 (D-PDD and TV distance). The relations between (in Def. 5), 1j (in Def. 6), and Ep. fq (in Def. 3 and 4) are as follows:
{ = TV 22 2 0
(8)
{ 2 Eq Ep 2 6 6 2
(9)
We denote the total variation distance between Ps and Q2 as TV. Intuitively, D-PDD is defined in such a way that after deployment, if we are uncertain of the performance of f. then the shift is deteriorating. In general, for simpler function classes such linear models, by looking at one region of the domain (P,) it may be possible to be certain about the performance of another region (Qz), this is captured in Eq. 8. For very complex function classes, 7/ can be low, hence ε > for most shifts. For simple function classes, can be high, in which case & may not be positive and hence a non-deteriorating shift. This thus highlights trade-off with selecting expressive functions to capture complex patterns in the data.
A.4.2 D3M algorithm in non-deteriorating shift
D3M aims to monitor and detect D-PDD in finite samples, which can inherently lead to false positives (FPR) when the shift is non-deteriorating. Therefore we set a tolerance factor a in Alg. to account for the test's robustness. We show that for D3M, the FPR of the detection shift in the data distribution. Furthermore, we show that the FPR can also be less than a in some cases. In the case of non deteriorating shift, by Def. 2, the following holds:
be close to for any
Vh E Hpp err Q,) S err(h: (f; = Eq S Ep
(10)
Note that D3M intuitively detects whether 6gg > 6p- Since the above equation shows that ey S 6p. given enough samples, the test will succeed. Recall that n is the number of samples given from Ps and m is the number of samples required from Q2. In the theorem below, the significance level refers to the desired FPR.
Theorem A.3. For y S when there is deteriorating shift (no D-PDD) in Eq. 10, for chosen significance level of a, the FPR of D3M is at most y + (1 - O (exp (-neī + d)) if
cp Eq ((-VO)
1- V8
dp (d) (n
m mED
(1I)
and 6, -g 0, where = (dp + In 1n (dp + In 2 and we define eo E -er((;
In the case of non deteriorating shifts (specifically ep > kg) the FPR may be even less than a given that and n are sufficiently large. The more samples from Qa we have, the lesser the FPR in these cases. For any general case, by setting 7 = a (i.e., ô = 1) in the above theorem, we immediately have:
Corollary A.4. For chosen significance level a, the FPR of D3M (Alg. 2) is no more than + (1 a:) O (exp (-neğ + d)).
Practical insights. The corollary asserts the robustness of D3M against unnecessarily flagging non- deteriorating shifts. Independent of the number of deployment samples m, for any given significance level a, the FPR is only slightly worse, with the additive term decaying exponentially in the number of training samples. For many practical ML pipelines that sy-and-large employ linear and forest models among others of manageable VC-dimension, having these guarantees means that D3M audit likely won't negatively impact the continuity and quality of the model usage.
A.4.3 D3M algorithm in deteriorating shift
When deteriorating shift occurs:
3h E Hp err(h; Q,) > ar(h;
(12)
However, this does not necessarily imply that 6g > Cp which is ultimately the condition monitored by D3M. In the following, we break down the possible scenarios.
Regime 1. Deteriorating shift and eg tp. In this case, Theorem A.5 demonstrates that the D3M algorithm detects deteriorating shift with provable high TPR. Here, the significance level a understood to be minus the desired TPR.
27

---

Theorem A.S. For B > O, when deteriorating shift occurs, for a chosen significance level of a, the TPR of D3M (Alg. 2) is at least (1 - (1 (1-0 (exp + +d)) if
(-neg
1+ (1
1))
m mEO o
{ { 2cs/
( 3
dp + In
(13)
and 6q Ep > 0, where 8 (dp + In (dp + In } and Eo E é Py).
Notably, E in the denominator indicates that as the shift becomes more deteriorating, D3M requires fewer samples m to detect, evidencing its effectiveness. Further, having a high-quality base classifier with low Ey is better for detection: this is seen through in Eq. 9 where low ef makes the monitoring more faithful at lower sample complexity m. Another remark is that m depends on dp which can be much less than d with n. being dependent the latter. The test, thus, can work for m significantly smaller than n. The dependency on y. is due to the requirement of satisfaction of the first condition in Def. 2. In the constrained optimization problems Algo. 1, the constraint satisfied but the population constraint will be satisfied either for a larger eo or for sufficiently large n as seen in the above theorem.
Regime 2. (Possible tradeoff) -Deteriorating shift but ey S Cp. In this case, Theorem A.6 demon- strates that either the false negative or false positive rates (FNR, FPR) should be high. The illustration in Fig. exemplifies this failure mode, and how low Ey can help alleviate it.
Theorem A.6. When deteriorating shift occurs and Eg S Cp. for a chosen significance level of a, the TPR of Alg. 2 is O(a).
If a is low, then by Theorem A.6 we have that the FNR is high. On the other hand, if a is high, then by Corollary A.4 we have that the FPR can be very high, thereby trading off the significance level of D3M to reduce FNR but loosening guarantees on FPRs.
A.4.4 Solutions for FNR/FPR tradeoff
This part provides a possible failure scenario illustrated in Fig. 4. Notably, we will highlight a badly trained base classifier / in the possible failure scenario (Fig. 4 (a)), if f is trained with lower ey. can move to the scenarios (Fig. 4 (b) and (c)) where the D3M algorithm can succeed.
(a)
(b)
(c)
Base classifier (f)
h2
Figure 4: Illustration of the FNR/FPR tradeoff and its remedy. The background color indicates the fixed ground truth. Positive and Negative points are from Pa (labeled) and the unlabeled points are from Qu. The solid black curve represents the deployed base classifier f. The dotted Pink (h1) and Blue (h2) curves represent the envelope boundary for H,p i.e., all the functions passing between these two curves are contained in Hp. (a) Failure scenario (i.e. Regime 2) where D3M algorithm fails. (b) No deteriorating shift scenario. (c) Deteriorating shift and the D3M algorithm succeeds. In summery, a decreasing on e could move the failure scenario to the solvable scenarios (a) or (b).
In Fig. 4 (a). if f is not well-trained, we will encounter a failure scenario. The disagreement of hi with f on Q. is larger than that of P., evidencing D-PDD. However, note that hy can maximize Ep more than any function (in Hy,) can maximize Eg. which implies 6p > Cq. If f is better trained in Fig. 4 (b). for all functions in Hp (curves between h2 and h2) disagreement with f on Pa is not less than that of Q Hence there is no deteriorating shift and D3M algorithm could provably address this. Alternatively, if ʃ is trained well and is closest to the ground truth Fig. 4 (c) the disagreement of lz2 with / on Qz is more than that of P,. Also, note that ép 0 since there is no function that can have any error on P,. However, hg can be the classifier to get non-zero eg which gives 6 > 6p- Hence (c) recovers the Regime and is solvable.
28

---

Practical implications. Training base classifiers with strong in-distribution generalization perfor- mance helps in reducing the likelihood of falling into Regime 2. Then, Theorem A.6 guarantees that with high probability, the desired TPR of D3M can be achieved modulo an exponentially decaying factor in the number of training samples. In this way, D3M is robust in monitoring deteriorating shifts with provable TPR guarantees, satisfying the robustness desiderata for PDD monitoring.
A.5 Proofs
Lemma A.7. For any 7 > O, > Ep. we have err(h; Qt) S for all h E Hp with probability at least 1 7 if
dp In
E O
(14)
(10 Eg)
Proof. We the generalization bound for agnostic learning in [63].
CE c
m
2
Pr
- Bh Hp err(h: Q1-1) - (atth:QQ111 Q1-1) 2 2d
(15)
X.Y
mm
Pr
Bh Hp eT; Q2) 2 err(h; Q1) +c]
(16)
x-Y
Pr
Bh E Hp ch; Q1) 2 +
(17)
x-
Choose -4-6 Ey for any μ Eq. Now,
c9 e - Sg
(18)
dp + In
m mEO o
(19)
(14 - (2) 2
Lemma A.8. For any h H. the err(h; Pt) S Ef Eo then with probability least 1 O (exp (-neğ d)), h will be in Hp
Proof. We use the generalization bound for agnostic learning in [63].
-
Pr X.Y~Pr
Bh E H. err(h: P3) - ēří (h; Pg)
(20)
Pr
[h E H err(h; P₂) 2 ērr(h; Pg) + +
(21)
x. x.Y~Pr
2
Pr
Bh E H. err(h: Pa) 2 - Eo +
(22)
X.Y x.Yapy
Choose eo to get
Bh H err(h; Pg) e, s -ede--0 x
Pr
(23)
Theorem A.9. For S. when there is no deteriorating shift, for chosen significance level of os
the FPR of Algo. most y + (1 (1-7) 7) O (exp (-neg d)) if
1 1 V8
(dx (n+
m mE
(24)
Ep Eq
and cp Eg 0, where 8 dn+In 4
dl+
29

---

Proof. We show that in the of deteriorating shift (which implies 6p 2 eg) the false positive rate cannot be more than a and also having more samples from Qx will decrease the false positive rate if Ep > Eq.
We assume that during pre- pre-training phase, while populating we discard disagreement from h & Hp L.., not satisfying the constraint err(h; Py) I ez. We cannot do the same during the detection phase since the detection phase is sensitvve Due to this, we have to account for & Hp in the FPR calculation.
Now, FPR can be written and bounded as follows. Let ja be the disagreement 1 a percentile of
FPR Pr [h; Q₁) 2 nd
(25)
= Pr[[(h & Hp) A (err(h; Q2) 2 n4] v ((h E Hp) A (err(h; Q1) 2 μb]
(26)
< Pr [(h Hpp V {{h Hp) (émr(: Qγ) 2 kHH
(27)
I Pr ih & Hp] + Pr [[er((: (h; Q₁) 2 24 {h HrH] Hp)
(28)
7 + (1 -1)) Pr [h & Hpl
(29)
FPR < 7 (1-7) O (exp n d))
(30)
where last equation comes from A.8 and y == Pr (h; Q,) 2 μ] 1[ E HpHl
Now, we derive sample complexity m in terms of y. Using A.7 on P with - a probability we get
dp In (p- Ep)
m E
(31)
2
dp
We use E Ω cp + +
from above while using A.7 on Q with 1 - probability to get
dptla tg $
dpp In 1 for 7 < a
m O
(32)
(Ep Eg)
Note that since the chosen μ was greater than Ep and we are dealing with the case Ep > Eq. we get that the chosen (a greater than fq. Thus the requirement of is is satisfied for A.7 while using for Q
This theorem shows that when there are non deteriorating shifts (specifically 6, Ex)) FPR may be even less than a, given m and n is sufficiently large. The more samples from Q., we have the lesser the FPR in these cases. For any general case, by setting ? a (i.e., 8 1) in the above theorem, we obtain the following:
Corollary A.10, For a chosen significance level a, the FPR of the D3M algorithm is no more than (1 a)) O (exp (-neğ + d)).
Note that this statement is independent of m and the distribution shift. If n is sufficiently large, the exponential term small. This often the case when the base classifier error y is small, which is an indicator that large number of samples (n) were available from P.. Ignoring non deteriorating shift (and Q. * Pa) cases while calculating & in Algo. 2 does not adversely affect the FPR of the test.
Lemma A.11. For any B > 0, Eg. there exists h Hp such that eir(h; Qy) 2 a with probability at least B if
dg + In 3
m 2
(33)
(En - 1
30

---

Proof. We use the generalization bound for agnostic learning case [63].
06
Rre
2m m
2
Pr
[3 E Hp (h; Q1-5) - err(hh; Q1-1)= 2
(34)
X.Y~Q
Pr
Ah E Hp (r; Q2) S err(h:; Q2) -c]
(35)
X,Y QT
Pr x
h/h E Hp (T; Q1) s <q - c)
(36)
where (a) follows from Def. 4 Choose Eq 1 for any μ < cq
c ém SB
(37)
dg + Ini 1 (Eq - 11)2
m O
(38)
Proposition A.12 (D-PDD and TV distance). The relations between & (in Def. 5), v (in Def. 6), and Ep. fy (in Def. 3 and 4) are as follows:
{ 5= TV 22n 2 0
(39)
{ 2 Eq Ep 2 E 26f
(40)
Proof. Recall the definition of U from 6. We first derive the Bayes enror in terms of TV distance. Let
A = E X Qx(x) s (x(x))
(41)
A' = {I e X Qx(x) > Px(x))
(42)
The TV distance is equal to half of the Lı distance. Note that Pz(A) + PA(A') = and similarly for Q..
TV(P,, Q=) = (P)(A) (PP + - Pa(A')
(43)
= Pa()) - (xx
(44)
Now, we use the definition of U and the above TV relation to get the following
erT (fbayes U) (orr (Foayes: Pf) + err (Foayes Q1-1)) P
(45)
(1 TV((2 Qx))
(46)
2
Next, with the above result and ry in Eq. 6 we derive Eq. 8
17 + err( fbayes U)= = min err(h; U) 2 neH,
min (err(h; + err(h; Q1-1))
(47)
h
21 + - TV = min
(err(h; P,) + err err Q1-5)) 2 bER,
min err(h; Q1
(48)
KEH,
2n + TV
min HEHp
(err(h; Py) err (1; Q,)) +1
(49)
2n TV
min hEHp
- (err(h: Q,) err (h: Pj))
(50)
TV - 2n
(err(h; Q,) err(h: p,)) E
(51)
hEH
For Eq. 9, we use Eq. 48 and the above result to get the following
Eq max err(h; Q,) = =1 min err(h; Q1-1) TV 27 =
(52)
k
k
With some abuse of notation, the same notation for both pdf and probability measure.
31

---

We can write inequality for errors similar triangle inequality as follows
err(h; (h; Py) x err Pg) + err(g; Pς)
(53)
err (h; Pa) + Crr(f (f: Pa) crr(h; Pa) of
(54)
6p
max err(h; Pу) < max err (h; Ps) + +e, 's- = 2 heHb
(55)
H
The last equality follows from the definition of Hp. Thus we get
Cq cp
Ep 2 5 - 2
(56)
By definition it follows that & 2 Eq Ep
Ep
Proposition A.13. For B > O, when the deteriorating shift occurs, for a chosen significunce level of o, the TPR of Algo. 2 is at least (1 3) (1 O (exp (-neğ + d)) if
2
1 + v8
dp + In }))
1
m mEO
(57)
- 25g/
3
dn+In 1 dp 8
and Eq - Ep > 0, where
Proof.
Similar to the proof of Theorem. A.3, we derive the statistical power (TPR) of the test as follows. Let μ be the disagreement at 1 - a percentile of ab
TPR Pr (e; Q5) S 14]
(58)
= 1 Pr [[h & Hpp A {err(h; v H₂} A (ēTT(h; Q,) S |H
(59)
2 Pr [fh & Hp] {(h E Hpp A ((T(h; sPlll
(60)
2 1 Pr [h & Hpp Pr [ h: Q2) S [h Pr]] E Hpl
(61)
(1 B) Pr |h E Hpp
(62)
TPR E (1 3) (1 - (exp (-ne) + d)))
(63)
where last equation comes from A.8 and == Pr [ Q1) S /T (h HpHl
Next, we derive the sample complexity m in terms of B. We show that there exists a 2 such that both A.7 (for P and a)) and A.11 (for Q and (3) hold.
cp
Kg
(64)
This implies some μ exists if c, tp > 0 We find optimal μ² such that the maximum of m in Eq. 14 and Eq. 33 is minimized.
dp + In 1
/ Cp
8
(65)
Eg 11
dp + In 3
4¹ Eo + voe,
(66)
1 + v8
Plugging this n² in Eq. 33 gives
( d In 1
d In d In 3
m meo
3
1
a
(67)
(p) cp) 2
Use Eq. 9 to get the result.
Ç in the denominator indicates that as the shift becomes more deteriorating, it is easier (fewer samples m) to monitor, indicating the effectiveness of the D3M algorithm. Also, having high-quality base classifier (low E,) is better for D3M which was also seen in Eq. 9 where low Ey makes the algorithm more faithful. Note that depends on dp which can be much less than d which depends on, suggesting that monitoring may be effective in few-shot settings. The dependence on n is due to the requirement of satisfaction of condition in Def. 2. In the optimization problems in Algo. 1, the empirical constraint is satisfied but the population constraint will be satisfied either for larger 6o or for sufficiently large seen in the theorem.
Next, we move to the regime where deteriorating shift occurs but 6y - kg S 0. As a negative result, the following theorem states that in such cases the statistical power of the test is low.
32

---

Theorem A.14, When deteriorating shift occurs and Ey S cp. for chosen significance level of the statistical power of the test in Alg. is (a))
Proof. From the proof of Theorem. A.5 have
TPR 2 (1 - 3) (1 - (exp + +d))
(68)
B = Pr ((rr( S Hp}]
(69)
Using A.7 on P and a we get
a ( (-n(u - ep) dp))
(70)
Using A.7 on Q we get
Pr (r( Qy) 2 2 [h E Hp)) E E (exp) (-n(u-Eg) + dp))
(71)
1 3 O (exp (-n(j² en) dp))
(72)
1 3 0 (a)
(73)
The last equation follows since we are dealing with the case where cp 2 C.. Thus, the TPR is (())
irrespective of the magnitude of n, as desired.
33

---

B Experimental Setup and Additional Details
B.1 Baselines Details
We compare D3M against several other methods from the literature that either detect distribution changes or can be converted into PDD monitoring protocol. Let X (x(+)) Ji=1 from Pa and
Y ( J== from Qx be given. These algorithms seek to accept or reject the hypothesis that
Pz = Q2 in distribution.
1. Deep Kernel MMD (MMD-D, [13I) The algorithm first learns a deep kernel by optimizing a criterion which yields the most powerful hypothesis test. With this learned kemel, permuta- tion tests are run multiple times in order to determine a true positive rate for the algorithm. We interface the authors' original source code with our repository and recycle their training procedures. Theirs can be found at https: // gi thub.
2. H-Divergence (H-Div, [14]) The algorithm fits Gaussian kernel density estimates for Pa- Qz. and their uniform mixture (P= + Qz) Then, permutation tests are performed using the test statistic H2 Q.)/2) ( H,({))) where H2 is the H-entropy with f(x, a) the negative log likelihood of x under distribution estimated by the Gaussian kernel density, order determine a true positive rate for the algorithm. This test statistic is an empirical estimate of the H-Min divergence. The choice of the particular H-divergence is a hyperparameter and is problem dependent, as well as the choice for how to generatively model the data distributions. The original paper further experimented with fitting Gaussian distributions as well as variational autoencoders (VAEs), both of which are not explored here. We interface the authors' original source code with our repository. Theirs can be found at httpp::/ //gi:::///iie thub. com/ a7b23 (1oo22/H-Divee Hitr//---iveggenee..e cree/main.
3. f T-Divergences (KL-Div for KL-Divergence, JS-Div for Jensen Shannon Divergence, [61]) f f-divergence generalizes several notions of distances between probability distributions commonly used in machine learning. this paper, we convert the Kullback-Leibler (KL) and the Jensen-Shannon (JS) divergences, particular cases of f-divergences, into permu- tation tests. More specifically, we first fit Gaussians on samples coming from Pg and Q₂ using maximum likelihood. In the case of KL-divergence, the empirical KL-divergence is computed between the fitted Gaussians whereas for the JS-divergence, we an additional Gaussian on the mixture distribution M and leverage the identity:
Jensen-Shannon
JS(P= 2 (KL
M) + (L(Qu||M)
We run permutation tests by permuting the samples in the union (X ~ PP) )")). It is worth noting that as with H-divergence, more elaborate generative models could be fitted onto samples X and Y. which we do not explore in this work.
4. Black Box Shift Detection (BBSD, (38) involves estimating the changes in the distribution of target labels p(y) between training and test data while assuming that the conditional dis- tribution of features given labels ((rly) remains constant. This is achieved by using a black box model's confusion matrix to identify discrepancies in the marginal label probabilities between the training and test distributions, allowing detection and correction of the shift. We borrow the experimental setup directly from [1].
5. Relative Mahalanobis Distance (RMD, Rel-MH, Rel. Mahalanobis, [39]) RMD modifies the traditional Mahalanobis Distance (MD) for out-of-distribution (OOD) detection by account- ing for the influence of -in-discriminative features. It subtracts the MD of a test sample to a background class inaseinndeeendent Gaussian from the MD to each class-specific Gaussian, effectively isolating discriminative features and improving OOD detection, especially for near-OOD tasks. We test for shift by performing a KS test directly on the distribution the RMD confidence scored computed on Q. and P..
6. Classifier two-sample test (CTST, C2ST, a.k.a. Domain Classifier, DC, [41]) Following the procedure presribed by the original work, a binary domain classifier is trained to predict whether a sample came from P. Q.. Then, a held-out mixture of data from P, and Qz. we compute the domain classifier's 's accuracy and compare its performance to random chance using binomial test.
34

---

7. Deep Ensemble (Deep Ensemble, [40]) An ensemble of neural networks trained inddpen- dently on the entire training dataset using random initialization. A KS test is then performed on the distribution of entropy values computed from each sample in P and Q.
B.2 Standard Benchmark Datasets
UCI Heart Disease. The UCI Heart Disease dataset (UCI) [21, 64] includes 76 variables gathered from four distinct patient cohorts located in Cleveland, Hungary, Switzerland, and the VA Long Beach. To reduce the impact of missing data, we focus on nine out of the 14 most frequently used age, sex, chest pain type, resting blood pressure, serum cholesterol, fasting blood sugar, resting ECG results, maximum heart rate achieved, and exercise-induced angina. The objective is to predict heart disease diagnosis, measured
features:
scale from 0 to 4 where 0 denotes disease and values through 4 indicate increasing severity related to arterial narrowing. Using the proposed setup in [19], the task is binarized, distinguishing between patients with a normal angiographic diagnosis (label 0) and those with any abnormal diagnosis (label greater than 0). The Cleveland and Hungary datasets serve the ID domain, while Switzerland and VA Long Beach datasets form the deteriorating OOD domain.
CIFAR-10/10.1. The CIFAR-IO image dataset [32] consists of 60,000 color images, each sized 32x32 pixels. divided into 10 different classes such as airplanes, cars, birds, and dogs. The dataset split into 50,000 training images and 10,000 test images, with each class represented equally. Due to its manageable size and diversity of categories, CIFAR-10 is commonly used for testing and comparing the performance of image recogmition models. The CIFAR- 10.1 dataset [2] is a test set designed to evaluate how well models trained on CIFAR-10 generalize to new, but similar, data. It consists of 2,000 images collected in a way that closely matches the original CIFAR- 10 distribution, but from separate data source to reduce the risk of overlap or memorization. CIFAR-10. was introduced to assess model robustness and identify potential overfitting to the original test set. Despite its similarity to CIFAR-10 10, many models perform slightly worse on CIFAR-IO. 10.1, highlighting the challenge of generalization in machine learning.
The CIFAR-10 dataset is used as the ID dataset with which the mean model of D3M is trained, while deteriorating shift from CIFAR-10. Thus, we gauge the
the CIFAR-10.1 dataset is considered ability to recognize CIFAR- 10.1 as deterioratingly OOD by D3M and its competing baselines.
Figure 5: (Left) Random samples from CIFAR-10.1. (Right) Random samples from CIFAR-10's test set. The images above are borrowed from Recht et. al. "Do CIFAR-10 Classifiers Generalize to CIFAR-10?" [2].
Camelyon17. The Camelyon17 dataset a challenging histopathology image classification dataset originally described by [65]. It comprises 327,680 color images (96x96) extracted from lymph node tissue slides, with binary labels indicating the presence of metastatic cancer in the central 32x32 region. Following the WILDS setup, treat the hospitals from which data collected as domains: hospitals 1, 2, and 3 are used as source domains for training, while hospital 5 serves as the target test
35

---

domain. We use the WILDS framework [4] handle dataset download, preprocessing. and domain partitioning.
Train
Val (OOD)
Test (OOD)
Hospital
Hospital
Hospital 3
Hospital
d Hospital
y Normal
y Tumor
Figure 6: Samples from the Camelyon17 dataset, where hospitals 1, 2, 3 are treated as ID, while hospitals and 5 are deterioratingly OOD. The image above borrowed from Koh and Sagawa et. al. "WILDS: A benchmark of in-the-wild distribution shifts" [4].
B.3
The GEneral Medicine INpatient Initiative (GEMINI) Dataset
Year
Patient Count
Label Ratio
Pre-2017
72316
3.99%
2017H2
17208
3.60 %
Age
Patient Count
Label Ratio
2018H1
18233
4.15%
18-52
33220
0.82 %
2018H2
18469
3,83%
52-66
33146
2.36 %
2019H1
19041
3.50%
66-72
31048
3.36
2019H2
18601
3,49%
-6 85
34055
4.77 %
2020H1
15575
4.50%
85+
32399
7.82
2020H2
11155
3,48%
2021H1
10625
3.46%
Table 4: Age Split Data Summary
2021H2
7396
2.95%
Table Temporal Split Data Summary
GEMINI Dataset: Study and Preprocessing. The GEMINI study is retrospective cohort study of adult patients and their clinical and administrative data[22, 23). This analysis used data from over 200,000 100,000 patients from the GEMINI database, spanning 7 different hospitals that participated in the GEMINI Study. Each patients information processed into 900 features including but not limited to: (i) laboratory results and vital results collected up 48-hours after admission, split into 6 hour intervals, (ii) patient demographic information: age, sex etc, (iii) Patient diagnosis using ICD-10-CA codes. Missing feature values are imputed based on simple averaging. The predictive task related to this data is 14-day mortality for patients based on these collected features. This dataset is temporal by nature data splits are arranged per half-years starting from 2017, where shifts in patient features become even more pronounced during the COVID period of 2020-2021. Since our D3M base models trained on data before 2018 8 surprisingly do not underperform on later splits (Figure 2), we make the assessment that this shift is non-deteriorating.
Furthermore, an important feature of this dataset is its inherent temporal covariate shift, the evidence of which is the elevated FPR of divergence/distaace-basee (MMD-D) and Relative Mahalanobis Distance (Rel-MH). Functionally, this means that patients' statie covariates as well as their labs, medications, and medical interventions have drifted over time, especially into the COVID years.
detectors of Deep Kernel MMD
36

---

Data Splitting and Shift. Based on this pre-processed data, 2 shifts analyzed: (i) temporal shift, and (ii) age-group shift. The temporal shift analysis splits data into half-years- 2018H1, 2019H2, etc. The baseline model uses 701 and prior data for training, and 2017H2 for validation; Tab. 3 shows patient statistics for this split. It is subsequently tested on unseen in distribution data and later splits. The different age groups are created by splitting the data into 5 equally sized groups based on ages of patients: (1) 18-52, (2) 52-66, (3) 66-72, (4) 76 85, (5) 85+; Tab. 4 shows patient statistics for this split. The reported analysis trains baseline model on group (18-52) and then tests on test-sets that contain some portion of data from the 5th group (85+) and the remaining as unseen in distribution data. The portions [0.0, 0.2. 0.4, 0.6, 0.8, 1.0] represent what percentage of the test set OOD (from group 5), whilst the remaining amount is IID (from group 1). For example a ratio of 0.2 means 20% of the test data from group S(OOD) whilst 80% is from group I(ID). We chose to experiment on such portions instead of just subsequent age groups as this process better displays the TPR of D3M as well as baselines with respect to the degree of shift performance deterioration.
B.4 Configurations for FEg and VBLL
Tabular features. In the tabular setting, we chain affine layers of similar hidden dimension coupled with exponential linear unit (ELU) non-linearities [66]. We use a standard dropout rate of 0.2 across all experiments and employ skip connections between hidden layers. This architecture is used in all experiments with the UCI Heart Disease dataset as well as the GEMINI dataset.
Convolutional features. For CIFAR-10/10, we pass the input through an initial convolution and max-pooling, then through sequence of same-dimension convolutions with batch normalization [67] and skip-connections. Another max-pool is applied, before the representation is flattened and forwarded through an affine layer to obtain a representation.
For Camelyon17 7 experiments, we employ pre-trained ResNets [37] and either train from scratch, finetune them during the training of the VBLL& layer, or freeze them while only training the VBLL layer.
Variational Bayesian Last Layers (VBLL). We borrow the implementation from [25] which can be readily coupled with the above neural feature extractors for end-to-end ELBO maxi- mization. In our experiments, we employ the VBLL variant for discriminative classification www.DDDccasssffiiocoo
and parametrize the covariance matrix of the normal distribution of logits as diagonal. As for VBLL's hyperparameters, the prior scale and wishart scale hyperparam- eters are as described the original manuscript, while we use the regularization factor to VBLL to be a factor of n-1 where is the size of the training set. This factor controls the weight of the KL estimate during ELBO maximization at training.
B.5 Sweeping and Hyperparameters
In each experiment, for each test sample size we run hyperparameter sweep in order to identify hyperparameters that (1) achieves the most consistent low FPR when tested in-distribution, and (2) achieves the highest deterioration monitoring TPR. As mentioned previously, the distribution of disagreement rates can be overfitted so that D3M flags any sample, regardless of whether they are ID or not, thus justifying the choice of selecting hyperparameters jointly satisfying (I) and (2).
Once the best sets are identified, we run 10 independent seeded runs for each test sample size m 10, 20, 50. We report used hyperparameters in Tables 5 and 6 for transparency and reproducibility. In CIFAR-1O/10.1. "Hidden dimension" refers to the dimensionality of the final output of FEg, and test size m is identical for D3M and other baseline algorithms for each set of experiments.
Common to all setups. For all experiments, the number of posterior samples K is set to 5000, the size of the empirical distribution of maximum disagreement rates |+| is set to 1000. For each experiment, once Train and Calibrate is completed, we deploy the model on 100 independent samplings of the questionable test data. If model deterioration occurs (this is the case for the standard benchmark experiments as well as the GEMINI dataset temporal shift experiment but not the GEMINI dataset age shift experiment), we report the number of times D3M flagged these deteriorating samples out of 100 as TPR. Finally, confidence statistics are aggregated and computed on TPRs reported from seeded, independent Train- Calibrate- Deploy D3M cycles. All optimization is done using Adam W
37

---

Table 5: Hyperparameters for UCI Heart Disease and GEMINI datasets
Group
Hyperparameter
UCI
GEMINI
Learning rate
10
1 x 10-3
Batch size Epochs
64
64
Train
50
50
Weight decay
10-1
x 10-4
Hidden dimension
16
128
Model
Num. hidden layers Dropout
4
4
0.2
0.2
Regularization factor
100.0
100.0
VBLL
Prior scale
1.0
1.0
Wishart scale
1.0
1.0
D3M
Sampling temperature
1.0
1.0
Test size
10, 20,
200
Table 6: Hyperparameters for CIFAR-10/10.1 and Camelyon17 datasets
CIFAR-10/10.1
Camelyon17
Group
Hyperparameter
Value
Group
Hyperparameter
Value
Learning rate
10-3
Batch size Epochs
64
Learning rate Batch size Epochs Weight decay
1 x 10 256
Train
10
Train
Weight decay
1 x 10
1 x 10
Hidden dimension
256
Num. mid layers
3
ResNet type
ResNet34
Model
Initial kernel size
9
Model
From pretrained
True True
Kernel size
7
Freeze features
Num. mid channels
128
Regularization factor
100.0
Regularization factor
10.0
VBLL
Prior scale
5.0
VBLL
Prior scale Wishart scale
1.0
Wishart scale
2.0
1.0
D3M
Sampling temperature
D3M
Sampling temperature
1.0
Test size
10, 20, 50
Test size
10, 20, 50
[68]. Finally, we start seeded, independent, identical runs beginning at seed = 57 since it is our favorite prime number, and increase seed by +1 for each run.
Only reporting TPRs of runs achieving low FPR. Importantly, we aggregate only TPRs achieved when FPR below 0.10 in-distribution is achieved. This FPR is calculated on a held-out ID validation set that D3M has not yet seen during Train nor Calibrate. Therefore, for all experiments we run seeded runs until 10 computed, as certain runs do not achieve the ID FPR tolerance desired.
recording ID FPR below 0.10 are found, and their TPR statistics
When deploying D3M in real-world healthcare setting, for instance, upon the completion of the Calibrate step, one could imagine validating this calibration step by computing an ID FPR score, independent of deployment. IF this ID FPR score is higher than tolerated threshold a, the practitioner could either increase the size of to eliminate noise from sampling, or consider finding another set of hyperparameters that would lead to more stable calibration.
B.6 Additional D3M Results on Standard Benchmark
We further tested D3M on 100 and 200 calibration/test samples on the same set of hyperparameters above. Table summarizes our findings.
38

---

UCI Heart Disease
CIFAR 10.1
Camelyon 17
100
200
100
200
100
200
D3M (Ours)
.931.10
.991.01
911.11
.991.01
1.01.00
1.0
Table 7:
True positive rates (TPR) for D3M across datasets and test sizes 100, 200.
At higher test sizes, D3M achieves near-perfect TPR. This is consistent across all experiments. This suggests that D3M could be an effective tool to monitor model deterioration. However, of note is that for test size 100, in the UCI Heart Disease and CIFAR-10/10. experiments, D3M is still logging unusually high standard deviation, where we suspect the noise to be coming from the sampling procedure from the VBLLg distribution of logits.
Trading off deployment-time computations with sweeping D3M is efficient when comparing to other disagreement-basse -aased detection and monitoring algorithms. However, the method is inherently noisier due to several levels of approximation to its idealized version (Algorithms and 2). In particular, we found that D3M is sensitive to the choice hy pyrrpaaraeteess order to achieve great performance. Therefore, the payment of computational cost is carried through prior to deployment the sweeping itself, rather than during deployment is done in [19]. We argue, however, that this is preferable in edge deployment scenarios, such as in hospitals or embedded systems, where real-time responsiveness and resource constraints are critical by front-loading the computational burden during the sweeping phase. One can imagine that for an Al terminal as part of a hospital computational infrastructure for instance, the ability perform robust detection and monitoring with minimal overhead at deployment time significantly enhances reliability and usability in practice.
B.7 Statement on the Usage of Computing Resources
All experiments were run High Performance Computing (HPC) clusters.
UCI Heart Disease. UCI Heart Disease experiments were run on GPU nodes with at minimum 8GB of GPU memory, 6 CPU cores, and 8GB RAM. The total runtime of D3M prior deployment is less than 5 minutes.
CIFAR-10/10.1. CIFAR-10/10011 experiments were run on GPU nodes with at minimum 24GB of GPU memory to accomodate the largest configurations of convolutions, 12 CPU cores, and 12GB RAM. The total runtime of D3M prior to deployment is less than 10 minutes.
Camelyon17. 1 experiments were run on GPU nodes with at minimum 80GB of GPU memory accomodate the largest ResNets during sweeping, 12 CPU cores, and 12GB RAM. The total runtime of D3M prior to deployment is less than hour.
GEMINI Dataset. Experiments on the GEMINI datset were run on GPU and CPU nodes. We request at minimum 16GB of GPU memory (when applicable), 9 CPU cores, and 32GB of RAM. The total runtime D3M prior to deployment is less than I hour. Although models for GEMINI were significantly smaller than vision models for Camelyon17 and CIFAR-10/10.11 1, due to the number of samples available as well as older CPU hardware, the runtime was significantly extended.
39

---

C
Ablations
To complement the theoretical analysis and main results, we provide additional ablation studies that clarify the empirical behavior of D3M. Each ablation isolates specific modeling or design choice, allowing us to assess its contribution to performance, stability, and efficiency. These results also serve validate
claims about scalability and robustness, while highlighting trade-off's between alternative design choices (e.g.. linear vs. VBLL heads). We organize this section as follows: (i) VBLL head vs. Linear head, (ii) computational costs, especially when compared with Detectron, (iii) sensitivity to hyperparameterss such as temperature and ID thresholds, and (iv) the relationship between empirical oversampling and the theoretical D3M framework. Importantly, where applicable. all ablation experiments are run with the same best hyperparameter sets which we isolated during our sweeps and reported in Appendix B.5.
C.1 VBLL Head vs. Linear Head
Implementing a VBLL head is what allows D3M to sample alternate hypotheses (re: Idealized D3M formulation, i.c. Algorithms and 2). It is important that transitioning to VBLL from classical linear head does not amount to significant decreases in base model performance on the ID task. The following table reports model accuracy, Fl score, AUROC, and Mathew's Correlation Coefficient (MCC), all computed on an ID validation set, along with deployment accuracy drop.
Table 8: Comparison of D3M vs. Linear head (Lin) across datasets.
Dataset
Model
Accuracy
Fl Score
AUROC
MCC
Acc. Drop OOD
UCI
D3M
0.76 = 0.02
0.75 = 0.02
0.86 = 0.01
0.51 = 0.04
-0.11 0.02
Lin
0.77 1: 0.01
0.76 = 0.01
0.87 + 0.01
0.52 1.52:00 0.02
CIFAR-10/10.1
D3M
0.70 = 0.02
0.70 = 0.02
0.96 = 0.00
0.67 0.02
-0.13 = 0.02
Lin
0.72 = 0.01
0.72 = 0.01
0.96 = 0.00
0.69 0.01
Camelyon17
D3M
0.94 0.01
0.94 = 0.01
0.98 = 0.00
0.88 0.01
-0.05 = 0.00
Lin
0.94 0.00
0.94 = 0.00
0.98 + 0.00
0.88 0.01
In the above, for multi-class classification datasets, F1, AUROC, and MCC are averaged across all pairs of classes. All hyperparameters of the feature extractors in the above setup are kept the same. We confirm that consistently across all metrics, the performance of D3M's base model with a VBLL head nearly matches that of base linear head. We argue that this is a worthwhile tradeoff in order to leverage hypothesis sampling with VBLL. We observe that there is virtually no performance drop for Camelyon17 We suspect this is due to the base model being finetuned from pretrained ResNet-34 on ImageNet-INe 1K, having learned more general representations, as opposed to our ConvNet feature extractors implemented for the CIFAR- 10/10.1 experiments that may have overfitted to the ID training dataset.
C.2 Computational Costs Analysis
To better understand the scalability of D3M relative existing approaches, we report the floating point operation counts (FLOPs) required by each algorithm across their main computational stages. Table 9 presents the breakdown of cost into (1) base model training and (2) calibration, along with the total compute required. We assume a query size of 100, and all hyperparameters are identical to those reported in the experimental setup.
We observe several consistent patterns across datasets and architectures. First, in D3M, the bulk of the computational burden lies in the one-time base training of the feature extractor, after which calibration incurs only a marginal cost. This scaling behavior arises because calibration in D3M consists of freezing the backbone in evaluation mode and drawing samples from the VBLL layers. As a result, no backpropagation or fine-tuning steps are required once the base model has been trained.
In contrast, Detectron's 's design requires tunitt on held-out subsets during calibration. This entails repeating gradient computations and updating parameters at deployment time. Consequently, the calibration phase is nearly expensive the initial training phase, and in some cases exceeds
40

---

Table Comparison of FLOPs between D3M and Detectron across computational stages.
Algorithm
Dataset
Base Model Training
Calibration
Total
D3M
UCI
0.23 GFLOPs
9.00 GFLOPs
9.23 GFLOPs
Detectron
UCI
0.15 GFLOPs
18.69 GFLOPs
18.84 GFLOPs
D3M
1IFAR-10/10.1
1.56 PFLOPs
0.13 PFLOPs
1.69 PFLOPs
Detectron
CIFAR-10010.1
1.56 PFLOPS
1.95 PFLOPs
3.51 PFLOPs
D3M
Camelyon17
12.02 PFLOPs
0.74 PFLOPs
12.76 PFLOPs
Detectron
Camelyon17
12.02 PFLOPs
11.04 PFLOPs
> 23.06 PFLOPs
it. For example, on Camelyon 17, calibration in Detectron adds over 11 PFLOPs on top of the 12 PFLOPs needed for training, while D3M incurs less than PFLOP in calibration overhead.
These results highlight that as feature extractors scale in size, D3M's ss compute is dominated by the unavoidable cost of initial training, whereas Detectron continues accrue significant additional cost in deployment. The difference grows more pronounced for large-scale datasets and larger-scale models, where Detectron effectively doubles its compute requirements, while D3M remains lightweight in its post-training stages. This distinction is especially important for real-world deployment settings, where calibration often needs to be performed repeatedly under striet resource constraints. In sum, FLOP comparisons show that D3M achieves up to 10x training reduction and 2x overall reduction versus Detectron, requiring no training data once the base model completes training.
C.3 Temperature Sensitivity and ID Thresholds
We conduct an ablation study to assess the effect of the temperature parameter used in sampling from the VBLL layers. Figure 7 report results across UCI, CIFAR-10/1O.1 1, and Camelyon17. Metrics include the false positive rate under II) data (FPR ID), true positive rate (TPR) for deteriorating OOD detection, and the mean disagreement rates both ID and OOD samples.
Across datasets, we observe a consistent trend: increasing the temperature broadens the effective sampling distribution, which amplifies disagreement rates for both ID and OOD samples. While this increases diversity among hypotheses, also skews the base ID disagreement rate distribution toward higher values. As a consequence, the separation between ID and OOD disagreement rates diminishes, leading to elevated FPR ID and reduced TPR for deteriorating OOD detection. For instance, on CIFAR- 10/10.1, 1, raising the temperature from 2 to 10 causes TPR to collapse from nearly perfect detection to just 0.34, despite higher disagreement values overall. Similar trends are evident in UCI and Camelyon17.
We further plot the gap between the mean ID and OOD disagreement rates across our datasets and report in Figure 8. We find that as the temperature increases, the mean disagreement rates in- and out-of-distribution tend closer to each other. Since D3M's core functionality is to track this difference, as the temperature increases, ID and OOD become harder to distinguish on known deteriorating tasks, which explains why the TPR D3M drops as temperature т grows.
These findings highlight the critical role of temperature as an important tradeoff parameter where high temperatures blur the ID-OOD boundary and degrade detection reliability. In practice, we find that values in the range 1 to 5 offer the most stable trade-off across datasets, providing both sufficient hypothesis diversity and reliable threshold calibration.
41

---

0.125
0.8
0.100
FPR 11
TPR
0.6
0.076
0.050
0.4
0.025
0.2
10
15
20
15
1.0
Disogreement Rate OOD
Dissgreemen Rate ID
0.8
0.8
0.0
0.6
0.4
0.4
0.2
20
15
Temperature T
Temperature
UCI
10/10-
Camelyon17
Figure 7: Ablation on the temperature parameter for D3M. We plot the ID FPR, TPR, ID mean disagreement rate, and deteriorating OOD mean disagreement rate across temperatures {1, 2, 5, 10, 20). We remark that as the temperature increases, softening the sampling logits distributions of D3M's Calibration and Deployment, while the ID FPR stays roughly within the same range, the TPR tanks significantly across datasets.
Finally, study the variation of the ID threshold during the Calibration step itself ID disagreement rates measured on bootstrapped ID validation subsets get appended to 4. The ID threshold here refers to the (1 a) Quantile of Φ where disagreement rates beyond the ID threshold labeled as deteriorating OOD by D3M.
Table 10: ID thresholds for 0.1 across 10 independent runs. Columns are the rounds t T 1000 from which the 1 - a quantile is computed for db
Dataset
=
t= 50
100
== 500
7 1000
UCT Heart Disease
0.457 + 0.011
0.468
0.007
0.467 + 0.004
0.468 + 0.004
0.470
0.000
CIFAR-IOM 10/10.1
0.452 + 0.013
0.464
0.009
0.463 - 0.008
0.465 + 0.005
0.463
0.005
Camelyon17
0.306 1 0.011
0.312 0.004
0.312 0.004
0.310 1 0.000
0.310 0.000
Table 10 shows that although some variability is exhibited for early < T. the threshold stabilizes with more independent rounds, as evidenced by the decrease standard deviation. Across our datasets, observe that the ID threshold seems to increase with more independent realizations of Calibration rounds, as observed by the pronounced increment going from
to t = 50, but stabilizes beyond this point. A rough recommendation is to choose T = / 2 50, but note that each round t < T has the same wall-clock time as they all involve the same 1. bootstrapping a random ID validation subset with replacement from ID validation set, 2. sampling K candidate hypotheses, and 3. computing maximum disagreements across all candidate hypotheses.
C.4 Oversampling of the ideal restricted hypothesis space Hp
We investigate the gap between the empirical version of D3M and Idealized D3M (Algorithms 1 and 2). In particular, we investigate whether hypotheses sampled during the Calibration and Deployment phases of D3M belong in Hpp = {h H. err(h, Pa) {f}. or that D3M oversamples H. and
42

---

0.10
UCI
CIFAR-10/10.
Disagroemen Gap (OOD ID)
Camelyon17
0.06
0.04
0.00
2
5
10
20
Temperature
Figure 8: Evolution of ID-OOD disagreement gap over temperatures We find that the gap decreases monotonically (with the exception of spike at = for Camelyon17) across datasets as increases. Since D3M's core functionality relies distinguishing deteriorating OOD disagreement rates from ID disagreement rates, too high temperatures result in worse disagreement gaps, which lead to significantly worse TPRs.
obtains hypotheses beyond it. Our experiment setup is follows. During Calibration, instead of sampling labels and computing maximum disagreement rates for bootstrapped unsupervised ID datasets, we sample labels for this bootstrapped set concatenated with the original training along with the true training labels. For the Monte-Carlo sample resulting in the maximum disagreement rate (only computed on the bootstrapped set), we compute the accuracy against the labels of the training set. For | 1000 independent runs on the UCI dataset and the CIFAR- 14-10/10.. datasets, we report the mean and standard deviation of training accuracy scores, the mean model's 's validation score, and the percentage drop.
Table 11: Comparison of accuracy of MaxDisRate hypothesis against the ID validation accuracy across datasets
Dataset UCI
MaxDisRate Acc. (Train)
Validation Ace. (ID Valid)
% Difference
0.65 + 0.02
0.76 0.02
-11%
CIFAR-10/10.1
0.55 + 0.00
0.70 + 0.02
- 115%
Camelyon17
0.94 + 0.01
We remark that for no oversampling occur, the sampled labels' training accuracy should be at least as good as the ID validation accuracy. Since we have significant drops on these above datasets in Table 11, we conclude that the sampled decision boundaries of D3M most likely lie beyond the desired set Hpp
We further remark that single sample of labels from Bayesian model is not representative of the model's predictive capabilities as the latter relies on averaging over large samples of labels. We therefore make the preliminary assessment that models achieving the maximum disagreement rate are "oversampling", i.e. they do not fall in Hp. In addition, the second sampling from the categorical distribution given by logits further derails our empirical D3M from the idcalized D3M as with respect to the training data, this can be seen as noisy assignment of labels for training accuracy computation.
In contrast, Detectron [19] optimizes a joint loss term trading off between a training error minimization and target unsupervised dataset's error maximization (with respect to its pseudolabels). As this tradeoff is tunable, it's possible to enforce stronger adherence to hypotheses within H, at the heavy price of requiring the training set to be at arm's length.
43

---

Given this significant deviation from the idealized setup where candidate hypotheses obtained following an optimization oracle returning hypotheses only within Hp. to what extent can one trust the verdicts outputted by D3M? Intuitively. D3M is ID-oversampling "by the same amount" during calibration and deployment, akin to a miscalibrated balance scale. Oversampling on an ID deployment sample would result in disagreement rates resembling those in 4, thus incurring more false positíves. For a deteriorating OOD deployment sample, the effect of oversampling exacerbates disagreement rates but only to the extent that it exacerbates disagreement rates for bootstrapped ID samples during the calculation of Φ. In contrast, oversampling might result in a wider dispersion of d, meaning the
quantile might be pushed closer resulting in more false negatives.
Our analysis reveals that D3M consistently samples hypotheses beyond the ideal restricted hypothesis space Hy. with training accuracies falling 11-15% below validation performance across datasets. Despite this departure from the idealized framework, D3M operates as a "miscalibrated balance scale" that maintains relative consistency between calibration and deployment phases. The systematic nature of this oversampling means both ID and OOD samples are subject to the same bias, potentially preserving discriminative power while increasing variance in & and risking higher false negative rates.
C.5 Comparisons with a fully Bayesian network
We hereby provide additional ablation comparing D3M's performance against fully Bayesian network. Bayesian networks offer the most principled approach to approximating the idealized D3M (Algorithms and 2) when viewing optimization in H, as sampling from distribution over hypotheses modeled by said network. Given that D3M's sampling is only restricted to the last layer, it follows that D3M's sampled hypotheses are less diverse than their fully Bayesian counterparts.
Without (double) sampling once again from the categorical distribution generated by the sampled logits, the maximum disagreement rates achieved across 10000 runs hover around 5% for both ID and deteriorating OOD samples, making them nearly indistinguishable by D3M. Instead, sampling candidate disagreeing predictions from softmaxed and temperature-scae allows us to bring the ID disagreement rates in + to around 30%-50% where deteriorating OOD samples often score 5%-10% higher, making them distinguishable. How would fully Bayesian model fare? The following table compares the mean ID disagreement rates of fully Bayesian (FB) model and VBLL model (D3M), all using the same of best hyperparameters for each dataset.
logits rather than argmaxing
Table 12: Comparison of mean ID disagreement rates in & of fully Bayesian and VBLL models.
Dataset
Mean ID DisRate (FB) Mean ID DisRate (VBLL)
FB Accuracy 0.79 E 0.02
UCI
0.30 0.04
0.44 0.01
CIFAR-10/10.1
0.36 + 0.01
0.42 + 0.03
0.70 + 0.02
Camelyon17
We remark that using the mean model of FB network the predictive function does not trade off performance as shown in the last column. Further, we observe that even without the double sampling scheme used for VBLL, the mean ID disagreement rate of FB does not collapse into 0.05 usually the case with VBLL without any adjustments.
In Figure 9, we compare the fully Bayesian neural network's 's decision against various instances of VBLL neural networks. Indeed, we find that when no double sampling is used, decision boundaries tend to collapse to the mean function. This is evidenced in very low disagreement rates for VBLL compared to fully Bayesian models. With double sampling, however, we observe that predictions are more stochastic, leading to increasingly more likely disagreement with respect to the mean model. This illustrates how we may approximate the boundary/decision diversity of a fully Bayesian network with VBLL and additional double sampling.
Notably, on UCI, while VBLL trains each element approximately second, a FB base model in D3M takes approximately 17 minutes. The gap worsens for CIFAR-10/10.1 as it takes FB and hours for 1EFAA-100/00, while only 2 minutes for VBLL. This example further showcases the computational overhead incurred when sampling iteratively rather than in parallel. One either reparametrizes layers, making each forward pass's s output essentially a sample of logits, requiring
44

---

(a)
(c)
(d)
(c)
0.0
0.2
0.4
0.6
0.8
1.0
P(class 1)
Figure Toy experiment the Two Moons dataset generator [69]. 500 points noisily sampled from each class at noise level 0.3. We visualize 200 total samples here for clarity. (Figure 9a) A fully Bayesian neural network trained on this dataset. The decision boundaries correspond the level set of the probabilities of predicting either class at level 0.5. Forward passing the full grid through this net, we find different decision boundaries pictured in green. (Figure 9b) comparable VBI.I. model is trained. Without our double sampling scheme, most decision boundaries samples collapse onto the mean model's boundary, highlighted by the yellow region. Figures 9c, 9d, and 9e depict probabilities under double sampling with temperatures that sampling predictions from gradually softer probabilities give noisier estimates across the space. Although this does not provide concrete continuous level set at 0.5, the colors depict the distribution of predictions from which D3M samples, which "simulates" the diversity of decision boundaries.
2, 10 respectively. We observe
sequential forward passes to collect MC logits, or one must lift all weight tensors to include a sample dimension and ensure every downstream operation including residual connections, batch norm, and other architectural components - tooadcast-comptiblee
While this is realizable, it offers no benefits to our two-stage sampling strategy. We find that when temperatures are tuned to yield ID disagreement rates the 30% 50% range, deteriorating OOD disagreement rates are easily discernable as on those inputs, D3M tends to disagree 5%-10% better, thus easily achieving high TPR.
45


Reviews:
[{"review_id": "CmkRfD9RTJ", "reviewer": "Reviewer_vf94", "summary": "This paper tackles the challenge of knowing when a machine learning model’s performance is getting worse after deployment especially in real-world settings where data changes over time and labels are not available. The authors introduce D3M, a method that checks for model disagreement to detect harmful changes in data. Unlike typical drift detectors, D3M aims to avoid false alarms when the model is still performing well and to reliably catch cases where performance is truly dropping. The method is supported by theoretical guarantees and shows strong results on both standard benchmarks and a large internal medicine dataset, making it promising for use in critical applications.", "questions": "1. Line 1: operating repeated twice\n2. Can the authors provide the definitions of deteriorating and non-deteriorating shifts directly in the introduction? \n3. Can the authors explain as to why measuring disagreement via a variational approach that too only for the last layer is a useful method for measuring disagreement? In my opinion, wouldn't there be limited diversity or limited choice of hypothesis class exploration if only the last layer is trained in that fashion?\n4. How do the results change (qualitative explanation is sufficient) if there are multiple classification heads instead of the variational training?\n5. After T rounds, can you provide a graph of how the base model (ID) threshold changes? Is it more or less very similar to each other?\n6. In addition to metrics such as TPR, FPR, can the authors also evaluate a unifying metric such as Matthew's correlation coefficient in order to holistically characterize performance?\n7. Can the authors also qualitatively compare their work to conformal prediction and risk control? Conformal prediction can also be an easy to adopt strategy to explain model failures and even identify distribution shifts.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths - The writing is very clear and the paper is well-structured. The paper offers theoretical justification and broad empirical evaluation.\n\nWeaknesses - See questions", "was_revised": true, "final_justification": "The authors have sufficiently addressed my concerns and I feel their results are very comprehensive.", "rebuttal": "Thank you so much for the helpful comments! We are encouraged that you find our writing clear and the paper well-structured. We hope to address your questions and concerns in the following.\n\n# Questions\n\n> Line 1: operating repeated twice\n\nThank you for pointing this out! We’ve removed the repeated word and proof-read carefully.\n\n> Can the authors provide the definitions of deteriorating and non-deteriorating shifts directly in the introduction?\n\nWe will qualitatively introduce post-deployment deterioration prior to discussing the key challenges with its monitoring, thank you for the suggestion!\n\n\n> Can the authors explain as to why measuring disagreement via a variational approach that too only for the last layer is a useful method for measuring disagreement? In my opinion, wouldn't there be limited diversity or limited choice of hypothesis class exploration if only the last layer is trained in that fashion?\n\nIn theory, the diversity should be severely limited. The ultimate key to the method is in fact a two-stage sampling. Each Monte Carlo (MC) realization corresponds to a set of logits sampled from the variational posterior. \n\nFrom here, one may argmax the logits to get labels for each MC realization, for each sample in our batch. In our testing, we found that even with up to $10,000$ MC realizations, the sampled logits are not diverse enough to the extent that their argmaxes largely are the same. Disagreement rates hover at less than 5% on both ID and OOD batches, making them practically indistinguishable.  \n\nThus, we sample *once again* from softmaxed, temperature-weighted logits of each MC realization in order to get more variability, allowing us to achieve 30%-50% ID disagreement rates across various experiments. Of course, all operations here are parallelized, making the computational costs acceptable, even comfortable. \n\nWith this in mind, consider now a fully Bayesian implementation. Indeed, even without the two-stage sampling above, we achieve acceptable ID disagreement rates. However, this comes at the cost of heavy computational overhead. One either reparametrizes layers, making each forward pass’s output essentially a sample of logits, requiring sequential forward passes to collect MC logits, or one must lift all weight tensors to include a sample dimension and ensure every downstream operation — including residual connections, batch norm, and other architectural components — is broadcast-compatible. \n\nImportantly, while this is realizable, it offers no benefits to our two-stage sampling strategy. We find that when temperatures are tuned to yield ID disagreement rates in the 30% to 50% range, deteriorating OOD disagreement rates are easily discernable as on those inputs, D3M tends to disagree 5%-10% better, thus easily achieving high TPR. \n\nIn summary, the limited choice of hypothesis classes is adequately rectified by the double-sampling described above and in the manuscript, which we found to be empirically sufficient for the purposes of D3M’s 2nd and 3rd stages. \n\n> How do the results change (qualitative explanation is sufficient) if there are multiple classification heads instead of the variational training? \n\nThis is the architecture and design philosophy of Deep Ensemble adapted to monitoring model deterioration. We report in Table 2 that this monitor underperforms compared to other baseline methods as well as D3M. While leveraging head disagreements is a sensible insight, we believe D3M is more appropriate when unlabeled test data is scarce. \n\n> After T rounds, can you provide a graph of how the base model (ID) threshold changes? Is it more or less very similar to each other?\n\nWe report the ID thresholds for $\\alpha=0.1$ across $10$ independent runs. Columns are the rounds $t \\leq T=1000$ from which the $1-\\alpha$ quantile is computed for $\\Phi$. \n\n| Dataset   | $t=5$          | $t=50 $         | $t=100 $        | $t=500$          | $t=T=1000 $        |\n|-----------|-------------|-------------|-------------|--------------|--------------|\n| UCI Heart Disease       | 0.457±0.011 | 0.468±0.007 | 0.467±0.004 | 0.468±0.004  | 0.470±0.000  |\n| CIFAR-10/10.1   | 0.452±0.013 | 0.464±0.009 | 0.463±0.008 | 0.465±0.005  | 0.463±0.005  |\n| Camelyon17  | 0.306±0.011 | 0.312±0.004 | 0.312±0.004 | 0.310±0.000 | 0.310±0.000 |\n\nThe above shows that although some variability is exhibited for early $t \\in T$, the threshold stabilizes with more independent rounds, as evidenced by the decrease in standard deviation. Importantly, even as early as 5 samples, the ID threshold is already stabilized.\n\n> In addition to metrics such as TPR, FPR, can the authors also evaluate a unifying metric such as Matthew's correlation coefficient in order to holistically characterize performance?\n\nFor tasks with known OOD deterioration we desire monitors to identically output 1 (positive flag). Therefore, the most important metric by which we compare detectors is the TPR. One may choose to view detection as a meta-learning problem wherein the sole input to the detector is the task, and the metric by which to assess the detector is how reliably it identifies OOD deterioration. The converse reasoning is true for FPR. \n\nRegarding the base classifier’s performance however, we report their accuracy, F1, AUROC, Matthew’s Correlation, as well as the accuracy drop when testing on deteriorating OOD held-out data below. We additionally provide classification performance comparisons with a Linear head instead of a VBLL layer, all else being equal, trained side to side. \n\n### UCI (MLPModel)\n\n| Model | Accuracy      | F1 Score      | AUROC        |  MCC         | Acc. Drop OOD | \n|-------|---------------|---------------|--------------|--------------|--------------|\n| D3M   | 0.76 ± 0.02   | 0.75 ± 0.02   | 0.86 ± 0.01  |0.51±0.04 | -0.11±0.02 |\n| Lin   | 0.77 ± 0.01   | 0.76 ± 0.01   | 0.87 ± 0.01  |0.52±0.02 | _ |\n\n### CIFAR-10/10.1 (ConvModel)\n\n| Model | Accuracy      | F1 Score      | AUROC        | MCC | Acc. Drop OOD |\n|-------|---------------|---------------|--------------|--------------|--------------|\n| D3M   | 0.70 ± 0.02   | 0.70 ± 0.02   | 0.96 ± 0.00  |0.67±0.02 | -0.13±0.02 |\n| Lin   | 0.72 ± 0.01   | 0.72 ± 0.01   | 0.96 ± 0.00  | 0.69 ± 0.01| _ |\n\n### Camelyon17 (ResNetModel)\n\n| Model | Accuracy      | F1 Score      | AUROC        | MCC | Acc. Drop OOD | \n|-------|---------------|---------------|--------------|--------------|--------------|\n| D3M   | 0.94 ± 0.01   | 0.94 ± 0.01   | 0.98 ± 0.00  |0.88±0.01 | -0.05±0.00 |\n| Lin   | 0.94 ± 0.00   | 0.94 ± 0.00   | 0.98 ± 0.00  |0.88±0.01 | _ |\n\nThe drops in OOD accuracy is symptomatic of model deterioration. \n\n> Can the authors also qualitatively compare their work to conformal prediction and risk control? Conformal prediction can also be an easy to adopt strategy to explain model failures and even identify distribution shifts.\n\nThank you for this suggestion! Conformal prediction (CP) methods typically rely on a held-out calibration set with labeled data to construct valid predictive intervals, making them challenging to apply in the fully unsupervised post-deployment setting we consider. However, in a lot of applied scenarios, labels do eventually *become* available either via explicit labeling or one of several outcomes realized within a said time period. The deployment-label free advantage of D3M wears off over time in a practical setting when compared to CP in this regard.\n\nWe also highlight that D3M is uniquely robust to non-deteriorating shifts that may still be flagged by CP methods due to covariate changes. While CP methods can signal increased uncertainty, they do not directly distinguish between the harmfulness of shifts or offer the same guarantees on FPRs.  We believe that the scope and assumptions of D3M differ from those of standard CP and risk control approaches.\n\n# Closing remarks\nThe manuscript will be updated with all ablation results presented in this rebuttal. \n\nThank you once again for your helpful feedback. We hope that our rebuttal addresses your questions and concerns, and we kindly ask that you consider a fresher evaluation of our paper if you are satisfied with our responses. We are also more than happy to answer any further questions that arise."}, {"review_id": "Zxzc93sTvZ", "reviewer": "Reviewer_ktyz", "summary": "The paper presents a methodology to address post-deployment deterioration of model performance due to domain shift in the data. The authors propose a train, calibrate, and deploy strategy where, with the base model, there is a variational bayesian last layer (VBLL) to model the posterior predictive distribution over the target classes. The VBLL layer helps to identify deployment time deterioration of the model based on the disagreement rate over K drawn samples from the variational posteriors. The method is evaluated on publicly available datasets (i.e., vision, healthcare) and shows comparable performance with the baselines in detecting domain shifts in data.", "questions": "1) How does the VBLL layer impact classification accuracy?\n2) How can the VBLL layer be extended to a regression problem?", "limitations": "1) Evaluation can be more thorough. Please select a new temporal dataset for evaluating the method\n2) Need to proofread the manuscript", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1) The codebase is available, structured well, and documented for review.\n2) The introduction of the paper is well placed and motivates the approach.\n3) The algorithm is clearly explained with appropriate figures and descriptions\n4) Section 2.3: Algorithmic insights is well explained\n5) Experiments are conducted on publicly available datasets\n6) The related work is properly structured.\n\nWeakness:\n1) The abstract has two “operating”. Needs to proofread the manuscript.\n2) Evaluation on the UCI Heart Disease, CIFAR 10.1, and Camelyon 17 datasets can be more thorough with holdout participants or a dataset subset. \n3) Temporal shifts are only shown for the UCI IM dataset with an artificial shift in the test data (i.e., it’s not evident in the original data). This compromises the motivation of the problem. I strongly suggest evaluating on another temporal dataset with an inherent temporal shift.\n4) The impact on the target classification due to the VBLL layer is not clearly discussed\n5) Please provide an ablation study on the parameters: temperature and alpha.", "was_revised": true, "final_justification": "Yes, please incorporate k-fold cross-validation results on the dataset in the appendix of the final draft to showcase the effectiveness of the framework.\n\n“Functionally, this means that patients’ static covariates as well as their labs, medications, and medical interventions have drifted over time, especially into the COVID years.”  – Please clearly mention this in the paper. In this version, it reads like “Monitoring results on artificially shifted test data from the IM dataset”.\n\nThe ablation on VBLL and the Linear classification layer is good. Please include it in the appendix. Also, I find the ablation on the temperature parameter intuitive. \n\n\nI would suggest including a short discussion on how to extend VBLL to a regression model in the appendix (as a future direction).\n\nThe rebuttal addresses most of my concerns with this paper. I am increasing my score to 5 (accept). Thanks!", "rebuttal": "Thank you very much for the helpful comments! We feel encouraged that you found our codebase as well as the introduction and the related work well structured! We are glad to hear that you found our algorithm clearly explained! In the following, we hope to address your questions and concerns. \n\n# Weaknesses\n\n> The abstract has two “operating”\n\nThank you! We edited the manuscript accordingly. \n\n> Evaluation on the UCI Heart Disease, CIFAR 10.1, and Camelyon 17 datasets can be more thorough with holdout participants or a dataset subset. \n\nFor all experiments, we construct D3M’s Phi training and ID splits by randomly splitting the original validation set. In each independent iterate of the experiment, random participants in the ID dataset as well as in the OOD dataset are selected from which run statistics contribute to the reported figures in the manuscript, thus each independent run “holds out” both ID and OOD participants. Could you perhaps mean a k-fold validation by splitting ID and OOD data into bundles and cross-validating TPRs? We are happy to run further experimentation to showcase the effectiveness of the framework if you could please elaborate on your suggestions on evaluation! \n\n> Temporal shifts are only shown for the UCI IM dataset with an artificial shift in the test data (i.e., it’s not evident in the original data). This compromises the motivation of the problem. I strongly suggest evaluating on another temporal dataset with an inherent temporal shift.\n\nOur apologies for the inadequate clarity in the experiments’ descriptions. The IM dataset (name withheld) is a private electronic health records (EHR) dataset from a network of hospitals, comprising of static and dynamic features for patients’ hospital visits. This dataset is temporal by nature as data splits are arranged per half-years starting from 2017, where shifts in patient features become even more pronounced during the COVID period of 2020-2021. For the predictive task of 14-day mortality, our D3M base models trained on data before 2018 surprisingly do not underperform on later splits (Fig. 2a), therefore we make the assessment that this shift is non-deteriorating. \n\nFurthermore, we also claim that an important feature of this dataset is its inherent temporal covariate shift, the evidence of which is in the elevated FPR of divergence/distance-based detectors of Deep Kernel MMD (MMD-D) and Relative Mahalanobis Distance (Rel-MH). Functionally, this means that patients’ static covariates as well as their labs, medications, and medical interventions have drifted over time, especially into the COVID years. \n\n> The impact on the target classification due to the VBLL layer is not clearly discussed\n\n> How does the VBLL layer impact classification accuracy?\n\n\nThank you for pointing this out. We hereby provide an ablation comparing the D3M base model (D3M) against the same feature extractor architecture coupled with a linear head (Lin), trained side-by-side for UCI Heart Disease, CIFAR-10, and Camelyon17, all else being equal. We report accuracy, F1, AUROC, and Matthew's correlation (for CIFAR-10, binary metrics are computed independently for each class and then averaging the results equally across all classes, without weighting by class frequency):\n\n### UCI (MLPModel)\n\n| Model | Accuracy      | F1 Score      | AUROC        |  MCC         |\n|-------|---------------|---------------|--------------|--------------|\n| D3M   | 0.76 ± 0.02   | 0.75 ± 0.02   | 0.86 ± 0.01  |0.51±0.04 |\n| Lin   | 0.77 ± 0.01   | 0.76 ± 0.01   | 0.87 ± 0.01  |0.52±0.02 |\n\n### CIFAR-10/10.1 (ConvModel)\n\n| Model | Accuracy      | F1 Score      | AUROC        | MCC |\n|-------|---------------|---------------|--------------|--------------|\n| D3M   | 0.70 ± 0.02   | 0.70 ± 0.02   | 0.96 ± 0.00  |0.67±0.02 |\n| Lin   | 0.72 ± 0.01   | 0.72 ± 0.01   | 0.96 ± 0.00  | 0.69 ± 0.01|\n\n### Camelyon17 (ResNetModel)\n\n| Model | Accuracy      | F1 Score      | AUROC        | MCC | \n|-------|---------------|---------------|--------------|--------------|\n| D3M   | 0.94 ± 0.01   | 0.94 ± 0.01   | 0.98 ± 0.00  |0.88±0.01 | \n| Lin   | 0.94 ± 0.00   | 0.94 ± 0.00   | 0.98 ± 0.00  |0.88±0.01 |\n\nWe notice that side by side, $FE_\\theta + VBLL_\\theta$ is slightly worse than its linear head counterpart. We argue, however, that the sampling benefits provided by the VBLL layer enables the functioning of D3M. When compared to multiple heads (Deep Ensemble baseline) for deteriorating OOD detection, our method achieves noticeably higher few-shot TPR. \n\n> Please provide an ablation study on the parameters: temperature and alpha.\n\nCertainly! The following table is the result of an ablation on the temperature parameter at query size $100$. \n\n### UCI \n| temp | fpr_id | tpr  | dis_rate_id | dis_rate_ood |\n|------|--------|------|------------------|-------------------|\n| 1    | 0.01   | 0.85 | 0.4359           | 0.4919            |\n| 2    | 0.07   | 0.49 | 0.5478           | 0.5785            |\n| 3    | 0.07   | 0.32 | 0.5864           | 0.6089            |\n| 5    | 0.09   | 0.12 | 0.6280           | 0.6381            |\n| 10   | 0.08   | 0.19 | 0.6520           | 0.6615            |\n| 20   | 0.04   | 0.11 | 0.6679           | 0.6721            |\n\n### CIFAR-10/10.1\n| temp | fpr_id | tpr  | dis_rate_id | dis_rate_ood |\n|------|--------|------|------------------|-------------------|\n| 1    | 0.14   | 0.98 | 0.4309           | 0.5189            |\n| 2    | 0.06   | 1.00 | 0.6405           | 0.7267            |\n| 3    | 0.06   | 0.96 | 0.7683           | 0.8289            |\n| 5    | 0.05   | 0.78 | 0.8837           | 0.9136            |\n| 10   | 0.05   | 0.34 | 0.9505           | 0.9619            |\n| 20   | 0.07   | 0.09 | 0.9744           | 0.9762            |\n\n### Camelyon17\n| temp | fpr_id | tpr  | dis_rate_id | dis_rate_ood |\n|------|--------|------|------------------|-------------------|\n| 1    | 0.09   | 0.99 | 0.1696           | 0.2561            |\n| 2    | 0.12   | 1.00 | 0.2891           | 0.3904            |\n| 3    | 0.09   | 1.00 | 0.3801           | 0.4712            |\n| 5    | 0.07   | 1.00 | 0.4774           | 0.5471            |\n| 10   | 0.08   | 0.61 | 0.5803           | 0.6118            |\n| 20   | 0.05   | 0.32 | 0.6295           | 0.6483            |\n\n\nBy increasing the temperature, samples are more diverse and disagreements are more pronounced, evidenced by the elevated mean disagreement rates. This not only hurts OOD-TPR but also increases FPRs, as the “base ID disagreement rate” distribution is further skewed toward 1, making it harder to differentiate between ID disagreement rates and deteriorating OOD disagreement rates. The correct temperature balancing sampling diversity with reasonable ID disagreement rates (and thus ID disagreement thresholds) is thus a hyperparameter that we tune. In general, values between 1 and 5 are preferred. \n\nWe respectfully note that $\\alpha$ is a user-defined significance level, akin to a tolerance in classical hypothesis testing. Thus, varying $\\alpha$ does not meaningfully reflect the behavior or robustness of the algorithm itself, but rather tunes the system’s operating point: a higher $\\alpha$ increases sensitivity but also false positives, while a lower $\\alpha$ increases specificity. As such, varying $\\alpha$ does not reflect the robustness of the method, but rather the practitioner’s risk tolerance — similar to choosing a 95% vs. 99% confidence level depending on deployment needs.\nWe provide results at a reasonable default $\\alpha=0.1$ and practitioners may adjust it depending on the criticality of their deployment context.\n\n# Questions\n> How does the VBLL layer impact classification accuracy?\n\nPlease see above. \n\n> How can the VBLL layer be extended to a regression problem?\n\nWe believe several works in the literature replace the categorical likelihood with a Gaussian likelihood, i.e. the VBLL layer would output a predictive distribution over real-valued targets—typically by learning a distribution over the regression weights and using Bayesian linear outputs to produce a mean and variance for the predicted value. The ELBO objective can then be adapted to use a Gaussian likelihood while retaining variational inference for the weights, allowing the model to capture epistemic uncertainty in regression. \n\nIn terms of deteriorating OOD detection for regression, we believe D3M has nice analogies with already-existing confidence region methods. We believe “the extent of disagreement” in classification is thus analogous to a $(1-\\alpha)%$ confidence region following a posterior distribution in Bayesian regression, perhaps reducing D3M to a sampling-based approach to OOD detection for regression. \n\n# Closing Comments:\n\nAblations reported in this rebuttal as well as other rebuttals will be published in a revised version of the manuscript. \n\nThank you once again for your valuable feedback. We hope that our rebuttal addresses your questions and concerns, and we kindly ask that you consider a fresher evaluation of our paper if you are satisfied with our responses. We are also more than happy to answer any further questions that arise."}, {"review_id": "IZTm63kWke", "reviewer": "Reviewer_TFQc", "summary": "This paper proposes methodology, D3M, for monitoring distribution shifts that may lead to performance degradation of deployed classifiers. \n\nThe central claim is that D3M addresses two desiderata that are not satisfied by any single prior method: (1) no need to access training data at deployment time and (2) only flag *deterioriating* shifts. \n\nThe method works by leveraging uncertainty estimates from a Bayesian neural network and monitoring for high variance/disagreement in the predictions. \n\nThe D3M method is competitive with Detectron (which requires training data at deployment time), though trails by substantial margins on some datasets. They also claim in writing that D3M has a number of efficiency benefits over alternative approaches.", "questions": "- I’m confused by the columns in Table 1. How can D3M be both deteriorating and non-deteriorating?\n- How does the use of VBLL compare with other Bayesian approaches?\n- Other typos and clarifications:\n    - There is a duplicated “operating operating” in the first sentence of the abstract.\n    - “Underlines in our results indicate that D3M is close to the best baseline.” What is “close” here? It seems somewhat arbitrary given the differences in performance and nearly 10 points in some cases.\n    - Please, clarify what exactly an N-shot scenario is. Also explain that the {10, 20, 50} in Table 2 correspond to shots in the caption.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The problem addressed in this paper (monitoring for distribution shifts) is very important in safety critical settings like medicine. The paper evaluates on interesting and realistic benchmarks from the medical domain.\n- The proposed methodology has at least one advantage over prior work like Detectron: it does not requiring training data at deployment time.\n- The method is competitive with prior work like Detectron.\n\n**Weaknesses**\n\n- In Section 2.3, the authors make a number of claims about D3M without concrete evidence or experiments:\n    - “D3M does not trade-off prediction accuracy.” To support this claim, the authors should provide some experiments demonstrating that the prediction accuracy with D3M is indeed equivalent to or better than the prediction of accuracy of other classifiers. However, no such experiments or results are included in the paper.\n    - “Potential for edge deployment.” The authors claim that D3M is particularly well-suited for edge deployments but do not provide any analysis of edge hardware and how their method will map onto it. Without a concrete analysis, this claim is speculative.\n    - “While VBLL allows more efficient sampling compared to to a fully Bayesian network, the price to pay is the lack of diversity of the sampled predictions.” To support this claim, I would expect to see a comparison of the diversity of predictions when using VBLL and a fully Bayesian network. Without it, I cannot judge the importance of the proposed remedies (*e.g.* temperate scaling).\n- In Section 3.2, the authors claim that “D3M is significantly more scalable and practical for real-world deployment [than Detectron]”. While it is a clear advantage that D3M doesn’t require training data, the authors do not provide any concrete evidence of the scalability of D3M and Detectron. I would recommend that the authors include some quantitative measure of cost (*e.g.* FLOPs) that demonstrates that D3M is indeed more scalable than Detectron.\n- The presentation of the results on the IM dataset [Temporal Shift] could be improved.\n    - I’m quite confused by the Y-axis in Figure 2b. The label says TPR@5%, but this is in a setting with only a non-deteriorating shifts, so the true positive rate is undefined?\n    - The legend in Figure 2 and Figure 3 could be improved. I’m not sure what some of the abbreviations correspond to. For example, is DC the abbreviation for detectron? **", "was_revised": true, "final_justification": "Prior to the rebuttal, my main concern was that the paper made several claims regarding the efficiency and scalability of the method that were not substantiated quantitatively. I think this is a major concern given that the proposed method's only advantage over prior work (e.g. detectron) is scalability, not quality. The author's response provided some quantification in terms of training and calibration FLOPs. So I raised my score, assuming that these will be integrated into the final results.", "rebuttal": "Thank you very much for the helpful comments! We are encouraged that you find our problem “very important in safety critical settings”, and that D3M is “competitive with prior work”. We hope to address your concerns and questions in the following. \n\n# Weaknesses\n\n> “D3M does not trade-off prediction accuracy.” [...]\n\nCertainly! We hereby provide an ablation comparing the D3M base model (D3M) against the same feature extractor architecture coupled with a linear head (Lin), trained side-by-side for UCI Heart Disease, CIFAR-10, and Camelyon17, all else being equal. We report accuracy, F1, AUROC, and Matthew's correlation (for CIFAR-10, binary metrics are computed independently for each class and then averaging the results equally across all classes, without weighting by class frequency):\n\n### UCI (MLPModel)\n\n| Model | Accuracy      | F1 Score      | AUROC        |  MCC         | Acc. Drop OOD | \n|-------|---------------|---------------|--------------|--------------|--------------|\n| D3M   | 0.76 ± 0.02   | 0.75 ± 0.02   | 0.86 ± 0.01  |0.51±0.04 | -0.11±0.02 |\n| Lin   | 0.77 ± 0.01   | 0.76 ± 0.01   | 0.87 ± 0.01  |0.52±0.02 | _ |\n\n### CIFAR-10/10.1 (ConvModel)\n\n| Model | Accuracy      | F1 Score      | AUROC        | MCC | Acc. Drop OOD |\n|-------|---------------|---------------|--------------|--------------|--------------|\n| D3M   | 0.70 ± 0.02   | 0.70 ± 0.02   | 0.96 ± 0.00  |0.67±0.02 | -0.13±0.02 |\n| Lin   | 0.72 ± 0.01   | 0.72 ± 0.01   | 0.96 ± 0.00  | 0.69 ± 0.01| _ |\n\n### Camelyon17 (ResNetModel)\n\n| Model | Accuracy      | F1 Score      | AUROC        | MCC | Acc. Drop OOD | \n|-------|---------------|---------------|--------------|--------------|--------------|\n| D3M   | 0.94 ± 0.01   | 0.94 ± 0.01   | 0.98 ± 0.00  |0.88±0.01 | -0.05±0.00 |\n| Lin   | 0.94 ± 0.00   | 0.94 ± 0.00   | 0.98 ± 0.00  |0.88±0.01 | _ |\n\nWe find that indeed, D3M trades off little to no prediction performance. We hypothesize further that this tradeoff is conditional on the generalizability of the representations used before the last layer, as MLPModels and ConvModels are trained from scratch, while the ResNet backbone in the Camelyon17 experiments are finetuned from the ImageNet models, although a lot more experiments are needed to validate this hypothesis.\n\n> “Potential for edge deployment.” [...]\n\nWe appreciate your observation regarding our claim about edge deployment. We agree that our discussion in this section was speculative and should have been more clearly framed as such. Our intent was to highlight that D3M’s source-free deployment mechanism offers practical advantages for local inference in e.g. hospital environments, where dedicated compute clusters may be unavailable or undesirable due to privacy and latency concerns. We acknowledge that further empirical study is needed to validate these claims, and we will revise the manuscript to reflect this more cautiously.\n\n> “While VBLL allows more efficient sampling compared to a fully Bayesian network, the price to pay is the lack of diversity of the sampled predictions.” To support this claim, I would expect to see a comparison of the diversity of predictions when using VBLL and a fully Bayesian network. [...]\n\n> How does the use of VBLL compare with other Bayesian approaches? \n\n\nWe found that without sampling once more from the logits in order to Monte-Carlo (MC) sample labels with which the base model can disagree with, the maximum disagreement rates achieved across $10000$ runs hover around 5% for both ID and deteriorating OOD samples, making them nearly indistinguishable by D3M. We found that sampling candidate disagreeing predictions from softmaxed and temperature-scaled logits rather than argmaxing allows us to bring the ID disagreement rates in $\\Phi$ to around 30%-50%, where deteriorating OOD samples often score 5%-10% higher, making them distinguishable. \n\nUnder a rough comparison, for query size 100 and 5000 MC samples, fully Bayesian networks with logit argmaxing naturally achieve ~30% ID disagreement rate on the UCI task and ~37% ID disagreement rate on CIFAR-10. Importantly, while fully Bayesian models can reach comparable disagreement rates even without additional sampling tricks, they do so at significantly greater computational cost: D3M + VBLL trains $\\Phi$ in 1 second while fully Bayesian D3M takes 17 minutes for UCI, for instance. In our setting, the goal is to highlight disagreement gaps between ID and deteriorating OOD data—not to maximize disagreement itself. Temperature scaling and softmax sampling already achieve this effectively, making added Bayesian complexity unnecessary.\n\nFor a more detailed discussion regarding this difference, please see our response to Reviewer **Vf94**. A complete report with comparisons of fully Bayesian networks to D3M + VBLL across all datasets across independent seeds *is on the way* and will be promptly added to a future revised version of the manuscript. \n\n> In Section 3.2, the authors claim that “D3M is significantly more scalable and practical for real-world deployment [than Detectron]”.  [...]\n\nCertainly! The following is a table listing the FLOPs at different computational stages of the algorithms. We assume a query size of 100, and the same hyperparameters as reported in the Appendix section. \n\n| Algorithm | Dataset | Base Model Training      | Calibration   | **Total** | \n |-------|-------|---------------|---------------|---------------|\n| D3M   | UCI |  0.23 GFLOPs | 9.00 GFLOPs | **9.23 GFLOPs**| \n| Detectron | UCI | 0.15 GFLOPs | 18.69 GFLOPs | **18.84 GFLOPs** | \n| D3M   | CIFAR-10/10.1 |  1.56 PFLOPs | 0.13 PFLOPs | **1.69 PFLOPs** | \n| Detectron | CIFAR-10/10.1 | 1.56 PFLOPs | >1.95 PFLOPs | **>3.51 PFLOPs** | \n| D3M   | Camelyon17 |  12.02 PFLOPs | 0.74 PFLOPs | **12.76 PFLOPs** | \n| Detectron | Camelyon17 | 12.02 PFLOPs | >11.04 PFLOPs | **>23.06 PFLOPs** | \n\nEventually, as the feature extractors grow in size, we notice that for D3M the cost is dominated by the training phase, whereas Detectron expends at least nearly the same compute during the calibration phase as its training phase. This is due to Detectron requiring finetuning on held-out subsets while D3M simply freezing the model in evaluation mode and sampling from the VBLL layers. Furthermore, gradient computations of Detectron need to be replicated on the deployment sample.\n\n> I’m quite confused by the Y-axis in Figure 2b. The label says TPR@5%, but this is in a setting with only a non-deteriorating shifts, so the true positive rate is undefined?\n\nOur apologies for this mistake! In figure 2b, the label should be saying FPR@5%. We have adjusted the manuscript accordingly. Indeed, since this is a non-deteriorating shift (following Figure 2a), we want models to resist flagging. Covariate shifts are present as evidenced by their flagging from baseline naive covariate shift detectors. \n\n> The legend in Figure 2 and Figure 3 could be improved. I’m not sure what some of the abbreviations correspond to. For example, is DC the abbreviation for detectron? **\n\nWe have revised the manuscript to make the legend correspondences with baseline algorithms clearer. DC here refers to a Domain Classifier (per Rabanser et. al. 2019), i.e. a separate model trained on the same inputs to discern between ID and OOD data (this, of course, requires labeled OOD data). A detailed description of the Domain Classifier will be included in the Appendix.\n\n# Questions \n\n> I’m confused by the columns in Table 1. How can D3M be both deteriorating and non-deteriorating?\n\nIndeed, these two columns refer to whether the algorithms 1. flags deteriorating shifts, and 2. resist flagging non-deteriorating shifts. Importantly, D3M achieves the latter, and provably so under certain theoretical conditions, detailed in Appendix A.\n\n\n> There is a duplicated “operating operating” in the first sentence of the abstract.\n\nThank you! This mistake is fixed. \n\n> “Underlines in our results indicate that D3M is close to the best baseline.” What is “close” here? It seems somewhat arbitrary given the differences in performance and nearly 10 points in some cases.\n\n We wanted to use the underlines to highlight situations where most baselines severely underperform while D3M is nearing the performance of the top baseline, though as you highlighted, this is inconsistent with CIFAR-10, query size 20, where D3M gets outperformed by both Detectron and Deep Ensemble at roughly the same performance. For this reason, we have removed the underlines. \n\n> Please, clarify what exactly an N-shot scenario is. Also explain that the {10, 20, 50} in Table 2 correspond to shots in the caption.\n\nCertainly! We appreciate you pointing out the lack of clarity. The N-shot scenarios correspond to the size of the unknown deployment dataset. Remote hospitals with scarcely an influx of patients would be more interested in the 10,20,50-deployment size scenarios, while major hospitals would already be enjoying high TPR from D3M. \n\nDue to this confusion, as well as “few-shot” colloquially referring to single digit shots, we have henceforth decided to adopt the terminology of “query size” to refer to the quantity above, and will edit the manuscript accordingly as to further avoid misunderstandings.\n\n# Closing Comments:\n\nAblations reported in this rebuttal as well as other rebuttals will be published in a revised version of the manuscript. \n\nThank you once again for your helpful feedback. We hope that our rebuttal addresses your questions and concerns, and we kindly ask that you consider a fresher evaluation of our paper if you are satisfied with our responses. We are also more than happy to answer any further questions that arise."}, {"review_id": "omuNcTY8m3", "reviewer": "Reviewer_88gc", "summary": "This paper proposes a method to determine whether a model has suffered from \"post-deployment deterioration\" (PDD). That is, the joint distribution of the covariates $x$ and labels $y$ have shifted such that the model's performance is worse. The paper distinguishes this from out of distribution (OOD) detection in that OOD detection does not ask whether or not the performance of the model has become worse. The method takes inspiration from the model disagreement literature, which checks if multiple models make different predictions on a test point. Here, the \"multiple models\" come from adding a \"Variational Bayesian Last Layers\" to an existing model's output layer (i.e., the model now outputs a distribution over logits). This distribution over logits is sampled from, passed through a softmax, and then the resulting probability distribution is sampled from with an increased temperature to get a prediction. The number of these sampled predictions that disagree with the base-model's argmax prediction is the \"model disagreement\". This process is repeated for a number of training datapoints to get a sense for typical model disagreement for in-distribution data. Model disagreement is computed the same way for input test data, and if the disagreement is higher than typical for in-distribution data, the test data are flagged. In experiments, the paper shows the proposed method to have often higher true positive rate (TPR) than similar methods with the same false positive rate (FPR); however, the TPR is fairly noisy across different random seeds. In the appendix, an \"idealized\" version of the algorithm is proposed and its theoretical properties (e.g., FPR and TPR related to its input parameters) are studied.", "questions": "1. How does the idealized algorithm that the theory is about relate to the actual algorithm? How much should users trust the theoretical results to apply to the real algorithm?\n\n2. When will the algorithm fail? Are there any indicators or situations that users should look out for?\n\n3. How can users reduce the variability of the proposed method that's seen in, e.g., Table 2? What could lower this variance and why?\n\n4. The paper seems to propose waiting to make PDD predictions until a large batch of test points has been accumulated. Why is this realistic? How would this work in time-sensitive situations?\n\nThank you!", "limitations": "I don't think the main text included much discussion of the proposed method's limitations besides pointing out the fact that its performance can be noisy -- the \"Conclusions and Limitations\" section actually mostly summarizes the strengths of the proposed method. As noted above, I would suggest really digging into the intuition behind the proposed method so that its failure case(s) can be better understood by readers.\n\nFurther, I think it's important to stress that the PDD task is fundamentally impossible without labels. If the distribution of $(x,y)$ changes such that $y\\mid x$ is totally different but the distribution of $x$ is the same, there's no way to detect this from labels. So I think it's really important to make the assumptions this paper is making more clear and emphasize how reality might come into conflict with those assumptions.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Weaknesses\n\nI've broken down comments here into various sections below.\n\n## What is the proposed method doing?\n\nI overall think it's not clear what this algorithm is really doing and why it should work. And I think it's really important when proposing new methods to give some kind of scientific understanding of the methods (unless the empirical results are so phenomenal that the research community just needs to know about this new method without really knowing anything about it, but I don't think that's the case here). I don't think the paper is currently meeting this standard, which is the main influence on my score. A few specific comments on this below:\n\n1. In the main text of the paper, I didn't see any discussion of why readers should think the proposed method would work. It has the flavor of existing OOD algorithms, but it's supposed to (1) detect OOD points, and (2) detect if those points are going to give bad performance from the model. I didn't really see why we would expect (2) to hold. Overall, I think there needs to be a lot more discussion of the intuition behind the algorithm in the paper.\n\n2. The theoretical discussion in the supplement has some intuition of why the proposed algorithm will work. This development proves conditions under which an \"idealized\" version of the algorithm will work well. I think this is really great, but could use some expansion. First, the idealized version of the algorithm is pretty idealized, and it's not clear how connected it is to the real version of the algorithm. The idealized version assumes access to all other models that have training error within $\\epsilon$ of the base model. I think the paper is arguing that this is approximated by some of the Bayesian sampling and increased temperature. But why does this approximate this model set well? The paper briefly implies (line 1010) that the Bayesian sampling and temperature will *over-approximate* the size of this model class, but it's not clear to me that this is true. I think careful exploration of the connection between the idealized and practical algorithm is needed to finish the story of this paper. Second, I really think some of the theoretical development needs to be moved to the main text, as it's what ties the entire paper together.\n\n3. There are no experients that get at the heart of what's going on with the algorithm. I think synthetic experiments are an excellent opportunity to really probe what's going on behind the scenes of an algorithm -- when should we expect the algorithm to fail or succeed? How well does the intuition behind the algorithm hold up empirically? Carefully designed synthetic experiments give the opportunity to answer all of these questions, so I would really recommend thinking about how that could be done for the current paper.\n\n## Theoretical results\n\nI thought that two potentially impractical assumptions were made in the theory.\n\n**Assumptions**\n- The theoretical results hinge on assuming the model class is correctly specified, which is a major assumption that is rarely true in practice.\n- Assumes that $m$ datapoints are available at once to make a decision of prediction on. Again, it's not clear to me how practical this setting is.\n\n\n**Clarity**\nThere were a few places towards the beginning of the theory where I found things either underdefined or just a little unclear. These are mostly towards the beginning of the theoretical development, as I started skimming towards the end as these built up.\n\n- In Lemma A.1: $h$ was not defined. Additionally, the proof of the lemma states that $err(f, Q_g) > err(f, P_g)$, where $Q$ is the deployment distribution and $P$ is the training distribution. Why must this be true given the assumptions of the theorem? What if $Q$ is a point-mass at a point that $f$ happens to get correct? And below line 970, a sup is taken over $A$ (I think sets $A$?) without saying what $A$ varies over. Finally, the proof starts by stating it is going to show that D-PDD implies PDD, but it never shows the reverse direction, despite the statement of the Lemma saying that PDD also implies D-PPD.\n- Definition 4 states that the algorithm detects when $\\epsilon_q - \\epsilon_p > 0$, then Definition 5 states that the algorithm detects when $\\xi \\geq 0$, and then the line after Definition 5 states that $\\xi \\geq \\epsilon_q - \\epsilon_p$. So these quantities aren't the same, so how can it be that the algorithm is simultaneously thresholding both?\n- Definition 5 states that $\\xi \\geq 0$. But why? Does this assume $f$ is the minimizer of $err(f; P_g)$? I don't think this was stated.\n- Eq (6) uses the notation $Q_{1-f}$, but I don't think the $1-f$ notation was ever defined.\n- Thm A.3 proves a bound on the FPR of the algorithm. The discussion around it implies that it bounds the FPR by $\\alpha$, the \"significance\" level of the algorithm. But I don't think the theorem proves this. In particular, it states that the FPR can be bounded by $\\alpha + (1-\\alpha)e^{-n\\epsilon_0^2 + d}$, where $d$ is the VC dimension of the hypothesis class. If $d$ is huge, then I'm pretty sure this bound can be arbitrarily larger than 1, whereas $\\alpha < 1$.\n- Towards better connecting the theory to the main paper: Theorem A.3 gives a required batch size $m$ to achieve a bounded FPR. It would be really interesting to evaluate $m$ for typical ML models and see if the result is practical. Is this telling us we need to batch up 94303 samples before making an effective decision about domain shift? Or is it saying we only need 20?\n\n\n## Experiments\n1.  The experiments are run in \"{10,20,50}-shot scenarios\". I wasn't totally clear on what this means. Does this mean that the model gets to see a batch of, e.g., 10 test datapoints before guessing whether there's model drift? It seems like this runs counter to the run in real time idea of the paper. In practice, I'd imagine that we want to make predictions one at a time. And then we have to make a \"domain shift yes/no\" guess when each datapoint is observed. How does the current experimental setup fit into this picture? Or are there really scenarios where we can wait to make a prediction until we've seen 50 test points?\n\n2. In the entire experiment depicted in Figs 2 and 3, it seems like the existing BSSD method is outperforming the proposed method. This doesn't seem to be discussed in the paper -- so what is this experiment showing that's good or promising about the proposed method?\n\n3. Figure 2b seems to be showing really bad performance of the proposed method, as it's true positive rate is nearly zero across the board. The text seems to indicate that this is a good thing because there's actually not much domain shift happening here. But in that case shouldn't there be no true positives at all? How are other methods getting high true positive rates if there is nothing to correctly flag?\n\n4. The x-axis in Fig 3 isn't described. I'm guessing this is an amount of domain shift, but it's not clear to me what the numbers mean.\n\n\n\n## Overall Comments\n1. Some references to specific Appendicies seem to be pointing to the wrong place (e.g., saying that baselines are described in B.4, but they're actually described in B.1). Please check the appendix references to make sure they're all pointing to the right place.\n2. \"D3m does not trade off prediction accuracy\" -- it does require the addition of a Bayesian head to the model, which I would assume could affect the accuracy.\n3. $\\tau$ was never defined in the main text\n4. The introduction makes it sound like there are few restrictions on the model class, but the paper seems to be entirely about classification models.\n5. I think the claim that \"we answer all desiderata\" it misleading, as one of the disiderata is that the method has a low FPR. I strongly suspect it is impossible to guarantee this in practice, but this claim makes it sound like the proposed method completely solves this issue.\n\n# Strengths\n\nI think with a properly motivated algorithm (along the lines of the theory in the appendix), this paper has the potential to be very strong. Proving conditions under which not only OOD detection is possible but a practically meaningful form of OOD (when the model performance is actually worse) is very impactful for applications, and I think this paper has the possibility to not only do that, but also give an explicit algorithm for doing so. I think it's especially strong that the algorithm has a mechanism to explicitly control its false positive rate, which I think will be important in practical settings; if I had to guess, I would say that hospital staff are going to be pretty susceptible to alarm fatigue from a black box ML system, so setting the FPR to be guaranteed low is important.", "was_revised": true, "final_justification": "I've increased my score from a 2 to a 4 (weak accept) after the discussion period. I think the authors have addressed essentially all of the issues I raised, and I think they've also addressed other reviewer's comments well.\n\nI've only increased to a weak accept rather than an accept because I think the proposed experiment showing oversampling isn't air-tight. In particular, I think this only shows that the sampling method can generate points that are outside the theoretical $\\epsilon$ ball that we want. But I think what we really want is to show that the method generates points completely covering the $\\epsilon$ ball. To be fair, though, the proposed experiment gets at *suggesting* this coverage, and it's not like I have any great ideas on how one would show this more precisely.\n\nBut overall, I vote for accepting the paper.\n\nBest,\n88gc", "rebuttal": "Thank you very much for your very thorough dissection of our paper and insightful comments! We appreciate that you find our algorithm is “properly motivated”, with “potential to be very strong”. We hope to address your concerns and questions in the following. \n\n# Weaknesses\n\n> “I didn't see any discussion of why readers should think the proposed method would work [...]”. \n\n> The theoretical discussion in the supplement has some intuition of why the proposed algorithm will work. [...]\n\nThank you for highlighting this critical point. The underlying intuition is that given a base classifier model, the disagreement rate is a quantity that differs in and out of distribution only when the base model will underperform on a OOD batch. The reason we track this discrepancy is due to Definition 2, Line 949, 3rd inequality $err(h;Q_f) > err(h; P_f)$ where the algorithm’s calibration step is generating empirical values for the RHS, while the Deployment step compares an empirical estimate of the LHS to an empirical distribution of $err(h; P_f)$. If we believe Definition 2 is sound, then Idealized D3M works because it’s designed to empirically track D-PDD, while D3M is a crude approximation. \n\nWe agree that sampling over-approximates the set of interest. It’s likely that our sampling produces hypotheses that would’ve achieved higher error than the base classifier on the training set. Empirically, we find that although this may be the case, the strength of disagreement is more significant on deteriorating OOD data. Guarantees on significant disagreement gaps between ID and OOD despite sampling beyond the set of interest are currently beyond what the theory provides. \n\n> some of the theoretical development needs to be moved to the main text, as it's what ties the entire paper together [...]\n\nWe agree with your assessment of the importance of the theoretical binds. For this reason, we are editing in a theoretical insights subsection along with an informal definition of D-PDD as well as an informal lemma linking D3M to the tracking of D-PDD, while referring readers to the Appendix for a more formal treatment. \n\n> There are no experients that get at the heart of what's going on with the algorithm. I think synthetic experiments are an excellent opportunity to really probe what's going on behind the scenes of an algorithm  [...]\n\nThank you for raising this point! We could demonstrate a proof of concept for the elevated OOD disagreement rates by the following experiment: we learn a posterior over decision boundaries on the synthetic Two Moons dataset in 2D. For ID and OOD samples, we illustrate how likely it is to sample a decision boundary that disagrees on those samples with respect to the mean decision boundary, building intuition for the discrepancy in disagreement rates. \n\n\n> The theoretical results hinge on assuming the model class is correctly specified, which is a major assumption that is rarely true in practice.\n\nIndeed, this is an important limitation with regards to results. However, we encourage readers to view the theory as a perspective on how one might go about modeling the deteriorating OOD problem, as well as the insights insights which we derive and track directly in D3M (tracking $err(h;Q_f) > err(h;P_f)$, for instance).\n\n> Assumes that  datapoints are available at once to make a decision of prediction on. Again, it's not clear to me how practical this setting is.\n\n> The paper seems to propose waiting to make PDD predictions until a large batch of test points has been accumulated. Why is this realistic? How would this work in time-sensitive situations?\n\n\nAn example of a practical setting would be D3M undergoing a silent trial at a hospital network, where patient data come in sequentially to fill up a batch before being processed with D3M. This is in contrast to paradigms of decision making where a decision is expected after an observation is recorded.\n\n> In Lemma A.1:  was not defined. [...]\n\nOur revised manuscript will provide the definition of $h$ following Definition 2. \n\nRegarding $Q$ being a point mass, this violates the TV condition unless $P$ already concentrates nearly all of its mass on it. Our lemma explicitly assumes a bounded TV shift and a nontrivial disagreement gap, ensuring that the difference in disagreement cannot be solely attributed to small distributional changes.\n\nWe edited the supremum to be over $A \\in \\mathcal{F}$ of the underlying $\\sigma$-algebra. \n\nWe respectfully note that the reverse direction (D-PDD → PDD) is indeed concluded: the final inequality shows that if the disagreement gap satisfies $err(f,Q_h​)−err(f,P_h​)\\geq 2(\\kappa+\\epsilon)$, then it necessarily implies $err(f, Q_g​)> err(f, P_g​)$, which is precisely the PDD condition under $g=g’$.\n\n> Definition 4, 5, Eq (6) \n\nOur manuscript has been revised with the suggested edits and clarifications. \n\n> Thm A.3 proves a bound on the FPR of the algorithm. \n\n> Towards better connecting the theory to the main paper\n\nWe appreciate the engagement with Theorem A.3. Indeed, the bound derived there is likely loose and primarily serves as a proof of concept—a theoretical guarantee that, in principle, our method can control the FPR under distribution shift. While the bound scales with the VC dimension of the hypothesis class and may be vacuous for large models, its main role is to motivate and justify the design of our algorithm. Empirically, we find that our method performs well with small batch sizes, suggesting that the theoretical sample complexity is conservative but directionally aligned with practice.\n\n## Experiments\n\n> The experiments are run in \"{10,20,50}-shot scenarios\" [...] \n\nAs per our discussion with Reviewer **Ktyz**, these are the sizes of the subset of data receiving a verdict from D3M. Importantly, methods in the literature typically give a verdict per point, whereas our setup’s goal is to give a verdict per batch or per data subset. We envision D3M as a monitor to a predictive model in deployment rather than a decision-making agent. Inputs on which the model predicts are bundled (one may imagine a moving-window of the last $m$ samples). A D3M flag would signify that a corrective action is in order, i.e. a re-calibration, a fine-tuning, a domain-adaptation modification, or even a potential retraining. \n\n> BBSD [...] \n\nBBSD is indeed outperforming our model on the IM dataset for the age shift scenario. We suspect it is due to a strong covariate shift, where BBSD is known to perform really well. As for why other distance-based monitors such as MMD-D and Rel-MH aren’t achieving the same performance, we suspect they might be overfitting their representations to the training set. \n\n> Figure 2b [...]\n\nThanks for catching this! The y-axis is supposed to be labeled FPR@5%, our manuscript is since fixed. Distance-based learners are particularly susceptible to this type of FPR since they pick up on covariate shifts. We argue based on Figure 2a that this scenario is not a deteriorating shift, and therefore good monitors should resist flagging this change. \n\n> The x-axis in Fig 3 [...] \n\nThank you for the catch! These are the mixing coefficients between patients of different age groups as described in Lines 1315-1323. We will expand Fig3’s caption to be more comprehensive. \n\n## Overall Comments\n\n> Some references to specific Appendicies seem to be pointing to the wrong place \n\nThank you very much! We have proof-read our manuscript carefully. \n\n> \"D3m does not trade off prediction accuracy\" [...] \n\nThe classification accuracy is affected but only so slightly. Please refer to corresponding section of the response to Reviewer **TFQc** for a closer look. \n\n> I think the claim that \"we answer all desiderata\" it misleading [...]\n\nOur empirical observations show that a low FPR is indeed achieved in practice. Intuitively, if the deployment sample came from the same training distribution, then its disagreement rate is beyond the $1-\\alpha$ quantile with probability $\\alpha$. In practice, FPRs may be slightly higher due to $\\Phi$ being an empirical estimate and further approximation errors. \n\n> $\\tau$ \n\nThank you for pointing this out. We have included its definition.\n\n# Questions\n\n> How does the idealized algorithm that the theory is about relate to the actual algorithm? [...] \n\nAs per the above and as you have commented, the Bayesian sampling essentially over-approximates beyond the confines of the subset of hypotheses of interest. Another alternative is to try to optimize for $h$ directly via gradient descent with access to the training dataset in a similar fashion to Detectron (See **TFQc**). Users of D3M should view the theoretical results as **qualitative guidance rather than quantitative guarantees**, since the idealized D3M with idealized assumptions (mainly optimization oracle) do not fully hold in practice.\n\n> When will the algorithm fail? Are there any indicators or situations that users should look out for?\n\nEmpirically during testing, we found that when the temperature is high, while the ID FPR remains consistent empirically, the deteriorating OOD TPR is heavily impacted (See Response to Reviewer **Ktyz**). \n\nTheoretically, a regime that our theory isn’t equipped to handle is Regime 2 (Line 1110), where we analyze a toy example and propose that well-trained base classifiers (low generalization error) are less likely to fall into this situation. \n\n> How can users reduce the variability [...]\n\nWe found that lowering the temperature below 1 helps reduce variance significantly at the cost of TPR, and are looking for hyperparameters and perhaps design changes that would rectify this. \n\n# Closing Remarks\n\nThank you once again for your helpful, attentive, and detailed feedback. We hope that our rebuttal addresses your questions and concerns, and we kindly ask that you consider a fresher evaluation of our paper if you are satisfied with our responses. We are also more than happy to answer any further questions that arise."}]

Author rebuttal:

Meta-review: This paper proposes D3M, a method to detect post-deployment deterioration due to data distribution shift. It adds a variational bayesian last layer and measures disagreement among output samples. This work tackles an important safety-critical scenario, does not require training labels, and is able to detect deteriorating shifts from non-deteriorating shifts in contrast to standard OOD settings. It provides theoretical analysis, and compares the practical and idealized version empirically. Experiments on public datasets and a large-scale medical dataset demonstrate its efficacy and efficiency.

Reviewers were critical of multiple aspects, including the lack of theoretical foundation, unsupported claims and missing ablations. The rebuttal with additional theoretical justification and experiments addressed almost all the concerns. As a result, all reviewers agree on the acceptance of this submission.

Some concerns remain, though they are not considered rejection-worthy:
- Not significant scalability improvement in absolute term compared to the baseline (though the relative reduction of FLOPs is large)
- Strong assumptions in the theoretical justification, admitted by authors
Decision: Accept (poster)