Title: Window-Based Distribution Shift Detection for Deep Neural Networks
Venue: NeurIPS 2023
Accepted: yes
Keywords: Distribution shift detection

Window-based detection
Abstract: To deploy and operate deep neural models in production, the quality of their predictions, which might be contaminated benignly or manipulated maliciously by input distributional deviations, must be monitored and assessed. Specifically, we study the case of monitoring the healthy operation of a deep neural network (DNN) receiving a stream of data, with the aim of detecting input distributional deviations over which the quality of the network's predictions is potentially damaged. Using selective prediction principles, we propose a distribution deviation detection method for DNNs. The proposed method is derived from a tight coverage generalization bound computed over a sample of instances drawn from the true underlying distribution. Based on this bound, our detector continuously monitors the operation of the network over a test window and fires off an alarm whenever a deviation is detected. Our novel detection method performs on-par or better than the state-of-the-art, while consuming substantially lower computation time (five orders of magnitude reduction) and space complexity. Unlike previous methods, which require at least linear dependence on the size of the source distribution for each detection, rendering them inapplicable to ``Google-Scale'' datasets, our approach eliminates this dependence, making it suitable for real-world applications. Code is available at [https://github.com/BarSGuy/Window-Based-Distribution-Shift-Detection](https://github.com/BarSGuy/Window-Based-Distribution-Shift-Detection).

Window-Based Distribution Shift Detection for Deep Neural Networks

Abstract

To deploy and operate deep neural models in production, the quality of their predictions, which might be contaminated benignly or manipulated maliciously by input distributional deviations, must be monitored and assessed. Specifically, we study the case of monitoring the healthy operation of a deep neural network (DNN) receiving a stream of data, with the aim of detecting input distributional deviations over which the quality of the network's predictions is potentially damaged. Using selective prediction principles, we propose a distribution deviation detection method for DNNs. The proposed method is derived from a tight coverage generalization bound computed over a sample of instances drawn from the true underlying distribution. Based on this bound, our detector continuously monitors the operation of the network over a test window and fires off an alarm whenever a deviation is detected. Our novel detection method performs on-par or better than the state-of-the-art, while consuming substantially lower computation time (five orders of magnitude reduction) and space complexity. Unlike previous methods, which require at least linear dependence on the size of the source distribution for each detection, rendering them inapplicable to 'Google-Scale' datasets, our approach eliminates this dependence, making it suitable for real-world applications. Code is available at https:// github.com/BarSGuy/Window-Based-Distribution-Shift-Detection .

1 Introduction

Awide range of artificial intelligence applications and services rely on deep neural models because of their remarkable accuracy. When a trained model is deployed in production, its operation should be monitored for abnormal behavior, and a flag should be raised if such is detected. Corrective measures can be taken if the underlying cause of the abnormal behavior is identified. For example, simple distributional changes may only require retraining with fresh data, while more severe cases may require redesigning the model (e.g., when new classes emerge).

In this paper we focus on distribution shift detection in the context of deep neural models and consider the following setting. Pretrained model f is given, and we presume it was trained with data sampled from some distribution P . In addition to the dataset used in training f , we are also given an additional unlabeled sample of data from the same distribution, which is used to train a detector D (we refer to this as the detection-training set or source set). While f is used in production to process a stream of emerging input data, we continually feed D with the most recent window W k of k input elements.

Yonatan Geifman

Deci.AI

The detector also has access to the final layers of the model f and should be able to determine whether the data contained in W k came from a distribution different from P . We emphasize that in this paper we are not considering the problem of identifying single-instance out-of-distribution or outlier instances [28, 21, 22, 16, 38, 35, 34, 11], but rather the information residing in a population of k instances. While it may seem straightforward to apply single-instance detectors to a window (by applying the detector to each instance in the window), this approach can be computationally expensive since such methods are not designed for window-based tasks; see discussion in Section 3. Moreover, we demonstrate here that computationally feasible single-instance methods can fail to detect population-based deviations. We emphasize that we are not concerned in characterizing types of distribution shifts, nor do we tackle at all the complementary topic of out-of-distribution robustness.

Distribution shift detection has been scarcely considered in the context of deep neural networks (DNNs), however, it is a fundamental topic in machine learning and statistics. The standard method for tackling it is by performing a dimensionality reduction over both the detection-training (source) and test (target) samples, and then applying a two-sample statistical test over these reduced representations to detect a deviation. This is further discussed in Section 3. In particular, deep models can benefit from the semantic representation created by the model itself, which provides meaningful dimensionality reduction, that is readily available at the last layers of the model. Using the embedding layer (or softmax) along with statistical two-sample tests was recently proposed by Lipton et al. [29] and Rabanser et al. [37] who termed solutions of this structure black-box shift detection (BBSD). Using both the univariate Kolmogorov-Smirnov (KS) test and the maximum mean discrepancy (MMD) method, see details below, Rabanser et al. [37] achieve impressive detection results when using MNIST and CIFAR-10 as proxies for the distribution P . As shown here, the KS test is also very effective over ImageNet when a stronger model is used (ResNet50 vs ResNet18). However, BBSD methods have the disadvantage of being computationally intensive (and probably inapplicable to read-world datasets) due to the use of two-sample tests between the detection-training set (which can, and is preferred to be the largest possible) and the window W k ; a complexity analysis is provided in Section 5.

We propose a different approach termed Coverage-Based Detection (CBD), which builds upon selective prediction principles [9, 14]. In this approach, a model quantifies its prediction uncertainty and abstains from predicting uncertain instances. First, we develop a method for selective prediction with guaranteed coverage. This method identifies the best abstaining threshold and coverage bound for a given pretrained classifier f , such that the resulting empirical coverage will not violate the bound with high probability (when abstention is determined using the threshold). The guaranteed coverage method is of independent interest, and it is analogous to selective prediction with guaranteed risk [14]. Because the empirical coverage of such a classifier is highly unlikely to violate the bound if the underlying distribution remains the same, a systematic violation indicates a distribution shift. To be more specific, given a detection-training sample S m , our coverage-based detection algorithm computes a fixed number of tight generalization coverage bounds, which are then used to detect a distribution shift in a window W k of test data. The proposed detection algorithm exhibits remarkable computational efficiency due to its ability to operate independently of the size of S m during detection, which is crucial when considering 'Google-Scale' datasets, such as the JFT-3B dataset. In contrast, the currently available distribution shift detectors rely on a framework that requires significantly higher computational requirements (this framework is illustrated in Figure 3 in Appendix A). A run-time comparison of those baselines w.r.t. our CBD method is provided in Figure 1.

In a comprehensive empirical study, we compared our CBD algorithm with the best-performing baselines, including the KS approach of [37]. Additionally, we investigated the suitability of singleinstance detection methods for identifying distribution shifts. For a fair comparison, all methods used the same (publicly available) underlying models: ResNet50, MobileNetV3-S, and ViT-T. To evaluate the effectiveness of our approach, we employed the ImageNet dataset to simulate the source distribution. We then introduced distribution shifts using a range of methods, starting with simple noise and progressing to more sophisticated techniques such as adversarial examples. Based on these experiments, it is evident that CBD is overall significantly more powerful than the baselines across a wide range of test window sizes.

To summarize, the contributions of this paper are: (1) A theoretically justified algorithm (Algorithm 1), that produces a coverage bound, which is of independent interest, and allows for the creation of selective classifiers with guaranteed coverage. (2) A theoretically motivated 'windowed' detection

algorithm, CBD (Algorithm 2), which detects a distribution shift over a window; this proposed algorithm exhibits remarkable efficiency compared to state-of-the-art methods (five orders of magnitude better than the best method). (3) A comprehensive empirical study demonstrating significant improvements relative to existing baselines, and introducing the use of single-instance methods for detecting distribution shifts.

2 Problem Formulation

We consider the problem of detecting distribution shifts in input streams provided to pretrained deep neural models. Let P ≜ P X denote a probability distribution over an input space X , and assume that a model f has been trained on a set of instances drawn from P . Consider a setting where the model f is deployed and while being used in production its input distribution might change or even be attacked by an adversary. Our goal is to detect such events to allow for appropriate action, e.g., retraining the model with respect to the revised distribution.

Inspired by Rabanser et al. [37], we formulate this problem as follows. We are given a pretrained model f , and a detection-training set, S m ∼ P m . Then we would like to train a detection model to be able to detect a distribution shift; namely, discriminate between windows containing in-distribution (ID) data, and alternative-distribution (AD) data. Thus, given an unlabeled test sample window W k ∼ Q k , where Q is a possibly different distribution, the objective is to determine whether P ̸ = Q . We also ask what is the smallest test sample size k required to determine that P ̸ = Q . Since typically the detection-training set S m can be quite large, we further ask whether it is possible to devise an effective detection procedure whose time complexity is o ( m ) .

3 Related Work

Distribution shift detection methods often comprise the following two steps: dimensionality reduction, and a two-sample test between the detection-training sample and test samples. In most cases, these methods are 'lazy' in the sense that for each test sample, they make a detection decision based on a computation over the entire detection-training sample. Their performance will be sub-optimal if only a subset of the train sample is used. Figure 3 in Appendix A illustrates this general framework.

The use of dimensionality reduction is optional. It can often improve performance by focusing on a less noisy representation of the data. Dimensionality reduction techniques include no reduction, principal components analysis [1], sparse random projection [2], autoencoders [39, 36], domain classifiers , [37] and more. In this work we focus on black box shift detection (BBSD) methods [29], that rely on deep neural representations of the data generated by a pretrained model. The representation extracted from the model will typically utilize either the softmax outputs, acronymed BBSD-Softmax, or the embeddings, acronymed BBSD-Embeddings; for simplicity, we may omit the BBSD acronym. Due to the dimensionality of the final representation, multivariate or multiple univariate two-sample tests can be conducted.

By combining BBSD-Softmax with a Kolmogorov-Smirnov (KS) statistical test [33] and using the Bonferroni correction [3], Rabanser et al. [37] achieved state-of-the-art results in distribution shift detection in the context of image classification (MNIST and CIFAR-10). We acronym their method as BBSD-KS-Softmax (or KS-Softmax). The univariate KS test processes individual dimensions separately; its statistic is calculated by computing the largest difference Z of the cumulative density functions (CDFs) across all dimensions as follows: Z = sup z | F P ( z ) -F Q ( z ) | , where F Q and F P are the empirical CDFs of the detection-training and test data (which are sampled from P and Q , respectively; see Section 2). The Bonferroni correction rejects the null hypothesis when the minimal p-value among all tests is less than α d , where α is the significance level and d is the number of dimensions. Although less conservative approaches to aggregation exist [20, 30], they usually assume some dependencies among the tests. The maximum mean discrepancy (MMD) method [18] is a kernel-based multivariate test that can be used to distinguish between probability distributions P and Q . Formally, MMD 2 ( F , P , Q ) = || µ P -µ Q || 2 F 2 , where µ P and µ Q are the mean embeddings of P and Q in a reproducing kernel Hilbert space F . Given a kernel K , and samples, { x 1 , x 2 , . . . , x m } ∼ P m and { x ′ 1 , x ′ 2 , . . . , x ′ k } ∼ Q k , an unbiased estimator for MMD 2 can be found in [18, 42]. Sutherland et al. [43] and Gretton et al. [18] used the RBF kernel K ( x, x ′ ) = e -1 2 σ 2 || x -x ′ || 2 2 , where 2 σ 2 is set to the median of the pairwise Euclidean distances

between all samples. By performing a permutation test on the kernel matrix, the p-value is obtained. In our experiments (see Section 6.4), we thus use four population based baselines: KS-Softmax , KS-Embeddings , MMD-Softmax , and MMD-Embeddings .

As mentioned in the introduction, our work is complementary to the topic of single-instance out-ofdistribution (OOD) detection [28, 21, 22, 16, 38, 35, 34, 11]. Although these methods can be applied to each instance in a window, they often fail to capture population statistics within the window, making them inadequate for detecting population-based changes. Additionally, many of these methods are computationally expensive and cannot be applied efficiently to large windows. For example, methods such as those described in [41, 27] extract values from each layer in the network, while others such as [28] require gradient calculations. We note that the application of the best single-instance methods such as [41, 27, 28] in our (large scale) empirical setting is computationally challenging and preclude empirical comparison to our method. Therefore, we consider (in Section 6.4) the detection performance of two computationally efficient single-instance baselines: Softmax-Response (abbreviated as Single-instance SR or Single-SR) and Entropy-based (abbreviated as Single-instance Ent or Single-Ent), as described in [4, 6, 21]. Specifically, we apply each single-instance OOD detector to every instance in the window and in the detection-training set. We then use a two-sample t-test to determine the p-value between the uncertainty estimators of each sample.

Finally, we mention [14] who developed a risk generalization bound for selective classifiers [9]. The bound presented in that paper is analogous to the coverage generalization bound we present in Theorem 4.2. The risk bound in [14] can also be used for shift-detection. To apply their risk bound to this task, however, labels, which are not available, are required. CBD detects distribution shifts without using any labels.

4 Proposed Method - Coverage-Based Detection

In this section we present Coverage-Based Detection (CBD), a novel technique for detecting a distribution shift based on selective prediction principles (definitions follow). We develop a tight generalization coverage bound that holds with high probability for ID data, sampled from the source distribution. The main idea is that violations of this coverage bound indicate a distribution shift with high probability. In Section 6.2, we offer an intuitive demonstration that aids in understanding our approach.

4.1 Selection with Guaranteed Coverage

We begin by introducing basic selective prediction terminology and definitions that are required to describe our method. Consider a standard multiclass classification problem, where X is some feature space (e.g., raw image data) and Y is a finite label set, Y = { 1 , 2 , 3 , ..., C } , representing C classes. Let P ( X,Y ) be a probability distribution over X × Y , and define a classifier as a function f : X → Y . We refer to P as the source distribution . A selective classifier [9] is a pair ( f, g ) , where f is a classifier and g : X → { 0 , 1 } is a selection function [9], which serves as a binary qualifier for f as follows,

A general approach for constructing a selection function based on a given classifier f is to work in terms of a confidence-rate function [15], κ f : X → R + , referred to as CF. The CF κ f should quantify confidence in predicting the label of x based on signals extracted from f . The most common and well-known CF for a classification model f (with softmax at its last layer) is its softmax response (SR) value [4, 6, 21]. A given CF κ f can be straightforwardly used to define a selection function: g θ ( x ) ≜ g θ ( x | κ f ) = 1 [ κ f ( x ) ≥ θ ] , where θ is a user-defined constant. For any selection function, we define its coverage w.r.t. a distribution P (recall, P ≜ P X , see Section 2), and its empirical coverage w.r.t. a sample S k ≜ { x 1 , x 2 , . . . x k } , as c ( θ, P ) ≜ E P [ g θ ( x )] , and ˆ c ( θ, S k ) ≜ 1 k ∑ k i =1 g θ ( x i ) , respectively.

Given a bound on the expected coverage for a given selection function, we can use it to detect a distribution shift via violations of the bound. We will now formally state the problem, develop the bound, and demonstrate its use in detecting distribution shifts.

Problem statement. For a classifier f , a detection-training sample S m ∼ P m , a confidence parameter δ > 0 , and a desired coverage c ∗ > 0 , our goal is to use S m to find a θ value (which implies a selection function g θ ) that guarantees the desired coverage, with probability 1 -δ , namely,

This means that under coverage should occur with probability of at most δ .

A threshold θ that guarantees Equation (1) provides a probabilistic lower bound, guaranteeing that coverage c of ID unseen population (sampled from P ) satisfies c > c ∗ with probability of at least 1 -δ . For the remaining of this section, we introduce the selection with guaranteed coverage ( SGC ) algorithm (Algorithm 1), which outputs a bound ( b ∗ ) and its corresponding threshold ( θ ).

The SGC algorithm receives as input a classifier f , a CF κ f , a confidence parameter δ , a target coverage c ∗ , and a detection-training set S m . The algorithm performs a binary search to find the optimal coverage lower bound with confidence δ , and outputs a coverage bound b ∗ and the corresponding threshold θ , defining the selection function. A pseudo code of the SGC algorithm appears in Algorithm 1.

Our analysis of the SGC algorithm makes use of Lemma 4.1, which gives a tight numerical (generalization) bound on the expected coverage, based on a test over a sample. The proof of Lemma 4.1 is nearly identical to Langford's proof of Theorem 3.3 in [26], p. 278, where instead of the empirical error used in [26], we use the empirical coverage, which is also a Bernoulli random variable.

Lemma 4.1. Let P be any distribution and consider a selection function g θ with a threshold θ whose coverage is c ( θ, P ) . Let 0 < δ < 1 be given and let ˆ c ( θ, S m ) be the empirical coverage w.r.t. the set S m , sampled i.i.d. from P . Let b ∗ ( m,m · ˆ c ( θ, S m ) , δ ) be the solution of the following equation:

The following is a uniform convergence theorem for the SGC procedure stating that all the calculated bounds are valid simultaneously with a probability of at least 1 -δ .

Theorem 4.2. ( SGC - Uniform convergence) Assume S m is sampled i.i.d. from P , and consider an application of Algorithm 1. For k = ⌈ log 2 m ⌉ , let b ∗ i ( m,m · ˆ c i ( θ i , S m ) , δ k ) and θ i be the values obtained in the i th iteration of Algorithm 1. Then,

Then,

Proof (sketch - see full proof in the Appendix B.1). Define, B θ i ≜ b ∗ i ( m,m · ˆ c i ( θ i , S m ) , δ k ) , C θ i ≜ c ( θ i , P ) , then,

We would like to note that in addition to Algorithm 1, we also provide a complementary algorithm that returns a tight coverage upper (rather than lower) bound, along with the corresponding threshold. Details about this algorithm and its application for detecting distribution shifts are discussed in Appendix F.

4.2 Coverage-Based Detection Algorithm

Our coverage-based detection (CBD) algorithm applies SGC to C target target coverages uniformly spread between the interval [0 . 1 , 1] , excluding the coverage of 1. We set C target = 10 , δ = 0 . 01 , and κ f ( x ) = 1 -Entropy ( x ) for all our experiments; our method appears to be robust to those hyper-parameters, as we demonstrate in Appendix E. Each application j of SGC on the same sample S m ∼ P m with a target coverage of c ∗ j produces a pair: ( b ∗ j , θ j ) , which represent a bound and a threshold, respectively. We define δ (ˆ c | b ∗ ) to be a binary function that indicates a bound violation, where ˆ c is the empirical coverage (of a sample) and b ∗ is a bound, δ (ˆ c | b ∗ ) = 1 [ˆ c ≤ b ∗ ] . Thus, given a window of k samples from an alternative distribution, W k ∼ Q k , we define the sum of bound violations , V , as follows,

where we obtain the last equality by using ˆ c j ≜ ˆ c ( θ j , W k ) = 1 k ∑ k i =1 g θ j ( x i ) . Considering Figure 2b in Section 6.2, the quantity V is the sum of distances from the violations (red dots) to the linear diagonal representing the coverage bounds (black line).

Algorithm 2: Coverage-Based Detection

When Q equals P , which implies that there is no distribution shift, we expect that all the bounds (computed by SGC over S m ) will hold over W k , namely δ (ˆ c j | b ∗ j ) = 0 for every iteration j ; in this case, V = 0 . Otherwise, V will indicate the violation magnitude.

Since V represents the average of k · C target values (if at least one bound violation occurs), for cases where k · C target ≫ 30 (as in all our experiments), we can assume that V follows a nearly normal distribution [25] and perform a t-test 1 to test the null hypothesis, H 0 : V = 0 , against the alternative hypothesis, H 1 : V > 0 . The null hypothesis is rejected if the p-value is less than the specified significance level α . A pseudocode of our coverage-based detection algorithm appears in Algorithm 2.

To fit our detector, we apply SGC (Algo- rithm 1) on the detection-training set, S m , for C target times, in order to construct the pairs { ( b ∗ j , θ j ) } C target j =1 . Our detection model utilizes these pairs to monitor a given model, receiving at each time instant a test sample window of size k (user defined), W k = { x 1 , x 2 , . . . , x k } , which is inspected to see if its content is distributionally shifted from the underlying distribution reflected by the detection-training set S m .

Our approach encodes all necessary information for detecting distribution shifts using only C target scalars (in our experiments, we set C target = 10 ), which is independent of the size of S m . In contrast, the baselines process the detection-training set, S m , which is typically very large, for every detection they make. This makes our method significantly more efficient than the baselines, see Figure 1 and Table 1 in Section 5.

5 Complexity Analysis

This section provides a complexity analysis of our method as well as the baselines mentioned in Section 3. Table 1 summarizes the complexities of each approach. Figure 1 shows the population-based approaches run-time (in seconds) as a function of the detection-training set size, denoted as m .

All population-based baselines are lazy learners (analogous to nearest neighbors) in the sense that they require the entire source (detection-training) set for each detection decision they make. Using only a subset will result in sub-optimal performance, since it might not capture all the necessary information within the source distribution, P .

Table 1: Complexity comparison. bold entries indicate the best detection complexity. m , k refers to the detectiontraining size (or source size), and window size, respectively. d refers to the number of dimensions after dimensionality reduction.

In particular, MMD is a permutation test [18] that also employs a kernel. The complexity of kernel methods is dominated by the number of instances and, therefore, the time and space complexities of MMD are O ( d ( m 2 + k 2 + mk )) and O ( m 2 + k 2 + mk ) , respectively 2 , where in the case of DNNs, d is the dimension of the embedding or softmax layer used for computing the kernel, and k is the window size. The KS test [33] is a univariate test, which is applied on each dimension separately and then aggregates the results via a Bonferroni correction. Its time and space complexities are O ( d ( m log m + k log k )) and O ( d ( m + k )) , respectively.

The single-instance baselines simply conduct a t-test on the heuristic uncertainty estimators (SR or Entropy-based) between the detection-training set and the window data. By initially determining the mean and standard deviation of the detection-training set, we can efficiently perform the t-test with a time and space complexity of O ( k ) .

The fitting procedure of our coverage-based detection algorithm incurs time and space complexities of O ( m log m ) and O ( m ) , respectively. Each subsequent detection round incurs time and space complexities of O ( k ) , which is independent of the size of the detection-training (or source) set. Our method's superior efficiency is demonstrated both theoretically and empirically, as shown in Table 1 and Figure 1, respectively. Figure 1 shows the run-time (in seconds) for each population-based detection method 3 as a function of the detection-training set size, using a ResNet50 and a fixed window size of k = 10 samples. Our detection time remains constant regardless of the size of the detection-training set, as opposed to the baseline methods, which exhibit a run-time that grows as a function of the detection-training set size. In particular, our method achieves a five orders of magnitude improvement in run-time over the best performing baseline, KS-Softmax, when the detection-training set size is 1,000,000 4 . Specifically, our detection time is 5 . 19 · 10 -4 seconds, while KS-Softmax's detection time is 97 . 1 seconds, rendering it inapplicable for real-world applications.

6 Experiments - Detecting Distribution Shifts

In this section we showcase the effectiveness of our method, along with the considered baselines, in detecting distribution shifts. All experiments were conducted using PyTorch, and the corresponding code is publicly available for replicating our results 5 .

10 1

10 2

10 3

10 4

10 5

10 6

Detection-training set size (

m

)

10

4

10

3

10

2

10

1

10 0

10 1

10 2

Run-time (s)

5.19×10

4

(s)

9.71×10 1 (s)

2.03×10

2

(s)

Methods

CBD (Ours)

KS-Embeddings

KS-Softmax

MMD-Embeddings

MMD-Softmax

6.1 Setup

Our experiments are conducted on the ImageNet dataset [7], using its validation dataset as proxies for the source distribution, P . We utilized three well-known architectures, ResNet50 [19], MobileNetV3-S [24], and ViT-T [8], all of which are publicly available in timm's repository [45], as our pre-trained models. To train our detectors, we randomly split the ImageNet validation data (50,000) into two sets, a detection-training or source set, which is used to fit the detectors 3 (49,000) and a validation set (1,000) for applying the shift. To ensure the reliability of our results, we repeated the shift detection performance evaluation on 15 random splits, applying the same type of shift to different subsets of the data. Inspired by [37], we evaluated the models using various window sizes, | W k |∈ { 10 , 20 , 50 , 100 , 200 , 500 , 1000 } .

6.1.1 Distribution Shift Datasets

At test time, the 1,000 validation images obtained via our split can be viewed as the in-distribution (positive) examples. For out-of-distribution (negative) examples, we follow the common setting for detecting OOD samples or distribution shifts [37, 21, 28], and test on several different natural image datasets and synthetic noise datasets. More specifically, we investigate the following types of shifts:

(1) Adversarial via FGSM : We transform samples into adversarial ones using the Fast Gradient Sign Method (FGSM) [17], with ϵ ∈ { 7 · 10 -5 , 1 · 10 -4 , 3 · 10 -4 , 5 · 10 -4 } . (2) Adversarial via PGD : We convert samples into adversarial examples using Projected Gradient Descent (PGD) [31], with ϵ = 1 · 10 -4 ; we use 10 steps, α = 1 · 10 -4 , and random initialization. (3) Gaussian noise : We corrupt test samples with Gaussian noise using standard deviations of σ ∈ { 0 . 1 , 0 . 3 , 0 . 5 , 1 } . (4) Rotation : We apply image rotations, θ ∈ { 5 ◦ , 10 ◦ , 20 ◦ , 25 ◦ } . (5) Zoom : We corrupt test samples by applying zoom-out percentages of { 90% , 70% , 50% } . (6) ImageNet-O : We use the ImageNet-O dataset [23], consisting of natural out-of-distribution samples. (7) ImageNet-A : We use the ImageNet-A dataset [23], consisting of natural adversarial samples. (8) No-shift : We include the no-shift case to check for false positives. To determine the severity of the distribution shift, we refer the reader to Table 3 in Appendix C.

6.2 Maximizing Detection Power Through Lower Coverages

In this section, we provide an intuitive understanding of our proposed method, as well as a demonstration of the importance of considering lower coverages, when detecting population-based distribution shifts. For all experiments in this section, we employed a ResNet50 [19] and used a detection-training set which was randomly sampled from the ImageNet validation set.

We validate the effectiveness of our CBD method (Algorithm 2) by demonstrating its performance on two distinct scenarios, the no-shift case and the shift case. In the no-shift scenario, as depicted in Figure 2a, we randomly

(a) No-shift case.

(b) Shift case.

0.9

Coveragebound

0.9

Coverage bound

Empirical coverage

Empirical coverage

0.8

×

Targetcoverage

0.8

×

Target coverage

Violations

0.7

0.7

0.6

Coverage

0.6

Coverage

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1 +

0.1

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Target coverage

Targetcoverage

selected a 1,000 sample window from the ImageNet dataset that did not overlap with the detection-training set. We then tested whether this window was distributionally shifted using our method. The shift scenario, as depicted in Figure 2b, involved simulating a distributional shift by randomly selecting 1,000 images from the ImageNet-O dataset. Each figure showcases the target coverages of each application, the coverage bounds provided by SGC , the empirical coverage of each sample (for each threshold), and the bound violations (indicated by the red circles), when they occur.

Both figures show that the target coverages and the bounds returned by SGC are essentially identical. Consider the empirical coverage of each threshold returned by SGC for each sample. In the no-shift case, illustrated in Figure 2a, it is apparent that all bounds are tightly held; demonstrated by the fact that each empirical coverage is slightly above its corresponding bound. In contrast, the shift case, depicted in Figure 2b, exhibits bound violations, indicating a noticeable distribution shift. Specifically, the bound holds at a coverage of 0.9, but violations occur at lower coverages, with the maximum violation magnitude occurring around a coverage of 0.4. This underscores the significance of using coverages lower than 1, and suggests that the detection power might lay within lower coverages.

6.3 Evaluation Metrics

In order to benchmark our results against the baselines, we adopt the following metrics to measure the effectiveness in distinguishing between in- and out-of-distribution windows: (1) Area Under the Receiver Operating Characteristic ↑ (AUROC) is a threshold-independent metric [5]. The ROC curve illustrates the relationship between the true positive rate (TPR) and the false positive rate (FPR), and the AUROC score represents the probability that a positive example will receive a higher detection score than a negative example [10]. A perfect detector would achieve an AUROC of 100%. (2) Area Under the Precision-Recall ↑ (AUPR) is another threshold-independent metric [32, 40]. The PR curve is a graph that depicts the relationship between precision (TP / (TP + FP)) and recall (TP / (TP + FN)). AUPR-In and AUPR-Out in Table 2 represent the area under the precision-recall curve where in-distribution and out-of-distribution images are considered as positives, respectively. (3) False positive rate (FPR) at 95% true positive rate (TPR) ↓ , denoted as FPR@95TPR, is a performance metric that measures the FPR at a specific TPR threshold of 95%. This metric is calculated by finding the FPR when the TPR is 95%. (4) Detection Error ↓ is the probability of misclassification when TPR is fixed at 95%. It's a weighted average of FPR and complement of TPR, given by Pe = 0 . 5(1 -TPR )+0 . 5 FPR . Assuming equal probability of positive and negative examples in the test set. We note that ↑ indicates larger value is better, and ↓ indicates lower values is better.

6.4 Experimental Results

In this section we demonstrate the effectiveness of our CBD method, as well as the considered baselines. To ensure a fair comparison, as mentioned in Section 6.1 ,we utilize the ResNet50, MobileNetV3-S, and ViT-T architectures, which are pretrained on ImageNet and their open-sourced code appear at timm [45]. We provide code for reproducing our experiments 5 .

Table 2 presents a comprehensive summary of our main results, which clearly demonstrate that CBD outperforms all baseline methods in the majority of cases (combinations of model architecture, window size, and metrics considered), thus providing a clear indication of the effectiveness of our approach. Our method demonstrates the highest effectiveness in the low to mid window size regime. Specifically, for window sizes of 50 samples,

our method outperforms all baselines across all architectures, with statistical significance for all considered metrics, by a large margin. This observation is true for window sizes of 100 samples as well, except for the MobileNet architecture, where KS-Softmax shows slightly better performance in the metrics of Detection Error and FPR@95TPR. The second-best performing method overall appears to be KS-Softmax (consistent with the findings of [37]), which seems to be particularly effective in large window sizes, such as 500 or more. CBD stands out for its consistent performance across different architectures. In particular, our method achieves an AUROC score of over 90% across all architectures tested when using a window size of 50 + , while the second best performing method (KS-Softmax), fails to achieve such a score even with a window size of 100 samples, when using the ResNet50 architecture.

It is evident from Table 2, that as far as population-based detection tasks are concerned, neither of the singleinstance baselines (SR and Entropy) offers an effective solution. These baselines show inconsistent performance across the various architectures considered. For instance, when ResNet50 is used over a window size of 1,000 samples, the baselines scarcely achieve a score of 90% for the threshold-independent metrics (AUROC, AUPR-In, and AUPR-Out), while other methods achieve near-perfect scores. Moreover, even in the low sample regime (e.g., window size of 10), these baselines fail to demonstrate any superiority over population-based methods.

Finally, based on our experiments, ResNet50 appears to be the best model for achieving low Detection Error and FPR@95TPR; and a large window size is essential for achieving these results. In particular, our method and KS-Softmax both produced impressive outcomes, with Detection Error and FPR@95TPR scores under 0.5 and 1, respectively, when a window size of 1,000 samples was employed. Moreover, we observe that when small window sizes are desired, ViT-T is the best architecture choice. In particular, our method applied with ViT-T, achieves a phenomenal 86%+ score in all threshold independent metrics (i.e., AUROC, AUPR-In, AUPR-Out), over a window size of 10 samples, strongly out-preforming the contenders (around 20% margin). Appendix D presents the detection performance on a representative selection of distribution shifts, analyzed individually.

Architecture

Method

Window size

AUROC

↑

/ AUPR-In

↑

/ AUPR-Out

↑

/ DetectionError

↓

/ FPR@95TPR

↓

10

20

50

100

200

500

1000

ResNet50

KS

Softmax

61

/

67

/

62

/

34

/

67

73

/

74

/

74

/

31

/

64

87

/

90

/

85

/

13

/

27

89

/

89

/

89

/

15

/

29

94

/

95

/

92

/

7

/

14

99

/

99

/

99

/

2

/

4

∗

100

/

100

/

100

/

0.4

/

0.9

Embeddings

72

/

74

/

73

/

28

∗

/

56

∗

68

/

73

/

74

/

24

/

48

81

/

84

/

79

/

18

/

37

75

/

76

/

79

/

22

/

44

76

/

79

/

79

/

20

/

40

84

/

87

/

84

/

13

/

26

86

/

88

/

84

/

13

/

26

MMD

Softmax

54

/

61

/

56

/

36

/

72

62

/

65

/

62

/

37

/

72

73

/

76

/

72

/

29

/

56

73

/

73

/

78

/

33

/

59

79

/

79

/

79

/

35

/

54

83

/

85

/

83

/

15

/

30

85

/

85

/

85

/

22

/

37

Embeddings

75

/

72

/

77

/

38

/

70

79

/

78

/

79

/

29

/

57

87

/

87

/

86

/

18

/

37

83

/

86

/

81

/

15

/

30

83

/

85

/

82

/

14

/

29

83

/

85

/

83

/

17

/

32

83

/

86

/

82

/

13

/

26

Single-instance

SR

56

/

65

/

55

/

34

/

68

72

/

73

/

72

/

32

/

63

71

/

75

/

72

/

28

/

56

77

/

78

/

79

/

25

/

50

84

/

85

/

83

/

19

/

40

87

/

88

/

87

/

14

/

28

88

/

88

/

89

/

15

/

30

Entropy

64

/

69

/

63

/

32

/

64

73

/

73

/

73

/

32

/

63

74

/

78

/

73

/

26

/

52

80

/

80

/

81

/

23

/

47

84

/

85

/

84

/

17

/

35

87

/

87

/

87

/

15

/

31

90

/

90

/

91

/

13

/

26

CBD (Ours)

78

/

70

/

82

∗

/

42

/

84

88

∗

/

91

∗

/

87

∗

/

15

∗

/

30

∗

95

∗

/

95

∗

/

93

∗

/

9

∗

/

17

∗

93

∗

/

93

∗

/

92

∗

/

10

∗

/

20

∗

97

∗

/

97

∗

/

97

∗

/

5

/

10

98

/

98

/

98

/

4

/

7

100

/

100

/

100

/

0.4

/

0.9

MobileNetV3-S

KS

Softmax

71

/

72

/

75

/

32

∗

/

63

∗

84

∗

/

84

/

83

/

21

/

43

89

/

91

/

88

/

13

/

27

92

/

93

/

91

/

10

/

20

95

/

97

/

94

/

5

/

11

96

/

96

/

97

/

6

/

11

100

/

100

/

100

/

1

/

2

Embeddings

63

/

67

/

63

/

37

/

75

65

/

66

/

67

/

37

/

75

77

/

78

/

76

/

27

/

54

72

/

73

/

76

/

27

/

53

84

/

83

/

86

/

22

/

43

86

/

87

/

86

/

15

/

30

79

/

81

/

81

/

18

/

36

MMD

Softmax

75

/

73

/

75

/

38

/

72

78

/

78

/

78

/

30

/

59

86

/

89

/

82

/

17

/

32

86

/

89

/

84

/

14

/

26

87

/

88

/

86

/

14

/

28

89

/

90

/

88

/

12

/

24

90

/

91

/

88

/

11

/

22

Embeddings

67

/

67

/

68

/

39

/

75

66

/

67

/

68

/

37

/

74

72

/

77

/

71

/

23

/

47

75

/

75

/

79

/

28

/

53

89

/

87

/

87

/

20

/

39

81

/

82

/

81

/

21

/

40

82

/

86

/

80

/

15

/

30

Single-instance

SR

58

/

62

/

60

/

37

/

74

65

/

70

/

66

/

30

/

60

86

/

87

/

86

/

19

/

39

86

/

88

/

84

/

15

/

30

93

/

93

/

93

/

11

/

22

96

/

97

/

95

/

5

/

10

98

/

98

/

97

/

3

/

7

Entropy

52

/

61

/

57

/

36

/

72

64

/

70

/

65

/

29

/

57

85

/

86

/

86

/

17

/

33

87

/

88

/

85

/

15

/

29

93

/

93

/

94

/

11

/

22

96

/

96

/

96

/

6

/

12

98

/

99

/

98

/

3

/

7

CBD (Ours)

80

∗

/

73

/

82

∗

/

40

/

80

80

/

85

/

76

/

20

/

40

94

∗

/

95

∗

/

93

∗

/

8

∗

/

15

∗

94

/

94

/

94

∗

/

11

/

21

95

/

96

/

95

/

6

/

13

97

/

98

/

96

/

4

/

8

99

/

99

/

99

/

1

/

2

ViT-T

KS

Softmax

62

/

68

/

66

/

30

/

59

85

/

86

/

83

/

19

/

41

82

/

82

/

83

/

21

/

42

90

/

91

/

91

/

11

/

22

88

/

89

/

90

/

11

/

22

95

/

96

/

95

/

5

/

11

98

/

98

/

98

/

3

/

6

Embeddings

68

/

66

/

71

/

38

/

76

76

/

83

/

73

/

19

/

38

82

/

86

/

80

/

17

/

34

81

/

82

/

81

/

20

/

39

81

/

84

/

80

/

17

/

34

76

/

75

/

82

/

22

/

44

84

/

83

/

86

/

19

/

38

MMD

Softmax

58

/

59

/

64

/

44

/

82

69

/

70

/

73

/

38

/

69

77

/

80

/

77

/

22

/

44

75

/

80

/

76

/

20

/

40

80

/

86

/

78

/

15

/

29

89

/

91

/

90

/

12

/

23

93

/

94

/

92

/

7

/

15

Embeddings

61

/

59

/

68

/

46

/

85

74

/

77

/

74

/

26

/

53

82

/

84

/

80

/

20

/

40

80

/

82

/

83

/

21

/

39

82

/

81

/

82

/

20

/

40

78

/

76

/

81

/

22

/

44

77

/

78

/

79

/

24

/

45

Single-instance

SR

67

/

70

/

70

/

31

/

61

78

/

76

/

77

/

29

/

58

76

/

79

/

76

/

23

/

45

89

/

90

/

86

/

14

/

29

91

/

93

/

91

/

9

/

20

97

/

97

/

97

/

5

/

11

99

/

99

/

98

/

2

/

5

Entropy

69

/

74

/

69

/

27

/

53

79

/

78

/

78

/

27

/

55

75

/

78

/

74

/

22

/

44

89

/

89

/

90

/

15

/

31

86

/

87

/

87

/

14

/

28

93

/

94

/

93

/

7

/

14

97

/

97

/

97

/

4

/

9

CBD (Ours)

89

∗

/

86

∗

/

90

∗

/

28

/

56

91

∗

/

87

/

92

∗

/

24

/

47

94

∗

/

93

∗

/

95

∗

/

13

∗

/

25

∗

95

∗

/

96

∗

/

96

∗

/

8

∗

/

15

∗

97

∗

/

96

∗

/

97

∗

/

8

/

16

98

∗

/

98

/

98

/

4

/

9

99

/

99

/

99

/

3

/

6

7 Concluding Remarks

We presented a novel and powerful method for the detection of distribution shifts within a given window of samples. This coverage-based detection algorithm is theoretically motivated and can be applied to any pretrained model. Due to its low computational complexity, our method, unlike typical baselines, which are order-ofmagnitude slower, is practicable. Our comprehensive empirical studies demonstrate that the proposed method works very well, and overall significantly outperforms the baselines on the ImageNet dataset, across a number of neural architectures and a variety of distribution shifts, including adversarial examples. In addition, our coverage bound is of independent interest and allows for the creation of selective classifiers with guaranteed coverage.

Several directions for future research are left open. Although we only considered classification, our method can be extended to regression using an appropriate confidence-rate function such as the MC-dropout [12]. Extensions to other tasks, such as object detection and segmentation, would be very interesting. In our method, the information from the multiple coverage bounds was aggregated by averaging, but it is plausible that other statistics or weighted averages could provide more effective detections. Finally, an interesting open question is whether one can benefit from using single-instance or outlier/adversarial detection techniques combined with population-based detection techniques (as discussed here).

Acknowledgments

This research was partially supported by the Israel Science Foundation, grant No. 710/18.

References

Hervé Abdi and Lynne J Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics , 2(4):433-459, 2010.

Ella Bingham and Heikki Mannila. Random projection in dimensionality reduction: applications to image and text data. In Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining , pages 245-250, 2001.

J Martin Bland and Douglas G Altman. Multiple significance tests: the bonferroni method. Bmj , 310 (6973):170, 1995.

Luigi Pietro Cordella, Claudio De Stefano, Francesco Tortorella, and Mario Vento. A method for improving classification reliability of multilayer perceptrons. IEEE Transactions on Neural Networks , 6(5):1140-1147, 1995.

Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In Proceedings of the 23rd international conference on Machine learning , pages 233-240, 2006.

Claudio De Stefano, Carlo Sansone, and Mario Vento. To reject or not to reject: that is the questionan answer in case of neural classifiers. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) , 30(1):84-94, 2000.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations , 2021. URL https://openreview.net/forum? id=YicbFdNTTy .

Ran El-Yaniv et al. On the foundations of noise-free selective classification. Journal of Machine Learning Research , 11(5), 2010.

Tom Fawcett. An introduction to roc analysis. Pattern recognition letters , 27(8):861-874, 2006.

Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-of-distribution detection. Advances in Neural Information Processing Systems , 34:7068-7081, 2021.

Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning , pages 1050-1059. PMLR, 2016.

Ido Galil, Mohammed Dabbah, and Ran El-Yaniv. What can we learn from the selective prediction and uncertainty estimation performance of 523 imagenet classifiers? In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=p66AzKi6Xim .

Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks. Advances in neural information processing systems , 30, 2017.

Yonatan Geifman, Guy Uziel, and Ran El-Yaniv. Bias-reduced uncertainty estimation for deep neural classifiers. In International Conference on Learning Representations , 2019. URL https://openreview. net/forum?id=SJfb5jCqKm .

Izhak Golan and Ran El-Yaniv. Deep anomaly detection using geometric transformations. Advances in neural information processing systems , 31, 2018.

Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations , 2015. URL http://arxiv.org/abs/1412. 6572 .

Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research , 13(1):723-773, 2012.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 770-778, 2016. doi: 10.1109/CVPR.2016.90.

Nicholas A Heard and Patrick Rubin-Delanchy. Choosing between methods of combining-values. Biometrika , 105(1):239-246, 2018.

Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations , 2017. URL https: //openreview.net/forum?id=Hkg4TI9xl .

Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations , 2019. URL https://openreview.net/ forum?id=HyxCxhRcY7 .

Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15262-15271, 2021.

Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al. Searching for mobilenetv3. In Proceedings of the IEEE/CVF international conference on computer vision , pages 1314-1324, 2019.

Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani. An introduction to statistical learning , volume 112. Springer, 2013.

John Langford and Robert Schapire. Tutorial on practical prediction theory for classification. Journal of machine learning research , 6(3), 2005.

Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting outof-distribution samples and adversarial attacks. Advances in neural information processing systems , 31, 2018.

Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference on Learning Representations , 2018. URL https: //openreview.net/forum?id=H1VGkIxRZ .

Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. Detecting and correcting for label shift with black box predictors. In International conference on machine learning , pages 3122-3130. PMLR, 2018.

Thomas M Loughin. A systematic comparison of methods for combining p-values from independent tests. Computational statistics & data analysis , 47(3):467-485, 2004.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations , 2018. URL https://openreview.net/forum?id=rJzIBfZAb .

Christopher Manning and Hinrich Schutze. Foundations of statistical natural language processing . MIT press, 1999.

Frank J Massey Jr. The kolmogorov-smirnov test for goodness of fit. Journal of the American statistical Association , 46(253):68-78, 1951.

Zachary Nado, Neil Band, Mark Collier, Josip Djolonga, Michael W Dusenberry, Sebastian Farquhar, Qixuan Feng, Angelos Filos, Marton Havasi, Rodolphe Jenatton, et al. Uncertainty baselines: Benchmarks for uncertainty & robustness in deep learning. arXiv preprint arXiv:2106.04015 , 2021.

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, and Balaji Lakshminarayanan. Detecting out-ofdistribution inputs to deep generative models using typicality. arXiv preprint arXiv:1906.02994 , 2019.

Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, and Lawrence Carin. Variational autoencoder for deep learning of images, labels and captions. Advances in neural information processing systems , 29, 2016.

Stephan Rabanser, Stephan Günnemann, and Zachary Lipton. Failing loudly: An empirical study of methods for detecting dataset shift. Advances in Neural Information Processing Systems , 32, 2019.

Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distribution detection. Advances in neural information processing systems , 32, 2019.

David E Rumelhart, Geoffrey E Hinton, Ronald J Williams, et al. Learning internal representations by error propagation, 1985.

Takaya Saito and Marc Rehmsmeier. The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets. PloS one , 10(3):e0118432, 2015.

Chandramouli Shama Sastry and Sageev Oore. Detecting out-of-distribution examples with gram matrices. In International Conference on Machine Learning , pages 8491-8501. PMLR, 2020.

Robert J Serfling. Approximation theorems of mathematical statistics , volume 162. John Wiley & Sons, 2009.

Danica J. Sutherland, Hsiao-Yu Tung, Heiko Strathmann, Soumyajit De, Aaditya Ramdas, Alex Smola, and Arthur Gretton. Generative models and model criticism via optimized maximum mean discrepancy. In International Conference on Learning Representations , 2017. URL https://openreview.net/forum? id=HJWHIKqgl .

Pauli Virtanen, Ralf Gommers, Travis E Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, et al. Scipy 1.0: fundamental algorithms for scientific computing in python. Nature methods , 17(3):261-272, 2020.

Ross Wightman. Pytorch image models. https://github.com/rwightman/pytorch-image-models , 2019.

A Shift-Detection General Framework

The general framework for shift-detection can be found in the following figure, Figure 3.

X source

Two-Sample Test(s)

CombinedTestStatistic&

ShiftDetection

Dimensionality

Reduction

Xtarget

B Proofs

B.1 Proof for Theorem 4.2

Proof. Define

Thus,

Consider the i-th iteration of SGR over a detection-training set S m , and recall that, θ i = κ f ( x z ) , x z ∈ S m (see Algorithm 1). Therefore, θ i is a random variable (between zero and one), since it is a function of a random variable ( x ∈ S m ). Let Pr S m { θ i = θ ′ } be the probability that θ i = θ ′ .

Therefore,

Given that at the i -th iteration θ i = θ ′ , and considering that B θ ′ is derived using Lemma 4.1 (refer to Algorithm 1), we obtain,

The following application of the union bound completes the proof,

C Exploring Model Sensitivity: Evaluating Accuracy on Shifted Datasets

In this section, we present Table 3, which displays the accuracy (when applicable) as well as the degradation from the original accuracy over the ImageNet dataset, of the considered models on each of the simulated shifts mentioned in Section 6.1.1.

D Extended Empirical Results

In this section, we present a detailed analysis of our empirical findings on the ResNet50 architecture. We report the results for each window size, | W k |∈ { 10 , 20 , 50 , 100 , 200 , 500 , 1000 } , and for several shift cases discussed in Section 6.1.1. In particular, we show the detection performance of all the discussed methods, for the following shifts: FGSM (Table 4), ImageNet-O (Table 5), ImageNet-A (Table 6), and the Zoom out shift, 90% (Table 7).

Method

Window size

AUROC

↑

/ AUPR-In

↑

/ AUPR-Out

↑

/ DetectionError

↓

/ FPR@95TPR

↓

10

20

50

100

200

500

1000

KS

Softmax

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

Embeddings

98

/

98

/

98

/

7

/

14

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

MMD

Softmax

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

Embeddings

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

Single-instance

SR

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

Entropy

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

CBD (Ours)

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

100

/

100

/

100

/

0

/

0

E Ablation Study

In this section, we conduct multiple experiments to analyze the various components of our framework; all those experiments are conducted using a ResNet50. We explore several hyper-parameter choices, including C target , δ , and κ f . More specifically, we consider C target ∈ { 1 , 10 , 100 } , and δ ∈ { 0 . 1 , 0 . 01 , 0 . 001 , 0 . 0001 } , and two different CFs κ f , namely SR and Entropy-based.

To evaluate the performance of our detectors under varying hyper-parameters, we have selected a single metric that we believe to be the most important, namely, AUROC [13]. Additionally, since performance may vary depending on window size, we display the average AUROC across all window sizes that we have considered in our experiments. These window sizes include: { 10 , 20 , 50 , 100 , 200 , 500 , 1000 } . In Figure 4, we summarize our findings by displaying the average AUROC value as a function of the chosen hyper-parameters. These results are presented as heatmaps.

(a)

κ

f

= (1

-

Entropy

)

.

(b)

κ

f

=

SR.

0.92

0.91

0.90

0.90

0.89

Ctarget

Ctarget

10

0.88

10

0.88

0.86

0.87

100

100

0.86

0.1

0.01

0.001

0.0001

0.84

0.1

0.01

0.001

0.0001

6

0.85

Figure 4a, displays our AUROC detector's performance when we use Entropy-based as our CF. We observe that the optimal choice of hyper-parameters is δ = 0 . 01 and C target = 10 , resulting in the highest performance. However, increasing the value of C target leads to a more consistent and robust detector, as changes in the value of δ do not significantly affect the detector's performance. Additionally, we note that using C target = 1 yields relatively poor performance, indicating that a single coverage choice is insufficient to capture the characteristics of the distribution represented by the sample S m . Similar results are obtained when using SR as the CF, as shown in Figure 4b. These results suggest that selecting a high value of C target and a low value of δ is the most effective approach for ensuring a robust detector. Finally, the heatmaps demonstrate that Entropy-based CF outperforms (by a low margin) SR, in terms of detection performance.

F Upper Coverage Bound Development and Application

In this section, we discuss the complementary problem to that outlined in Section 4.1, which is solved by Algorithm 3; we also show its application for detecting distribution shifts, Algorithm 4. Finally, we introduce a unified algorithm, Algorithm 5, for detecting distribution shifts, while accounting for both under-coverage and over-coverage , details follow.

Problem statement. For a classifier f , a detection-training sample S m ∼ P m , a confidence parameter δ > 0 , and a desired coverage c ∗ > 0 , our goal is to use S m to find a θ value (which implies a selection function g θ ), such that the coverage satisfies,

This means that over coverage should occur with probability of at most δ .

The pseudo code of the algorithm that finds the optimal coverage upper bound (with confidence δ ) -SGC-UP , appears in Algorithm 3.

Algorithm 3: Selection with guaranteed coverage - Upper bound ( SGC-UP )

1 Input: train set: S m , confidence-rate function: κ f , confidence parameter δ , target coverage: c ∗ .

2 Sort S m according to κ f ( x i ) , x i ∈ S m (and now assume w.l.o.g. that indices reflect this ordering).

To deduce Equation (6) we make use of Lemma F.1, which is nearly identical to Lemma 4.1.

Lemma F.1. Let P be any distribution and consider a CF threshold θ with a coverage of c ( θ, P ) . Let 0 < δ < 1 be given and let ˆ c ( θ, S m ) be the empirical coverage w.r.t. the set S m , sampled i.i.d. from P . Let b ∗ ( m,m · ˆ c ( θ, S m ) , δ ) be the solution of the following equation:

The following is a uniform convergence theorem for SGC-UP (Algorithm 3) procedure stating that all the calculated bounds are valid simultaneously with a probability of at least 1 -δ .

Theorem F.2. Assume S m is sampled i.i.d. from P , and consider an application of Algorithm 3. For k = ⌈ log 2 m ⌉ , let b ∗ i ( m,m · ˆ c i ( θ i , S m ) , δ k ) and θ i be the values obtained in the i th iteration of Algorithm 3. Then,

The proof for Theorem F.2 can be easily deduced from the proof of Theorem 4.2, given in Appendix B.1.

F.1 Application of Upper Bound Algorithm for Detecting Distribution Shifts

In this section, we demonstrate the application of Algorithm 3 for detecting shifts in distribution. Specifically, Algorithm 3 is useful in identifying distribution shifts towards 'over-confidence,' where the model exhibits excessive confidence in its predictions on input data. It's important to note that this scenario was not empirically observed in our experiments, leading to the exclusion of this section from the main body of the paper.

Our 'over-confidence' detection algorithm is analogous to Algorithm 2. We start by applying SGC-UP to C target target coverages uniformly spread between the interval [0 . 1 , 1] , excluding the coverage of 1 . Each application j of SGC-UP on the same sample S m ∼ P m with a target coverage of c ∗ ; up j produces a pair: ( b ∗ ; up j , θ up j ), which represents a threshold and a bound, respectively.

Then,

Define, δ up (ˆ c | b ∗ ) = 1 [ˆ c ≥ b ∗ ] , thus, given a window of k samples from an alternative distribution, W k ∼ Q k , we define the sum of upper bound violations , V up , as follows,

where we obtain the last equality by using ˆ c up j ≜ ˆ c ( θ up j , W k ) = 1 k ∑ k i =1 g θ up j ( x i ) .

Similarly to V in Equation (4), the term V up denotes the magnitude of the violation, where V up = 0 indicates an absence of over-confidence in the predictions.

A pseudocode of our algorithm for detecting over-confidence in predictions appears in Algorithm 4.

Algorithm 4: Coverage-Based Detection - over confidence

F.2 Unified Algorithm for Distribution Shift Detection

In this section, we introduce a unified algorithm designed to detect shifts in distribution, accommodating both scenarios of over-confidence and under-confidence in predictions.

Define sum of violations (SOV),

A pseudocode of our unified algorithm for detecting both over and under confidence in predictions appears in Algorithm 5.

Algorithm 5: Coverage-Based Detection - Unified

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the topic of model monitoring through the lenses of input distribution shift detection during test time. They propose a distribution deviation detection method based on selective prediction coverage bound violation counting. They detect data distribution shift based on windows and studies the adaptation of single-instance confidence functions for this task. They compare to baselines such as the KS test and MMD method over the softmax or penultimate layer embeddings, and two single instances baselines based on the maximum softmax response and the output entropy with a t-test. The experimental benchmark is over ImageNet with popular models and shifts based on adversarial attacks, data corruption, and semantically different data.\n\nSTRENGTHS:\n- They present an original perspective on drift detection inspired from selective classification focused on real-world application.\n\n- The algorithm exhibits great time complexity, space complexity, and good performance.\n\n- Clean theoretical formulation of their algorithm, easy to understand.\n\n- Extensive experimental benchmark and comprehensive ablation study on the hyperparameters.\n\n- Overall, the paper is well-written and easy to follow.\n\nWEAKNESSES:\n- The choice of confidence-rate function might be a bottleneck of the method. Some models can exhibit over-confident predictions even to far OOD data. Relying on the softmax output as a confidence method might present a risk.\n\n- The formulation by counting the violations of the coverage bounds is interesting but it lacks motivation. It is hard to understand why computing the coverage bound would be more powerful than the t-tests with the confidence function values for each $x_k$.\n\n- Contribution (1) does not seem very useful. Why would someone desire to limit coverage without controlling risk?\n\n- In Table 1, the time complexities for MMD and KS might be inflated, as they could be improved by slightly changing the implementation. E.g., for MMD, the term with $O(dm^2)$ time complexity could be computed offline on a fit step.\n\n### Secondary observations\n\nIn line 375 the authors defined FPR@95TPR but following on in the paper they call it TNR@95TPR. Figure 4 would be easier to read if the AUROC numbers were written in the heatmap instead of the colorbars.\n\nQUESTIONS:\n1. The test set evaluation setup is unclear to me. The testing windows are pure out-of-distribution/corrupted data? What about mixed windows with in-distribution and ood data, which would be a more realistic setting. Have the author tried windows with a mixture of in-distribution and out-of-distribution data? I would strongly encourage adding an experiment in these lines to strengthen the experimental part of the paper.\n\n2. For the case of k=1000, is there a single window? It would mean that you compute the prediction for each method over the 15 seeds and compute the metrics in these predictions?\n\n3. Could the authors elaborate more on the claim that \"detection power might lay within lower coverages\" (line 362)? To my understanding, lower coverage simply gives a higher (more selective) threshold $\\theta$ for the CF, thus, it rejects more less confident samples.\n\n4. Could you explain in more details the novelty of Algorithm 1, Lemma 4.1 and Theorem 4.2 w.r.t references [11,12]?\n\n5. A more open ended less important question. Why do some baselines and your method decrease/oscillate performance with window size on some types of drift?\n\nLIMITATIONS:\nThe authors do not discuss the limitations of their work in great detail. A greater effort in this direction would be appreciated. For instance, on ImageNet-O, there appears that MMD Embeddings exibits better performance. So maybe the presented method is more appropriated to some types of drifts. Also, I would argue that a variation of 5% in AUROC on the ablation of $\\delta$ and $C_{target}$ shows some sensitivity to hyperparameters.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-29"}

comments: SUMMARY:
The paper proposes a window-based distribution shift detection method to detect whether the samples in the window are in-distribution (ID) or OOD. The method takes a window containing samples as input and computes a confidence score using the proposed coverage-based detection algorithm. The method is evaluated on various OOD benchmarks when imagenet is ID dataset. The results demonstrate that the proposed method is computationally more efficient than the existing methods and achieve high accuracies in majority cases.

STRENGTHS:
- The idea of detecting OOD samples within a window is interesting and in a different direction than the majority of OOD detection literature.
- The paper aims to present a theoretically supported and computationally efficient algorithm for this problem.

WEAKNESSES:
- Lack of intuition and convoluted description of the method: The intuition behind the proposed method is unclear. I didn't quite understand why this method should perform better than the existing ones. The theoretical part should be explained after giving some motivation for the method. 

- Lack of experiments: The paper presents results only when ImageNet is ID dataset. There should be experiments with different ID dataset to understand if the results generalize to other datasets.

- Some experimental details are not clear: It is unclear to me how windows are created in test time. Can a window contain both ID and OOD samples? Since the method labels the whole window as OOD or ID, I wonder how the method's performance changes with different ID and OOD sample ratios within a window.

QUESTIONS:
Please address my concerns in the weaknesses section.

LIMITATIONS:
Limitations of the method are not discussed. 
I don't see any potential negative societal impact of the work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a methodology to detect distribution shifts in windows (set) of observations for pre-trained deep learning models on computer vision tasks. The method compares the empirical coverage of a selection function to pre-determined computed bounds. During the fit stage, a guaranteed coverage selection algorithm determines C_target different bounds. During inference, the method computes the empirical coverage of the window to compute the total violations to the bounds. Finally, since V follows a normal distribution, a t-test statistic detects distribution shifts. The proposed method requires orders of magnitude less time and memory than the current SotA for large datasets. The paper presents a comprehensive comparison with SoTA methods in ImageNet.\n\nSTRENGTHS:\n- The paper focuses on identifying distribution shifts, a very important problem with multiple applications.\n- The methodology is appealing and very simple.\n- The proposed method is theoretically supported.\n- The results demonstrate the method is faster and requires less memory than the current SoTA.\n- The paper is well written, and the ideas are clearly motivated.\n- The experimental setting is comprehensive and well-designed.\n- Code to replicate results is provided.\n\nWEAKNESSES:\n- The proposed method performs better on smaller windows (k~50), which is contrary to the main claims of the paper that the proposed method can leverage population-level information.\n- One of the main applications to motivate the problem is determining the need to retrain pre-trained models. However, the size of windows (k) used in the experiments seems small for this application, especially since large models are potentially deployed in datasets with millions of observations.\n- The performance benefits of larger samples (k~1,000) are smaller than existing methods such as KS.\n- The paper only tests the simple method where V is an average of the C_target scalars.\n- Coverage is an established metric, and it could be directly used as a distribution-shift detector for the type of application discussed in the paper. It would be interesting to compare the method with a single coverage target value to better understand the benefits of the extension proposed in the paper. Appendix E presents this ablation study but is missing some important information. For instance, what value of C^* is used when C_target=1?\n- Limitations are not properly discussed. It seems that the accuracy advantages of the method are limited to the less useful setting where k is small.\n\nQUESTIONS:\n- Which target coverage is used in the appendix when C_target=1? \n- Have authors considered using other datasets than ImageNet?\n\nLIMITATIONS:\nThe paper does not properly discuss limitations (see above).", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a new method for detecting distribution shifts within a given window of samples. This coverage-based detection algorithm is theoretically motivated and can be applied to pre-trained models. The authors claim the low computational complexity of their framework, and experiments show that the framework outperforms the baselines across a variety of distribution shifts such as pixel-space adversarial examples.\n\nSTRENGTHS:\n1. The paper is well-written with a good storyline that formulates the proposed framework.\n2. The window setting is an attractive problem that has yet to be well explored by past works. The target models comprehensively include convolution-based and attention-based.\n3. The paper well defines the domain and assumptions of the task, making it clearly comparable to OOD frameworks.\n3. The authors motivate the framework by theoretical analysis, as well as conducting empirical experiments that demonstrate the effectiveness of the framework.\n\nWEAKNESSES:\n1. From the perspective of pixel-space adversarial attack detection, authors only evaluate the most basic baselines of FGSM and PGD. It will be constructive to the validation of the work if authors conduct additional experiments on the CW attack [1] and GAN perturbations [2].\n\n2. Authors intend to claim that the framework is effective to detect naturally (semantically) mutated images. However, the authors did not elaborate sufficient validations on the line of research for semantic adversarial examples [3]. It will be beneficial to the work if the authors have discussions on the effectiveness of the work for this line of research.\n\n3. The framework should be evaluated on scalable benchmarks proposed in recent years such as WILDS [4] V1/V2.\n\n[1] Towards Evaluating the Robustness of Neural Networks\n\n[2] Generating Adversarial Examples with Adversarial Networks\n\n[3] Semantic Adversarial Attacks: Parametric Transformations That Fool Deep Classifiers.\n\n[4] WILDS: A Benchmark of in-the-Wild Distribution Shifts\n\nQUESTIONS:\nPlease address my concerns and questions in the weakness section.\n\nTo summarize, this paper is a fair contribution to the OOD research. Although the paper is well-structured with a clear storyline, given several major concerns stated in the weakness section, I will rate it as borderline accept at this stage. I will keep track of authors' responses, and consider revising the rating based on the soundness of responses.\n\nLIMITATIONS:\nThe authors discussed potential improvements and future works in section 7.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA method to detect shifts in the input distribution of deep neural-networks is presented. The method takes a window of samples as input and calculates deviation from a tight coverage generalization bound. Here, coverage is empirically defined as the fraction of input samples that are non-rejected based on a confidence function, i.e. those which are predicted with sufficiently high confidence. The method is evaluated on standard image classification datasets using various commonly used CNN architectures.\n\nSTRENGTHS:\nThe method is well motivated and the detection metric ('sum bound violations') is defined and derived in a principled manner. Since the method is based on principles of selective classification, a by-product is a confidence threshold $\\theta$ and a corresponding selection function $g_\\theta(x)$, which are useful to have in practical situations. What is interesting is that the method does not detection a distribution-shift based on uncertainty or confidence of the input samples, but rather by the deviation of coverage from pre-calculated bounds at several target values based on these uncertainties. Similar to reference [12], the method does not rely on uncertainty values being calibrated.\n\nWEAKNESSES:\n1. The derivation of the main steps of the algorithm (Section 4.1) mirrors closely the derivation of the selection-based classifier in Section 3 of the paper cited as reference [12] in the manuscript. This isn't a weakness or limitation in my view, but it is worth noting the close conceptual similarities.\n\n2. What is concerning though is the rather weak experimental evaluation:\n\n    a. The authors claim that single-instance will not perform as well as their window-based method. However, the only single-instance baselines compared against are softmax response and entropy. These are rather weak baselines and there are several stronger baselines (several of which have been cited by the authors) that should have been compared against. \n\n    b. It would have been really interesting and useful to see this method applied with different types of uncertainty values apart from softmax (feature-based such as Mahalanobis, predictive uncertainty from Bayesian neural networks, etc.). That would have served as a much stronger evidence of its superiority over other methods such as KS and MMD. Admittedly, it's not possible to cover all aspects in one paper, but this would have been a rich addition in my view.\n\n    c. It is mentioned that 49000 out of 50000 samples in the imagenet validation set are used as detection training set. I am curious why use the validation data for this purpose. Why not select 49000 samples from the training set of 1 million+ images? 1000 samples to apply the shift to seems rather small. Can the authors please explain this?\n\n3. Somewhat confusing and overlapping notation. $k$ is used for $\\lceil \\log_2 m \\rceil$ in theorem 4.2 and also as size of window $W_k$. Would improve readability if different variables were used for these two different quantities.\n\nQUESTIONS:\nThere are questions of a few technical aspect which I'm hoping the authors can help clarify.\n\n1. It is mentioned that in reference 12, \"a risk generalization bound was developed for selective classifiers\" but that application of that would require labels, but that theirs doesn't. Given the duality between risk- and coverage-based analysis in the two papers (ref. 12 and this manuscript), can the authors provide some more details to support this statement  please?\n\n2. The bounds are calculated on a dataset of size $m$ while the test window is of a different size $k$. Should these not be matched? \n\n3. What value of $k$ is used for the visualization in Fig. 2? It would be good to understand the impact of the window-size $k$ on the curves for both the no-shift case and the shift case. Specifically, would the coverage bound and empirical coverage continue to remain nearly identical even when $k$ drops to a low value?\n\n4. Should the inequality in Step 9 of Algorithm 1 be $\\geq$ instead of $\\leq$?\n\nLIMITATIONS:\nNo societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors tackle the problem of out-of-distribution detection on a stream of instances. They introduce a method for coverage-based detection (CBD) that creates bounds and thresholds for monitoring by applying a newly derived selective with guaranteed coverage algorithm and then conducts a t-test on the violation sum between a window of newly received samples and a given detection dataset. The method is then analyzed for complexity and evaluated experimentally against ImageNet with various architectures. The method scales up with the size of the detection-training set size given more favourable complexity than previous baselines.\n\nSTRENGTHS:\n1. The case is made that at large scales of data the provided method is more practical for implementation and can match or outperform existing methods for detecting distribution shift on streams of data.\n\n2. The authors provide a wide range of data and analysis to back up their claims: \n2.1 Derivation of the SGC and CBD algorithms for estimating coverage and detecting violations.\n2.2 Lower complexity\n2.3 Experimental validation on ImageNet with various architectures and types of distribution shifts.\n2.4 Showing a walkthrough of bound and threshold identification which helps \t\n\n3. They provide a thorough explanation of their method and how it differs from existing baselines like KS and MMD based tests that are popular both in research and industry. Based on the results here I could easily see CBD replacing either of these methods in existing setups.\n\nWEAKNESSES:\n1. Experimental setup:Window-based methods in any task rely quite heavily on the dataset in question. It would have been great to see a more diverse set of benchmarks for out-of-distribution detection if they could be modified to the given problem statement. E.g. Wilds (Koh et. al., 2021) That being said I appreciate the various shifts applied to test on different kinds of distribution shifts.\n\n2. Presentation: I found Table 2 hard to parse and dense. I understand it is hard to fit in all the data but there could be some improvements.\n\n3.\n\nQUESTIONS:\n1. It seems that while the method scales with the size of the detection-training set, it converges with other methods as the window size increases (except with ViT-T architecture). Increasing the window size leads to a bigger jump in evaluations like Detection Error once the window size is above a certain threshold. What is the relationship between m, k with respect to out-of-distribution evaluation and is there any reason why someone might prefer to scale up the window size rather than the detection-training set and thus nullify the complexity advantages of this method?\n\nLIMITATIONS:\n1. Requiring access to the final layer of weights in the model rather than being fully black-box like some of the other methods. \n2. The authors don't highlight any particular limitations for practitioners to watch out for. Even suggestions around tuning the threshold parameters in different data settings could be helpful (or perhaps they are correct that the defaults they use are sufficient for all cases).", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-21"}

comments: SUMMARY:
This paper draws attention to the problem of window-based distribution shift detection and introduces a new method for it. The method is theoretically sound and demonstrated empirically to be effective on datasets with multiple types of shifts.

STRENGTHS:
- The proposed method is evaluated on extensive datasets with various types of shifts and outperforms selected baselines.
- The computational complexity is impressively low. 
- Due to its lower computational complexity, the proposed method is scalable compared to the baselines.

WEAKNESSES:
There aren't significant weaknesses in my opinion, but several minor ones might need clarification:
- I am concerned with the novelty of Algorithm 1, selection with guaranteed coverage, since most parts here are similar to SGRC in [1].  Is coverage the only modification here?
- The paper set out with a window-based shift detection task, which feels like a generalization from the single-instance task to me. However, there are no results of $|W_k|=1$. Can the proposed method still outperform baselines in this case? (Authors don't have to add new experiments here).
- The goal is to detect if there is a shift in a window $W_k$ of test data. I'd like to know how the proposed method will behave when only a portion of $W_k$ is from a different distribution.

[1] "Selective Classification for Deep Neural Networks", 2017

QUESTIONS:
- In equation 2, what does $|m\cdot \hat c(\theta, S_m)|$ intuitively stand for? Why not just add up to $m$?
- Is this typo? line 55: "read-world".

LIMITATIONS:
See the weaknesses above.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-25