MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Neural Networks
Authors: Charles Arnal
Felix Hensel
Mathieu Carrière
Théo Lacombe
Hiroaki Kurihara
Yuichi Ike
Frederic Chazal
Venue: Accepted by TMLR ()

Abstract: Despite their successful application to a variety of tasks, neural networks remain limited, like other machine learning methods, by their sensitivity to shifts in the data: their performance can be severely impacted by differences in distribution between the data on which they were trained and that on which they are deployed. In this article, we propose a new family of representations, called MAGDiff, that we extract from any given neural network classifier and that allows for efficient covariate data shift detection without the need to train a new model dedicated to this task. These representations are computed by comparing the activation graphs of the neural network for samples belonging to the training distribution and to the target distribution, and yield powerful data- and task-adapted statistics for the two-sample tests commonly used for data set shift detection. We demonstrate this empirically by measuring the statistical powers of two-sample Kolmogorov-Smirnov (KS) tests on several different data sets and shift types, and showing that our novel representations induce significant improvements over a state-of-the-art baseline relying on the network output.

Published in Transactions on Machine Learning Research (05/2024)
MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Neural Networks
Charles Arnal'
Inria Université Sacia-y
Felix Hensel"
Inria
Université Paris-Sackes Saris-Saclay
Mathieu Carrière
Inria
Université Côte d'A A'urr
Théo Lacombe
LIGM, Université Gustave Biffel
Hiroaki Kurihara
Pujitsu Limited
Yuichi Ike
Institute of Mathematics for Industry Kyushu University
Frédéric Chazal
Inria Université Paris-Son Paris-Saclay
Reviewed on OpenReview: https: // openreview. net/ met/ forum? id= kaHIK448qq
Abstract
Despite their successful application to a variety of tasks, neural networks remain limited, like other machine learning methods, by their sensitivity to shifts in the data: their perfor- mance can be severely impacted by differences distribution between the data on which they were trained and that which they deployed. In this article, we propose a new family of representations, called MAGDiff, that extract from any given neural network classifier and that allows for efficient covariate data shift detection without the need to train a new model dedicated to this task. These representations are computed by comparing the activation graphs of the neural network for samples belonging to the training distribution and to the target distribution, and yield powerful data- and task-adapted statistics for the two-sample tests commonly used for data set shift detoction. We demonstrate this empiri- cally by measuring the statistical powers of two-sample Kolmogorov-Smirnov (KS) tests several different data sets and shift types, and showing that our novel representations induce significant improvements over a state-of-the-art baseline relying the network output.
"These authors contributed equally this work.

---

Published in Transactions on Machine Learning Research (05/2024)
1 Introduction
During the last decade, neural networks (NN) have become immensely popular, reaching statc-of-the-art performances in wide range of situations. Nonetheless, once deployed in real-life settings, NN can face various challenges such as being subject to adversarial attacks (Huang et al.] 2017), being exposed to out- of-distributions samples (samples that were not presented at training time) (Hendrycks & Cimpel 2016), or more generally being exposed to a distribution shift: when the distribution of inputs differs from the training distribution (e.g., imput objects are exposed to a corruption due to deterioration of measure instruments such cameras or sensors). Such distribution shifts are likely to degrade performances of presumably well-trained models (Wiles et al., 2021b), and being able to detect such shifts is a key challenge in monitoring NN once deployed in real-life applications. Though shift detection for univariate variables is a well-studied problem, the task gets considerably harder with high-dimensional data, and seemingly reasonable methods often end up performing poorly (Ramdas et al. 2014).
In this work, we introduce the Mean Activation Graph Difference (MAGDiff), a new approach that harnesses the powerful dimensionality reduction capacity of deep neural networks in a data- and task-adapted way, The key idea, further detailed in Section 4 is to consider the activation graphs generated by inputs they are processed by a neural network that has already been trained for a classification lask, and to compare such graphs to those associated to samples from the training distribution. The method can thus be straightforwardly added as a diagnostic tool on top of precxisting classificrs without requiring any further training: it is easy to implement, and computationally inexpensive. As the activation graphs depend on the network weights, which turn have been trained for the data and task at hand, one can also hope for them to capture information that is most relevant to the context. Hence, our method can easily support, and benefit from, any improvements in deep learning.
Our approach is to be compared to Black box shift detection (BBSD), a method introduced in Lipton et al. (2018): Rabanser et al. (2019) that shares similar philosophy. BBSD uses the output of a trained classifier to efficiently detect various types of shifts (see also Section 4 in their experiments, BBSD generally beats other methods, the runner-up being a much more complex and computationally costly multivariate two-sample test combining an autocneoder and the Maximum Mcan Discrepancy statistic (Gretton et al. 2012).
Our contributions are summarized as follows.
1. Given any neural network classifier, we introduce a new family of representations MAGDiff, that is obtained by comparing the activation graphs of samples to the mean activation graph of each class in the training set.
2. We propose to use MAGDiff a statistic for data set shift detection. More precisely, combine our representations with the statistical method that was proposed and applied to the Confidence Vectors (CV) of classifiers in Lipton et al. (2018), yielding a new method for shift detection.
3. We experimentally show that our shift detection method with MAGDiff outperforms the state-of- the-art BBSD with CV on variety of datasets, covariate shift types and shift intensities, often by a wide margin. Our code is provided in the Supplementary Material and will be released publicly.
2 Related Work
Detecting changes outliers in data can be approached from the angle of anomaly detection, well-studied problem (Chandola et al.] 2009), or out-of-distribution (OOD) sample detection (Shafaei et al. 2018). Among techniques that directly frame the problem as shift detection, kernel-based methods such as Maximum Mcan Discrepancy (MMD) (Gretton et al. 2012 Zaremba el al. 2013) and Kernel Mean Matching (KMM) (Grelton et al. 2009 Zhang et al. 2013)) have proved popular, though they scale poorly with the dimensionality of the data Ramdas et al. 2014h. Using classifiers to test whether samples coming from two distributions can be correctly labeled, hence whether the distributions can be distinguished, has also been attempted; see, e.g.,
Kim et al. (2021). The specific cases of covariate shift (Jang et al.) 2022 Uehara et al. 2020 Rabanser al.
2

---

Published in Transactions on Machine Learning Research (05/2024)
Lipton et al. 2018 have been further investigated, from the point of view of causality and ant anticausality (Schölkopf et al. 2012) 2 Moreover, earlier investigations of similar questions have arisen from the fields of economics (Heckman 1977] and epidemiology (Saerens et al., 2002).
2019) and label shift (Storkey 2009
Among the works cited above, Lipton et al. (2018) and Rabanser et al. (2019) are of particular interest to us. In Lipton et al. (2018), the authors detect label shifts using shifts in the distribution of the outputs of a well-trained classifie;; they call this method Black Box Shift Detection (BBSD). In RRabanser et al.) (019)) the authors observe that BBSD tends to generalize very well to covariate shifts, though without the theoretical guarantees it enjoys in the label shift case. Our method is partially related to BBSD. Roughly summarized, we apply similar statistical tests-combinet univariate Kolmogorov-Smirnov tests-to different features Confidence Vectors (CV) in the case of BBSD, distances to mean activation graphs (MAGDiff) in ours. Similar statistical ideas have also been explored in Alberge et al. (2019) and SarrShalom et al. (2022)
while neural network activation graph features have been studied in, e.g., Lacombe et al. (2021) and Horta ct al. (2021). The related issue of the robustness of various algorithms to diverse types of shifts has been recently investigated in Wiles et. al. (2021a).
3 Background
3.1 Shift Detection with Two-Sample Tests
There can often be a shift between the distribution Po of data on which a model has been trained and tested and the distribution P1 of the data on which it is used after deployment; many factors can cause such a shift, e-g,, change in the environment, in the data acquisition process, or the training set being unrepresentative. Detecting shifts is crucial to understanding, and possibly correcting, potential losses in performance; even shifts that do not strongly impact accuracy can be important symptoms of inaccurate assumptions or changes in deployment conditions.
Additional assumptions can sometimes be made on the nature of the shift. In the context of a classification task, where data points are of the shape (2.y) with x the fcature vector and y the label, a shift that preserves the conditional distribution p(zly) (but allows the proportion of each label to vary) is called label shift. Conversely, covariate shift occurs when p(y]æ)) is preserved, but the distribution of p(x) is allowed to change. In this article, we focus the arguably harder case of covariate shifts. See Section 5 for examples of such shifts in numerical experiments.
Shifts can be detected using two-sample tests: that is, a statistical test that aims at deciding between the two hypotheses
Ho: Po = P, and H1 : Po o Pı,
given two random sets of samples, Xo and X1, independently drawn from two distributions Po and P1 (see, e.g., Heumann & Schomaker 20033) for an introduction to hypothesis testing). To do so, many statistics have been derived, depending on the assumptions made on Po and P,. In the case of distributions supported on R. one such test is the univoriate Kolmogorov-Smérnon (KS) test, of which we make use in this article. Given, as above, two sets of samples Xo, X1 C R, consider the empirical distribution functions F.(z) == Card(X.) L Leg= for i = 0, and E I, where Card denotes the cardinality. Then the statistic associated with the KS test and the samples is T == supseR [Fo(z) - (=)). If Po Pi, the distribution of T is independent of Po and converges to a known distribution when the sizes of the samples tend to infinity (under mild assumptions) (Smirnov] 1939 Hence approximate p-values
be derived. The KS test can RR,", p-value p. can be
also be used to compare multivariate distributions: if IPo and IP; are distributions computed from the samples by comparing the i-th entrics of the vectors of Xo. X1 C IDD using the univariate KS test, for i = 1....,.. D. A standard and conservative way of combining those p-values is to reject Ho if min(p1...... PD) < a/D, where a is the significance level of the test. This is known as the Bonferront correction (Voss& & George 1995). Other tests tackle the multidimensionality of the problem more dircctly, such as the Marimum Meon Discrepancy (MMD) test, though not necessarily with greater success (sce, e.g..
Ramdas et al. (2014)).
3

---

Published in Transactions on Machine Learning Research (05/2024)
3.2 Neural Networks
We now recall the basics of neural networks (NN), which will be our main object of study. We define a neural network as (finite) sequence of functions called layers fi,...: fu of the form fe: R - R* , + oe x + be), where the parameters We E R= and by E [R are called the weight matrix and the bias vector respectively, and de is an (element-wise) activation map (e.g., sigmoid or ReLU). The neural network encodes a map F: IR + RP given by F=fz .... 0.. f1- We sometimes use F to refer to the neural
network as a whole, though it has more structure.
When the neural network is used as classifier, the last activation function or is often taken to be the softmaz function, so that Fx)); can be interpreted as the confidence that the network has in a belonging to the i-th class, for i = 1...., D. For this reason, use the terminology confidence vector (CV) for the output F(x) E RR. The true class of x is represented by label y (O....,0. 1, 0...0..... ..... 0) E TR that takes value 1 at the coordinate indicating the correct class and 0 elsewhere. The parameters of each layer (We, by) are typically learned from a collection of training observations and labels {(z,. yn) by minimizing a cross- entropy loss through gradient descent, in order to make F(Z,)) as close to yn as possible on average over the training set. The prediction of the network on new observation x then given by arg max;= F(z):. and its (lest) accuracy is the proportion of correct predictions on a new set of observations {(1r,, ) that is assumed to have been independently drawn from the same distribution as the training observations. In this work, we consider NN classifiers that have already been trained on some training data and that achieve reasonable accuracies on test data following the same distribution as training data.
3.3 Activation Graphs
Given an instance x Fo E Rd, a trained neural network fi..... f with Te+1 = fé(xe) = (We + by) and a layer Se: Rn - Re++, we can define a weighted graph, called the activation graph Gç(z) of x for the layer fe, as follows. We let V = Ve+ be the disjoint union of the two sets V = {L.... .......,)) n2) and Ve+1 = {I,..... ... n. The edges are defined as = Ve x V441- To each edge (i.j) E Eg, we associate the weight (x)(2)) = ie(j. ze(i), where xx(i) (resp. ie(i.i) denotes the i-th coordinate of age E E (resp. entry (i, of Wt E Reeesxme). ). The activation graph (z(z) is the weighted graph (V, F, {=)r(=))), which be conveniently represented as ny ne++1 matrix whose entry (i.j) is (x). A simple illustration of this definition can be found in the Supplementary Material. Intuitively, these activation graphs fraphs-first considered in Gebhart et al. (2019) represent how the network "reacts" to a given observation x at inner-level, rather than only considering the network output (i.e,, the Confidence Vector).
4 Two-Sample Statistical Tests using MAGDiff
4.1 The MAGDiff representations
Let Pu and P1 be two distributions for which we want to test Ho Pu PI. As mentioned above, two-sample statistical tests tend to underperform when used directly on high-dimensional data. It is thus common practice to extract lower-dimensional representations (z)) from the data x ~ P,, where v: supp Po U supp IP, + AM, Given a classification task with classes I,...,,, D, we define a family of such representations as follows. Let T; supp Po U supp P1- + V be any map whose codomain V a Banach space with norm || llv. For each class E (1..... D), let Po,, be the conditional distribution of data points from Po in class i. We define
(x (Tx)
E7,, [T,,'
While our exposition is restricted to fully-connected feedforward neural networks for the sake of concision, our representa- tions are well-defined for other types of neural nets (e.g., recurrent neural nets). In particular, they adapt seamlessly to the case of convolutional layers: such a layer have many zeroes, and what follows applics without further modifications.
always be represented
fully-connected layer whose weight matrix is constrained to
"Here, in the remainder of the article, we commit minor abuse of notation: Po and P1 are distributions on both the features and the labels, t.e. (x, 1) ~ P;, but often write e P, indicate that x has heen drawn from (Pt)a, the marginal of F, with respect to the features. To avoid any confusion, we always let. the letter (possibly with subscript) indicate features.
4

---

Published in Transactions on Machine Learning Research (05/2024)
for x E supp PoUsupp Pt. Given a fixed finite dataset Fi,..., lid
Fo, we similarly define the approximation
my
-0=)=
m Σ
llx
where 2i...., Fin, are the points whose class is i. This defines a map v supp Po Suupp Pi + RD.
The map T: supp Po U supp P1 + V could priori take many shapes. In this article, we assume that we are provided with a neural network F that has been trained for the classifying task at hand, as well as a training set drawn from Fg. We let T be the activation graph G₃ of the layer fi of represented as a matrix, so that the expected values (Ge(")(( (for i = 1...., D) are simply mean matrices, and the norm || Ilv is the Frobenius norm || ll2. We call the resulting D-dimensional representation Meon Activation Graph Difference (MAGDiff):
MAGDiff(z); = (G)(z) 1 Σ Ge(z))ll.
for i = -,..., D, where of zi:-.., Tm, are, above, samples of the training set whose class is i. Therefore, for given new observation x, we derive a vector MAGDiff(1) E RD whose i-th coordinate indicates whether æ activates the chosen layer of the network in similar way "as training observations of the class i".
Many variations are possible within that framework. One could, e.g., consider the activation graph of several consecutive layers, use another matrix norm, or apply Topological Data Analysis techniques to compute a more compact representation of the graphs, such as the topological uncertainty (Lacombe al., 2021). In this work, we focus on MAGDiff for dense layers, though it could be extended to other types.
4.2 Comparison of distributions of features with multiple KS tests
Given as above a (relatively low-dimensional) representation v: supp P. supp P1 - RN and samples iid sid Po and si,,..,,%m, ~ F1, one can apply multiple univariate (coordinate- wise) KS tests with In
Bonferroni correction to the sets (x1)..... (zz,) and (zf)..... (a'a))) as described in Section 3 If is well-chosen, difference between the distributions PP and PI (hard to test directly due to the dimensionality of the data) will translate to a difference between the distributions (zx) and (xx') for respectively. Delecting such difference serves as proxy for lesting Hn Po = Pi. In our experiments, we apply this procedure to the MAGDiff representations defined above (see Section 5.1 for a step-by-step description). This is a reasonable approach, as it is a simple fact that a generic shift in the distribution of the random variable æ ~ Po will in turn induce a shift in the distribution of (xx), as long as is not c however, this does not give us any true guarantee, as it does not provide any quantitative result regarding the shift in the distribution of (x). Such results are beyond the scope of this paper, in which we focus on the good experimental performance of the MAGDiff statistic.
Po and x' ~ IP
4.3 Differences from BBSD and motivations
The BBSD method described in Lipton el al. (2018) and Rabanser et al. (2019) is defined a similar manner, except that the representations V on which the multiple univariate KS tests are applied are simply the Confidence Vectors (CV) F(x) RD of the neural network F (or of any other classifier that outputs confidence vectors), rather than newly proposed MAGDiff representations. In other words, they detect shifts in the distribution of the inputs by testing for shifts in the distribution of the outputs of given classiffer
Both our method and theirs share advantages: the features are task- and data-driven, as they are derived from a classifier that was trained for the specific task at hand. They do not require the design or the training of an additional model specifically geared towards shift detection, and they have favorable algorithmic complexity, especially compared to some kernel-based methods. In particular, combining the KS tests with the Bonferroni
S the Supplementary Material, Scction 3 for elementary proof.
4This corresponds to the best-performing variant their method, denoted BBSDs (as opposed to, e.g., BBSDK) in
Rabanser et al. (2019).
5

---

Published in Transactions on Machine Learning Research (05/2024)
correction spares from having to calibrate statistical tests with permutation test, which can be costly as shown in Rabanser et. al. (2019). A common downside is that the Bonferroni correction can be overly conservative; other tests might offer higher power. The main focus of this article is the relevance of the MAGDiff representations, rather than the statistical tests that we apply to them, and it has been shown in Rabanser et al. (2019) that KS lests yield state-of-the-arl performances; as such, we did not investigate alternatives, though additional efforts in that direction might produce even better results.
The nature of the construction of the MAGDiff representations is geared towards shift detection since it is directly based on encoding differences (i.e,, deviations) from the mean activation graphs (of Po). Moreover, they are based on representations from deeper within the NN, which are less compressod than the CV passing through each layer leads to a potential loss of information. Hence, we can hope for the MAGDiff to encode more information from the input data than the CV representations used in Rabanser et al. (2019) which focus on the class to which a sample belongs to, while sharing the same favorable dimensionality reduction properties. Therefore, we expect MAGDiff to perform particularly well with covariate shifts, where shifts in the distribution of the data do not necessarily translate to strong shifts in the distribution of the CV. Conversely, we do not hope for specific case of label shifts; all the information relative to labels available to the network is, in sense, best summarized in the CV, as this is the main task of the NN. These expectations were confirmed in our experiments.
representations to bring significant improvements over CV in the
5 Experiments
This experimental soction devoted to showcasing the use of the MAGDiff representations and its benefits over the well-cstablished bascline CV when it comes to performing covariate shift detection. As detailed in Section 5.1 combine coordinate-wise KS tests for both these representations. Note that in the case of CV, this corresponds exactly to the method termed BBSDs in Rabanser et al. (2019). Our code is provided in the Supplementary Material, as well as a more thorough presentation of the datasots and parameters used.
5.1 Experimental Settings
Datasets. We consider the standard datasets MNIST (LeCun et al. 1998), MNShT (FMNIST)
(Xiao et al. 2017), CIFAR-10 (Krizhevsky & Hinton 2009 SVHN (eezerr et al. 2011). as well as a lighter
version of ImageNet (restricted to 10 classes) called Imagenette [maa 2023).
Architectures. For MNIST and FMNIST, we used simple CNN architecture consisting of 3 convolutional layers followed by 4 dense layers. For CIFAR-10 and SVHN, we considered (a slight modification, to account for input images of size 32 32, of) the ResNet18 architecture (He et al.) 2015). For Imagenette, we used pretrained ResNet18 model provided by Pytorch (Res) 2018). With these architectures, we reached a test accuracy of 98.6% MNIST, 91.1% on FMNIST, 94.1% SVHN, 81% on CIFAR-10 and 99.2% for Imagenette, validating the "well-trained" assumption mentioned in Section Note that we used simple architectures, without requiring the net works to achieve state-of-the-ard accuracy.
Shifts. We applied three types of shift to our datasets: Gaussian noise (additive white noise), Gaussian blur (convolution by a Gaussian distribution). and Image shift (random combination of rotation, translation, zoom and shear), for six different levels of increasing intensities (denoted by I, II,..., VI), and fraction of shifted data 8 E {0.25, 1.0). For each dataset and shift type, we chose the shift intensities in such a manner that the shift. detection for the lowest intensitics and low 8 is almost indetectable for both methods (MAGDiff and CV), and very casily detectable for high intensities and values of 8. Details (including the impact of the shifts on model accuracy) and illustrations can be found in the Supplementary Material.
6

---

Published in Transactions on Machine Learning Research (05/2024)
Sample size. We ran the shift detection tests with sample size: {10, 20, 50, 100, 200, 500, 1000} to assess how many samples a given method requires to reliably detect a distribution shift. A good method should be able to detect a shift with as few samples as possible.
Experimental protocol. In all of the experiments below, we start with a neural network that is pre- trained on the training set of a given datasct. The test set will be referred to as the clean set (CS). We then apply the selected shift (lype, intensity, and proportion 5) to the clean set and call the resulting set the shifted set SS; it represents the target distribution P1 in the case where P1 o Fo.
As explained in Section 4 for cach of the classes i == 1...., D (for all of our datasets, D == 10), we compute the mean aclivation graph of chosen dense layer Se of (a random subset of size 1000 of all) samples in the training set whose class is i; this yields D mean activation graphs GI...., Gp. We compute for each sample x in CS and each sample in SS the representation MAGDiff(#), where MAGDiff(x); = || - Gilla for = D and Gz(z) is the activation graph of x for the layer fe (as explained in Section 4 Doing so, we obtain two sets (MAGDiff(1) (1: ar E CS} and (MAGDiff(z") | x' E SS} of D-dimensional features with the same cardinality as the test set.
Now, we estimate the power of the test for given sample size m and for type I error of at most 0.05; in other words, the probability that the test rejects Ho when H1 is true and when it has access to only m samples from the respective datasets, and under the constraint that it does not falsely reject Hy in more than 5% of cases. To do so, we randomly sample (with replacement) m elements "',..., ''m from SS, and consider for each class = I....,D D the discrete empirical univariate distribution qi of the values MAGDiff (z1)..... ,,,i)k....,MM.iEEE.. MAGDiff(r'm)) Similarly, by randomly sampling m elements from CS, we obtain another discrete univariate distribution pi (see Figure | for an illustration). Then, for each i I,..., D, the KS test is used to compare P; and qi to obtain a p-value X; and reject Ho if min(A),....AAa < a/D, where is the threshold for the univariate KS test at confidence 0.05 (ef. Section 3.1.. Following standard bootstrapping protocol, we repeat that experiment (independently sampling m points from CS and SS, computing p- values, and possibly rejecting Ho) 1500 times; the percentage of rejection of Ho is the estimated power of the statistical test (since Ho is false in this scenario). We use the asymptotic normal distribution of the standard Central Limit Theorem to compute approximate 95%-confidence intervals on our estimate.
Clean set
Shifted set
600
500
400
300
200
100
200
400
200
300
MAGDiff,
MAGDiff,
Figure 1: Empirical distributions of MAGDiff; for the 10, 000 samples of the clean and shifted sets (MNIST, Gaussian noise, 5 = 0.5, last dense layer). For the clean set, the distribution of the component MAGDiff, of MAGDiff exhibits peak close to O. This corresponds to those samples whose distance to the mean activation graph of (training) samples belonging to the associated class is very small, i.e., these are samples that presumably belong to the same class as well. Note that, for the shifted set, this peak close to 0 is substantially diminished, which indicates that the activation graph of samples affected by the shift is no longer as close to the mean activation graph of their true class.
To illustrate that the lest is well calibrated, we repeat the same procedure while sampling twice m elements from CS (rather than m elements from SS and m elements from CS). which allows us to estimate the type I error (i.e., the percentage of incorrect rejections of Ho) and assert that it remains below the significance level of 5% (see, e.g., Figure
TThat is, the number of elements from the clean and shifted sets which the statistical tests are performed; see the paragraph Experimental protocol for more details. 6Te same sample that mentioned the Sample size paragraph.
7

---

Published in Transactions on Machine Learning Research (05/2024)
Power Function of Sample Size
MNIST: Gaussian noise, shift intensity: II
MNIST: Gaussian noise, shift intensity: IV
MNIST: Gaussian noise, shift intensity: Vi
1.0
1.0
1.0
power estimation
0.8
power estimation
0.8
power estimation
0.8
0.6
power MAGDIFf
etror CV
0.4
0.4
0.4
Type eroo MAGDIff
0.2
0.2
0.2
10a
10 sample size
10
10a sample size
10a
10 sample size
Figure 2: Power and type error of the statistical test with MAGDiff (red) and CV (green) representations w.r.t. sample size (on a log-scale) for three different shift intensities (II, IV, VT) and fixed á = 0.5 for the MNIST dataset, Gaussian noise and last layer of the network, with estimated 95%-confidence intervals.
We experimented with a few variants of the MAGDiff representations: we tried reordering the coordinates of each vector MAGDiff(x) E IR in increasing order of the value of the associated confidence vectors. We also tried replacing the matrix norm of the difference to the mean activation graph by either its Topological Uncertainty (TU) (Lacombe et al., 2021), or variants thereof. Early analysis suggested that these variations did not bring increased performances, despite their increased complexity. Experiments also suggested that MAGDiff representations brought no improvement over CV in the case of label shift. We also tried to combine (i.e., concatenate) the CV and MAGDiff representations, but the results were unimpressive, which we attribute the Bonferroni correction getting more conservative the higher the dimension. We thus only report the results for the standard MAGDiff.
Competitor. We used multiple univariate KS tests applied to CV (the method BBSDs from Rabanser ct al. 2019)) as the baseline, which we denote by "CV" in the figures and tables, in contrast to our method denoted by "MAGDiff". The similarity in the statistical testing between BBSDs and MAGDiff allows us to easily assess the relevance of the MAGDiff features. We chose them as our sole competitors as it has been convincingly shown Rabanser al. (2019) that they outperform on average all other standard methods, including the use of dedicated dimensionality reduction models, such as autoencoders, or of multivariate kernel tests. Many of these methods are also either computationally more costly (to the point where they cannot be practically applied to more than thousand samples) or harder to implement (as they require an additional neural network to be implemented) than both BBSDs and MAGDiff.
5.2 Experimental results and influence of parameters.
We showcase the power of shift detection using our MAGDiff representations in various settings and compare it to the state-of-the-art competitor CV. Since there were large number of hypee-rarameters in our experiments (datasets, shift types, shift intensities, etc.), we started with a standard set of hyper- parameters that yielded representative and informative results according to our observations (MNIST and Gaussian noise, as in Rabanser et al. (2019), 8 = 0.5, sample size 100, MAGDiff computed with the last layer of the network) and let some them vary the successive experiments. We focus on the well-known MNIST dataset allow for easy comparisons, and refer the Supplementary Material for additional experimental results that confirm our findings on other datasets.
Sample size. The first experiment consists of estimating the power of the shift detection test function of the sample size (a common way of measuring the performance of such test) using either the MAGDiff or the baseline CV representations. Figure 2 shows the powers of the KS tests using the MAGDiff (red curve) and CV (green curve) representations with respect to the sample size for the MNIST dataset. Here, we choose to showcase the results for Gaussian noise of intensities 11, IV and IV with shift proportion = 0.5.
It can clearly be seen that MAGDiff consistently and significantly outperformed the CV representations. While in both cases, the tests achieved power of 1.0 for large sample sizes (m 1000) and/or high shift
8

---

Published in Transactions on Machine Learning Research (05/2024)
intensity (VI), MAGDiff was capable of detecting the shift even with much lower sample sizes. This particularly striking for the low intensity level IT, where the test with CV was completely unable to detect the shift, even with the largest sample size, while MAGDiff was capable of reaching non-trivial power already for a medium sample size of 100 and exceptional power for large sample size. Note that the tests were always well-calibrated. That is, the lype error remained below the significance level of 0.05, indicated by the horizontal dashed black line the figures.
To further support our claim that MAGDiff outperforms CV average in other scenarios, provide, in Table [ averaged results over all parameters except the sample size. Though the precise values obtained are not particularly informative (due to the aggregation over very different sets of hyper-parameters), the comparison between the two rows remains relevant. In the Supplementary Material, a more comprehensive experimental report (including, in particular, the CIFAR-10 and Imagenette datasets) further supports our claims.
Averaged
power (%)
Sample size
10
20
50
100
200
500
1000
MAGDiff
7.4
17.1
27.6
40.7
54.7
71.4
80.4
CV
4.0
9.8
15.6
24.7
35.3
49.7
59.2
Table 1: Averaged test power of MAGDiff and CV over all hyper-parameters except sample size (dataset, shift type, 6, shift intensity). A 95%-confidence interval for the averaged powers has been estimated via bootstrapping and is, in all cases, strictly contained =0.1% interval.
Impact of Shift Intensity
MNIST: Gaussian noise, sample size: 100
MNIST: Gaussian noise, sample size: 100
MNIST: Gaussian noise, sample size: 100
1.0
power
1.0
1.0
power MAGDIfT
Cower estimatioon
0.8
Type eeror CV
power estimation
power estimation
0.8
Type error MAGDIM
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
shift intensity
shift intensity
shift intensity
Figure 3: Power and type error of the test with MAGDiff (red) and CV (green) features w.r.t. the shift intensity for Gaussian noise on the MNIST dataset with sample size 100 and d= 0.25 (left), ò == 0.5 (middle), 5 1.0 (right), for the last dense layer. The estimated 95%-connidencc intervals displayed around the curves.
Shift intensity. The first experiment suggests that MAGDiff representations perform particularly well when the shift is hard to detect. In the second experiment, we further investigate the influence of the shift intensity level and (which is, in sense, another measure of shift intensity) on the power of the tests. We chose fixed sample size of 100, which was shown to make for challenging yet doable task. The results in Figure X confirm that reprosentations were much more sensitive to weak shifts than the CV, with differences in power greater than 80% for some intensities.
Shift typc. The previous experiments focused on the case of Gaussian noise; in this experiment, we investigate whether the results hold for other shift lypes. As detailed in Table 2 the test. with MAGDiff representations reacted to the shifts even for low intensities of I, I1, and III for all shift types (Gaussian blur being the most difficult case), while the KS test with CV was unable to detect anything. For modium high intensities III, IV, V and VI, MAGDiff again significantly outperformed the bascline and reaches powers close to 1 for all shift types. For the Gaussian blur, the shift remained practically undetectable using CV.
9

---

Published in Transactions on Machine Learning Research (05/2024)
Impact of Shift Type
Power of the test (%)
Shift intensity
Shift
Feat.
11
III
IV
V
VI
MD
7.2 1.3
29.3 2.3
61.9 2.5
93.3 1.3
98.9
100.0 0.0
GN
CV
0.0
0.1
1.5 0.6
17.6 - 1.9
72.3
98.9 0.5
MD
3.0
1.0
2.3
85.0 2..1
92.4 1.3
GB
CV
0.0
0.0
0.0
0.3
1.3 0.6
5.3 1.1
32.7 1.5
2.5
78.5
90.6 = 1.5
98.9 = 0.5
IS
CV
0.0
0.1 + 0.2
2.7
11.5 1.6
37.0
2.4
86.3 + 1.7
Table Power of the two methods (our method, denoted as MD, and CV) as function of the shift intensity for the shift types Gaussian noise (GN), Gaussian blur (GR) and Image shift (TS) on the MNIST dataset with = 0.5, sample size 100, for the last dense layer. Red indicates that the estimated power is below 10%, blue that above 50%. The 95%fccnfeeeece intervals have been estimated as mentioned in Section
MAGDIff with respect to different layers. The NN ar- chitecture we used with MNIST and FMNIST had several dense layers before the output. As a variation of our method, we investigate the effect on the shift detection when com- puting our MAGDiff representations with respect to different layer More precisely, we consider the last three dense lay- ers denoted by f-1, 8_2 and f-3, ordered from the closest to the network output (f-1) to the third from the end (L-s)- The averaged results over all parameters and noise types are in Table 3 In the case of MNIST we only observe a slight increase in power when considering layer l-3 further from the output of the NN. In the case of FMNIST, on the other hand, clearly see much more pronounced improvement when switching from &-1 to l-3. This hints at the possibility that features derived from encodings further within the NN can, in some cases, be more pertinent to the task of shift detection than those closer to the output.
Choice of Layer Averaged power (%)
Features
Dataset
CV
{-1
f-2
t-3
MNIST 25.1
51.9
53.0
56.4
FMNIST 46.2
44.9
47.6
53.7
Table 3: Averaged performance of the various layers for MAGDiff over all other parameters (for MNIST and FMNIST), compared to BBSD) with CV. 95% confidence interval for the averaged powers was estimated and is in all cases contained in a #0.1% interval.
6 Conclusion
In this article, derive new representations MAGDiff from the activation graphs of trained NN classifier. We empirically show that using MAGDiff representations for data set shift detection via coordinate-wise KS tests (with Bonferroni correction) significantly perperooms the baseline given by using confidence vectors established in Lipton et al. (2018). while remaining equally fast and easy to implement, making MACDiff rep- resentations an efficient tool for this critical task.
Our findings open many avenues for future investigations. We focused on classification of image data in this work, but method is general one and can be applied to other settings. Moreover, adapting our method to regression tasks well as to settings where shifts occur gradually is feasible and starting point for future work. Finally, exploring variants of the MAGDiff representations-conn considrrinniinnnnnncccooo several layers of the network at once, extending it to other types of layers, extracting finer topological information from the activation graphs, weighting the edges of the graph by backpropagating their contribution to the output, etc. etc.-could also result. increased performance.
Since 18 only has single dense layer after its convolutional layers, there choice made the case of CIFAR-10, SVHN and Imagenette.
10

---

Published in Transactions on Machine Learning Research (05/2024)
References
Pre-trained weights for resnet https://pytorch.. ctttas//pytorrd..oo//vvstsooooos http:/c/hc.asaaaa//////////ataaa
ressell..ssnet11... 2018. Accessed: 10/05/2023.
Imagenette dataset. httpss//ggtabb.ccmmmmaaaaaaaaagt 2023. Accessed: 10/05/2023.
Florence Alberge, Clément Feutry, Pierre Duhamel, and Pablo Piantanida. Detecting covariate shift with Black Box predictors. In International Conference Telecommuniications (ICT 2019), Hanoi, Vietnam, April 2019. doi: fr/hal-02172275
Guy Dar-Shalom, Yonatan Geifman, and Ran El- Yaniv. Distribution shift detection for deep neural networks, 2022.
10897
Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: A survey. ACM Comput. Surv., 41, 07 2009. doi: 10.1145/5511880154415558
Thomas Gebhart, Paul Schrater, and Alan Hylton. Characterizing the shape of activation space in deep neural networks. In 2019 18th IEFE International Conference On Machine Learning And Applications (ICMLA), pp. 1537 1542. IEEE, 2019.
Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, and Bernhard Schölkopf. Covariate shift by kernel mean matching. Dataset shift in machine learning, 3(4):5, 2009.
Arthur Gretton, Karsten M. Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723 773, 2012. URL httpp:/mmmr..or org/ html
(appers/v33/reetonnit
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016
IEEE Conference on Compuler Vision and Pallern Recogrition (CVPR), pp. 770 778, 2015.
James Heckman. Sample selection bias as a specification crror (with an application to the estimation of labor supply functions). National Bureau of Economic Research, Inc, NBER Working Papers, 01 1977.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint orXiv:1610.02/ 021106,,.11 2016.
C. Heumann and M. Schomaker. Introduction to Statistics and Data Analysis: With Exercises, Solutions and Applications in Springer International Publishing, 2023. ISBN 978303111888 URL https:
7/books. //books.google... www.goooog.......o.o...ooo fttggle...rboooss1ddd.e. wtw.://ooooss.dd-sssga.....
Vitor A.C. Horta, Ilaria Tiddi, Suzanne Little, and Alessandra Mileo. Extracting knowledge from deep neural networks through graph analysis. Future Generation Computer Systems, 120: 118, 2021. ISSN 0167-739X. doi: https::/doo..rrg/ wtw.://///.........iiio. http://uuuurrr222222222.0 URL http:::/ww..cceenedreeeee com/ acierceeArricceee///////71111111
Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, and Pieter Abboel. Adversarial attacks on neural network policies. arXiv preprint 1xXie:17020020 02284, 2017.
Sooyong Jang, Sangdon Park, Insup Lee, and Osbert Bastani. Sequential covariate shift detection using classifier two-sample tests. Tn Kamalika Chaudhuri, Stefanie Jegelka, Te Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 9845-9880. PMLR, 17-23 Jul 2022. URL https: httpp:///rron //pppeedd//ccc wtt.:gs.ww.....se.eeeeeeeee wwtpppesssee/eeppeepepppppp http://jann2aaahthh html
Gopinath Kallianpur. The topology of weak convergence of probability measures. Journal of Mathematics and Mechanics, pp. 947-969, 1961.
Immn Kim, Aaditya Ramdas, Aarti Singh, and Larry Wasserman. Classification accuracy as a proxy for two-sample testing. The Annals of Statistics, 19(1):411-44 -434, 2021.
11

---

Published in Transactions on Machine Learning Research (05/2024)
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical Report 0, University of Toronto, Toronto, Ontario, 2009.
Théo Lacombe, Yuichi Ike, Mathieu Carrière, Frédéric Chazal, Marc Glisse, and Yuhei Umeda. Topological Uncertainty: monitoring trained neural networks through persistence of activation graphs. In 30th In- ternational Joint Conference on Artificial Intelligence (IJCAI 2021), pp. 2666 2672. International Joint Conferences on Artificial Intelligence Organization, 2021.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-base learning applied to docu- ment recognition. Proc. IEEE, 06:2278-2324, 1998.
Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. Detecting and correcting for label shift with black box predictors. In 35th International Conference on Machine Learning (ICML 2018), volume 80, pp. 3122 3130. PMLR, 2018.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Un- supervised Feature Learning 2011, 2011. URL http://ufldl sttt:// edt/hhousemuumborrrrrp222011 housenumbers, pdf
stanford
Stephan Rabanser, Stephan Günnemann, and Zachary Lipton. Failing loudly: An empirical study of methods for detecting dataset shift. In Advances in Neural Information Processing Systems 39 (NeurIPS 2019), pp. 1396 1408. Curran Associates, Inc., 2019.
Aaditya Ramdas, Sashank J. Reddi, Barnabás Póczos, Aarti Singh, and Larry A. Wasserman. On the decreasing power of kernel and distance based nonparametrie hypothesis lests in high dimensions. In AAAI Conference on Artificial Intelligence, 2014.
Marco Saerens, Patrice Latinne, and Christine Decaestecker. Adjusting the outputs of a classifier to new priori probabilities: a simple procedure. Neural computation, 14(1):21-41, 2002.
Bernhard Schölkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. On causal and anticausal learning. Proceedings of the 29th International Conference on Machine Learning, ICML 2012, 2, 06 2012.
Alireza Shafaei, Mark W. Schmidt, and J. Little. Does your model know the digit 6 is not a cat? a less biased evaluation of "outlier' detectors. ArXiv, (bss/189.04729,, 2018.
Nikolai V Smirnov. On the estimation of the discrepancy between empirical curves of distribution for two independent samples. Bull. Math. Unio. Moscon, 2(2):3-14, 1939.
Amos Storkey. When training and test sets are different: Characterizing learning transfer. Dataset Shift in Machine Learning, pp. 3-28, 01 2009. doi: 10.7551 10.7551/mitprene //tp:p/rss/7777022277505500.3.30
Masatoshi Uehara, Masahiro Kato, and Shota Yasui. Off-policy evaluation and learning for external validity under a covariate shift. Advances in Neural Information Processing Systems, 33:49-61, 2020.
Simon Voss and Steve George. Multiple significance lests. BMJ, 31046696)) 1995. ISSN 0959-8138. doi: 00.1136/bmj.311 1.111.6666.107.. 1073. URL hattps://wee wwwpwwwww....................... http://.ca///cooee//////////////
Olivia Wiles, Sven Gowal, Florian Stimberg, Sylvestre Alvise-Rebulfi, Ira Kiena, Krishnamurthy Dvijotham, and Taylan Cemgil. A fine-grained analysis on distribution shift, 2021a. URL https://arxiv.. orgg /bs// 2110. 11328]
Olivia Wiles, Sven Gowal, Florian Stimberg, Sylvestre Alvise-Rebuff, Ira Ktena, Krishnamurthy Dvijotham, and Taylan Cemgil. A fine-grained analysis on distribution shift. arXiv preprint arxi::2110. 11328, 2021b.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnise a novel image dataset for bench- marking machine learning algorithms, 2017. URL
cite
Dataset is freely available at http://gghull.oommmaaanncceeeeee ffasion mnist. Benchmark is available at http://siieommtte.eeeeeeeeeeeeea
12

---

Published in Transactions on Machine Learning Research (05/2024)
Wojeiech Zaremba, Arthur Gretton, and Matthew Blaschko. B-test: A non-parametrie, low variance kernel two-sample test. In C.J. Burges, T.. Bottou, M. Welling, 7. Ghahramani, and K.Q. Weinberger (eds.), Ad- vances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013. URL https:
//proceedings.neure hatpppp.cpppppSSSS1.a400lllffle0
Paper. Paper.pdf
Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. In Proceedings of the 30th International Conference on International Conference
Machine Learning Volume 28, 1CML'LL 13, pp. 111 819 III 1277 JMLR.org, 2013.
13

---

Published in Transactions on Machine Learning Research (05/2024)
A Appendix
A.1 Additional Information on the Experimental Procedures.
Datasets.
The number of samples in the clean sets (i.e., the test sets) of the datasets we investigated are
88 follows:
MNIST, FMNIST and CIFAR-10: 10'000,
SVHN: 26'032.
Imagenette: 3'925.
A sample from each dataset can be seen in Figure
5
7
Figure 4: Sample images from all datasets used in the paper. From left to right: MNIST, FashionMNIST,, CIFAR-10, SVHN and Imagenetle.
Shifts. In order illustrate the effect of the shift types described Scction 5.1 of the main article, we show the effects of the shifts and their intensities on the MNIST dataset in Figure For the detailed parameters of each shift intensity (per dataset) we refer to the associated code.
clean
II
III
IV
VI
7
7
7
7
7
7777777
7
7
7
7 77
7
Figure 5: Ilnstratioon of intensities of the shift types Gaussian noise (top row), Gaussian blur (middle row) and Image shift (bottom row) - on a sample from the MNIST dataset.
A remark on the experimental setup As explained in Subsection 5.1 of the main text, we perform each our statistical tests on random subsamples from a clean set CS and a shifted set SS that has been crafted from CS by applying a controlled shift its elements. We could alternatively have crafted SS by applying the shift to another "clean" set drawn from the same distribution as CS. The advantage of our current setup is that it ensures that the only difference between the distributions of CS and SS comes from the shift, which we completely control, as opposed to some (small) pre-shift difference between the two clean sets that could arisc from the sampling process in this alternative setup. In practice, however, both setups should yield ext remely similar results; indeed, we randomly subsample from CS and SS to get the sets on which we perform the tests, and those sets are typically of size 100, whereas CS and SS are size 3'925, 10'000 or 26'032 depending on the datasct. Hence the subsampled sets from the CS and from SS should have almost no element in common (by which we mean the same picture appearing shifted in one of the sets and unshifted in the other), in the other setup.
14

---

Published in Transactions on Machine Learning Research (05/2024)
An example of activation graph computation We illustrate the definition of the activation graph
from Subsection 3.3 using the same notations. A layer Se TR R is shown on the left of Figure 6
[-1
Here ne and 1 = 3, and the weight matrix associated to fe is We = 2
Given an input a to -2
3
the network that corresponds to input æt (1, -2) to the layer Se, the associated activation graph Ge(æ) is shown the right of Figure G
-1
2
4
3
1
2
0
8
1
-2
Figure 6: On the left, neural network layer fe with the associated weights and input vector On the right, the associated activation graph.
15

---

Published in Transactions on Machine Learning Research (05/2024)
MNIST
Sample size
Shift
Int.
Feat.
10
20
50
100
200
500
1000
MD
0
7.7 + 1.3
11.2 1.6
29.3 + 2.3
5..9
94.7 1.1
9.1
II
CV
0.0 = 0.2
0.1
0.0
0.1 0.1
0.3 0.3::00
0.6
6.1
MD
8.6 1.4
26.1
6..9
1.3
100.0
100.0
100.0 0.0
GN
IV
CV
0.1
1.0
3.8
17.6
2.5
99.3
100.0 - 0.0
MD
1.9
54.2
93.5+133
100.0-0.0
100.0
100.0
100.0 0.0
VI
CV
3.7 1.0
20.4
65.6
9.5
100.0-000
100.0 - 0.0
100.0 0.0
1.9 0.7
2.7
3.9
1.0
9.5
22.2
40.7 + 2.5
II
CV
0
0.1
0.0
0.0
0.2
0.1
0.1 0.1
MD
4.8 18=11
18.1
203
63.1
9.2
100.0-00.
100.0
GB
IV
CV
0.0 0.2
0.0
0.1
0.3
1.5
11.3
44.9
MD
9.2115
2.
2.5
9.
100.0-00 0.0
100.0
100.0 0.0
CV
0.1
0.4
1.6
5.1
2.11
8..5
100.0 0.0
MD
3,5 0.9
1.
18
32.7
2.43
98.0
100.0 0.0
II
CV
0.2
0.0
0.2
0.2
0.7
6.3
28.4
MD
5.6 = 1.2
1..
42.1
7..
9.7
100.0 0.0
100.0 0.0
IS
IV
CV
0.1 0.2
0.9
2.
1.8
50.0
99.5 0.3
100.0 0.0
MD
10.4
34.0
2.3
0.5
100.0
0.0
100.0 0.0
VI
CV
1.3 = 0.6
73
317
83. 1.9
100.0
0.0 100.0 - 0.0 100.0 0.0
Table Power the statistical test with MAGDiff (abbreviated as MD) CV representations for the shift types Gaussian noise (GN), Gaussian blur (GB) and Image shift (I8), three different shift intensities (II, IV, VI) and fixed 0.5 for the MNIST dataset. The estimated 95%-confidence intervals are indicated.
A.2 Additional Experimental Results
Sample size. To further support our claims, we include comprehensive results of the power with respect to the sample size for the MNIST, Imagenette and CIFAR-10 datasets Tables 4 and We provide all results for the shift intensities 11, IV and VI, for all shift types, and fixed d 0.5 for MNIST, CIFAR-10 rospectively 5 = 1.0 for Imagenette (the were chosen that the task is comparatively casy at high shift intensity and hard at low shift intensity for both methods).
collect the plots of the estimated powers of the test for multiple cases, in addition to the presented in the main article. Note that the only situation in which MAGDiff is very slightly outperformed by the baseline CV, is the case of FMNIST, when consider MAGDiff repre- sentations of layer 1-1. In all other cases, shift. detection using MAGDiff representations clearly outperforms the baseline of CV by a large margin.
Shift intensity. In Figures and
Model accuracy. In Figure 10 we show the impact of the shift type and intensity the model accuracy. It is interesting to note that, even in cases where the model accuracy is only minimally impacted (e.g., for Gaussian blur on the MNIST and FMNIST datasets), our method can still reliably detect the presence of the shift.
Norm variations. As mentioned in the main paper, many variations of MAGDiff are conccivable. Here, we present some experimental results for variations on the type of norm that is used to construct the MAGDiff representations. In Figure [I we show the results where, instead of the Frobenius-norm, we consider the spectral norm well as the operator norm || Iloe induced by the sup-norm on vectors. The spectral norm is equal to the largest singular value and || al defined by:
|
M 11x00 = sup
max
mig
2/0
|
S
16

---

Published in Transactions on Machine Learning Research (05/2024)
Imagenette
Sample size
Shift
Int.
Feat.
10
20
50
100
200
500
1000
MD
0.7 = 0.4
2.2.0.1
6.3 1.2
16.7 + 1.9
46.2 2.5
93.3 + 1.3
99.9 0.1
II
CV
2.4
0.8
4.1
2.7
4.1
1.0
4.2 1.0
7.3 1.3
1.5
MD
0.5
0.9
1.3
22.1
2.1
60.7
2.5
98.5 0.6
100.0 0.0
GN
TV
CV
1.6
4.1
0.9
1.0
4.7
7,5 1.3
1.5
0.9
3.6
133
2.11
2.5
9.6
100.0-0.0 0.0
VI
CV
2.7
4.9
3.9
5.3
6.3
1.8
2.4
MD
0.3
0.8
4.1
1.83
2.5
91.7
0.2
II
CV
2.1
3.7
3.9
3.7
1.3
11.3
1.0
MD
1.5
3.5
9.1
2.3
2..9
99.1
100.0 -.000-..0 0.0
GB
CV
238
4.5100
3.7
1.1
6.2
13
2.2
MD
1.3
5.0
1..2+199
5.55
1.5
100.0 100.0-0.0 0.0
CV
2.1
0.9
3.9
5.0
1.3
18.5
4 2.5
MD
0.5
1.1
1.7
1.7
1.76
24.5
1.9
II
CV
2.5
3.7
3.0
3.1
1.0
1.2
1.5
MD
0.3 0.7
2.9
9.1
2..1
2..
98.3 -
IS IV
CV
1.7
0.9
3.9
3.0
4.1
1.4
12.8 1
MD
0.6
0.8
1.
18
44.3
1.2
99.9 0.1
VI
CV
0.7
3.9
1.0
0.7
4.6111
6.1 1.2
8.3
15.5 1.8
Table 5: Power the statistical test MAGDiff (abbreviated as MD) CV representations for the shift types Gaussian noise (GN), Gaussian blur (GB) and Image shift (18), three different shift intensities (II, IV, VI) and fixed for the Imagenette dataset. The estimated 95%-confidence intervals are indicated.
CIFAR-
Sample size
Shift
Int.
Feat.
10
20
50
100
200
500
1000
MD
1.8
0.7
5.3 +
1.1
16.7 1.9
47.0 + 2.5
86.9 1.7
100.0
0.0
100.0
-.0
II
CV
2.3
0.8
4.5
6.9
19.1
+ 2.0
08.3:22
88.3 + 1.6
99.9 + 0.1
1.. 2.8
36.7
2..
0.4
100.0
0.0
100.0
0.0
CN
IV
CV
2.
6.7
11.7.166
29.
6.
9.4
100.0
0.0
1..7 2.7
49.2
91.2
9.1
100.0 0.0
100.0 0.0
VI
CV
2.8
7.3
142
37.5
77.3
99.9
100.0 0.0
0.5
2.8
6.6
18.9+2.0
4.5
1.3
100.0 0.0
I
CV 0.8
4.0
3.4
6.8
1.
2.3
58.9 2.5
MD 187
5.7+12
19.
4..7
1.5
0.1
100.0 0.0
GB
IV
CV 2.8
6.2
7.3
1..9
2..9
1.9
99.8 0.2
MD 2.7
1.2
23.4121
62.5.144
9..1
100.0
0.0
100.0 0.0
VI
CV 3.9
8.9
14.7.188
44.211.55
8.8
0.0
100.0
0.0
MD 0.3
1.3
3.6
6.7
1..6
6..1
92.6 - 1.3
II
CV
2.800.
3.9
2.8
1.4
7.9
31.5 1.9
MD 0.4
2.4
3.9
16.1
3.5
8..2
IS
TV
CV
2.7
1.0
2.9
6.7
10.8
2..1+2.2
53.1 2.5
MD
1.
3.9
8.9
22.8
0.8 2.5
100.0 0.0
VI
CV
2.7
4.1111
4.+
9.5
15.5
1.8 4..5
83.2 - 1.9
Table Power the statistical test with MAGDiff (abbreviated as MD) CV representations for the shift types Gaussian noise (GN), Gaussian blur (GB) and Image shift (IS), three different shift intensities (II, IV, VT) and fixed 0.5 for the CIFAR-10 dataset. The estimated 95%-ccofidence intervals are indicated.
17

---

Published in Transactions on Machine Learning Research (05/2024)
MNIST: Gaussian noise, sample size:
MNIST: Gaussian blur, sample size: 100
Image shift, sample
1.0
power estimation
0.8
power estimation
power estimation
0.8
0.6
0.6
0.4
0.4
0.2
0.0
0.0
0.0
111
IV
V1
11I
IV
I1I
IV
shift intensity
shift intensity
Shift intensity
MNIST: Gaussian noise, samp size: 100
MNIST: Gaussian blur, sample size: 100
MNIST: Image shift, somple size: 100
1.0 power CV
CV
power
Type error
Typel error CV
error MAGDIF
Type MAGDIf
Dower eztmnation
Dower extimaaion
0.6
power CV
0.6
0.6
power
Type lerror CV
Type error MAGCIff
0.2
0.2
0.0
intersity
intensity
Intensity
Figure 7: Power and type I error of the test with MAGDiff (red) and CV (green) representations w.r.t. the shift intensity for various shift types on the MNIST dataset with 0.5, sample size 100, for layers l_1 (top row) and L- (boltom row).
for M E mx,, Comparing to Figure 7 we observe that the results for the Frobenius norm and the spectral norm are almost identical. However, while the results for the || ll are still better (in almost all cases) than those of the baseline CV, they less powerful than those of the Frobenius norm.
18

---

Published in Transactions on Machine Learning Research (05/2024)
FMNNIST Gaussian sample
FMNIST: blur, sample
FMNNST: Image sample 100
power CV
power CV
power CV
power MAGDIff
power MACDIT
power MAGDIIf
Type
Type
Typel
0.8
Type ciror MAGDI!
0.8
Type crror MAGDI!
0.0
Type: crror MAGDI!
02Wer coliratton
DoWer estirattion
0.6
0.4
0.4
0.2
0.0
intersity
shift intersity
intersity
noise, sample
blur, sample
FMNNST: Image sample
power CV
power CV
MMEDIf!
MAGDIFf
Type
Typel CV
0.8
Type error MAGDif
Type error MAGDIF
Doweer estmation
Doweer estimeeiio
0.6
power CV
0.6
0.6
MASDFf
Type leror
Type lerror MAGDiff
0.2
0.2
0.2
intensily
shift intersity
intensity
Figure 8: Power and type 1 error the test with MAGDiff (red) and CV (green) representations w.r.t. the shift intensity for various shift types the FMNIST dataset with 5 0.5, sample size 100, for layers l-1 (top row) and L-3 (bottom row).
A.3 Theoretical observations regarding the preservation of shift distributions by continuous functions
In the main article, we mentioned the fact that under generic conditions, two distinct distributions remain distinct under the application of non-constant continuous function (though this does not necessarily trans- late to good quantitative guarantees). In this section, make this assertion more formal and provide an elementary proof.
Let X be separable metric space, and denote by P(X) the set of probability measures on X equipped with its Borel o-algebra. Let Cx(X) be the real bounded continuous functions on X. We consider the weak convergence topology on P(X); remember that a subbase for this topology given by the sets
UfaD = {-
/ fdw EJa, M.
for f Cx(X) and a < e (see for example Kallianpur 1961
Now let X, Y be two such separable metric spaces with their Borel o-algebra, Any measurable map F; X Y induces map
F. -PY) 4 F.(1).
where F2(p) is the pushforward of μ by F, that the measure on P(Y) characterized by F.()( [(F-1 ((F-((A) for any Borel set A Y.
Fact 1. F X + Y is continuous, then F. P(X) P(Y) is continuous for the weak convergence topology.
Proof. Given f E CxX and a < b R. we see that F, (U,,b)) Uoo which is enough to conclude by the definition of subbases.
19

---

Published in Transactions on Machine Learning Research (05/2024)
CIFAR10: Gaussian sample 100
CIFARID: Gaussian sample 100
CIFAR1O: Image shift. sample size:
power CV
power CV
power MAGD!!
power KAGDIT
Type
Type
0ower coliatton
0.8
0.8
Type crror MAGDIff
power estirattion
0.8
----
Typei error MAGGIf!
power CV
0.6
Dower NAGD!!
0.4
Type eeror MAGDIT
0.4
0.4
0.2
0.0
intersity
shift intersity
shift intersity
SVHN: Gaussian noise, sample 100
SVHN: Gaussian sample size: 100
SVHN: Image sample 200
1.0
power CV
power CV
MMEDIf!
MAGDIFf
Typel error
Type CV
0.8
0.8
Type error MAGDif
Type error MAGDIf
Dower estmatoon
Dower estimeeion
0.6
Dower CV
0.6
0.6
MAGD#ff
Type
Type leror MAGDiff
0.4
0.2
0.2
0.-
intensily
shift intensity
intensity
Imagenette: Gaussian sample 200
Imagenette: Gaussian blur, sample size: 200
Imagenette: Image shift, sample size: 500
1.0
power CV
1.0
power CV
1.0
MAGDIf"
power MAGDIFf
Power estimetiin
0.6
0.8
Type larror CV
type error MAGDiff
power estimation
0.6
0.8
Typc crror
Type I erarr CV
estimation
0.6
0.8
power MAGDitt
power CV
Type crror CV
0.4
0.4
0.4
Type crror MAGDIf
0.2
0.2
0.2
0.0
IV shift intensity
III
intensity
shift intensity
Figure 9: Power and type I error of the test with MAGDiff (red) and CV (green) representations w.r.t. the shift intensity for various shift types on the CIFAR-10, SVHN (with 6 0.5) and Imagenette (with ó 1) datasets. Sample sizes and values of 5 were chosen to make the plots expressive as possible (low power for low shift intensity, high power for high shift intensity), as the difficulty of the task varies depending on the shift lype and dataset.
The following result follows from standard arguments; we give an elementary proof for the convenience of the reader.
Proposition 1. Let F X > R be continuous and non-constant for X a separable metric space, and let v ME (.((X)) P(RR. Then the complement F--(()))) = F( of the set F-+((v) dense open set of P(X) for the weak topology.
Proof. As R is separable and metric, is easy to show that the singleton {u) E P(B) is closed (see for example (Kallianpur) [961 Thm 4.1)). As we know from Fact [ that F, is continuous, we conclude that F. ([v) is closed and F. ([v))² is open.
It remains to show that it is dense in P(X). Let u belong to (()(()), and let V c P(X) be open set containing μ. We have to show that F. ((v)) n V is non-empty. Thanks to the definition of the weak topology, we can assume (by potentially taking a subsct of V) that
V
n
p P(X) s.t. / fıdā ejan
20

---

Published in Transactions on Machine Learning Research (05/2024)
for some fn E Cx(X) and d1 b1 .1....,......................... Ons dn R with a, < b, for all i. Let 21 be any point in the support of μ.. Then (x 6)) > 0 for all 5 > 0 by definition of the support. As F is non-constant, there exists 22 E X such that F(22) is not equal to F( Let us assume that F(x1) > F(22) (the proof is similar if (x2)) > F(x1)). By continuity, there exists c > such that F(x) > F(22) for any z E B c). Define m = n ()), O. For E|0, 1[: we define new measure Ht as follows for any measurable set A, we let
Pt (A) = / (A)B(a1,))) + (1 (L))))(((().))n tmbzed
For any such EJ0,, 16, observe that
P. F.(m))(((22); tool) (((()))))))t
(F)) ty ((11,))
(F))))((((2))) to0),
which shows that F.(M) + F-(ee), hence that ue E (F
On the other hand, we see that 1Sx fidu fx fidul 2tm|lfillso for I..... n. Since μ V n ((X) s.t. fx Sadj Elas,bih thus 1t E for EJO, small enough. This shows that Vn (( is non-empty, and thus we conclude that (fv)) is dense in P(X).
As direct corollary, get the following statement, where generic, above, means that the property is true for any random variable x' whose distribution belongs to fixed dense open set of the space of distributions IR"
Corollary 1. Let F: : R" - R& be non-constant continuous function represented by neural network, and let x be a random variable on R For generic random variable x' on IR", the distribution of F(x') will be different from that of F(x).
Proof. R" is a separable metric space, and if F is non-constant, so is at least one of its coordinate functions ::: R" - R, to which Proposition then applies. If the distribution of Fe(x") is different from that of (.(r), then the distribution of F(x') is different from that of F(z).
21

---

Published in Transactions on Machine Learning Research (05/2024)
MNIST: model accuracies Gaussian noise
MNIST: model accuracies for Gaussian
MNIST: model accuracies for Image shift
90.50
0.25
90.55
0.5
95
1.0
38.50
90
#cCurecy (%6
95
90.45
SCCrsccy (%6
85
50.40
04
80
90.30
0.25
70
0.5
$8.25
0.5
1.0
50.70
65-
1.0
N
II
V1
VI
shift intensity
shift intensity
Shyft intensity
FMNIST: model accuracies for Gaussian noise
FMNIST. model accuracies for Gaussian blur
FMNIST: model accuracies for Image shift
92.0
0.5
85
1.0
90.5
80
accuracy (50
90.0
acturacy
15
70
60-
0.25
0.25
60-
0.5
35
1.0
1.0
intensity
Intensity
shift intensity
CIFAR10: model accuracies for Gaussian noise
CIFAR10: model accuracies for Gaussian blur
CIFAR10: model accuracies for Image shift
70-
70
60-
(50
accuracc 156
60
40
accuracy
50
30
80-
50-
0.25
0.25
30-
0.5
0.5
0.5
1.0
V1
IV
shift intensity
shift intensity
shift intensity
SVHN: model accuracies for Gaussian naise
SVHN: model accuracies for Gaussian blur
SVHN: model accuracies for Image shift
90
80
00
85
70
80
60
accurscy (%)
50
65
0.25
0.25
60-
0.25
30
1.0
1.0
55
1.0
shift intensity
intensity
intensity
Imagenetter model accuracies for Gaussian noise
Imagenette: model accuracies Gaussien blur
Imagenette: model accuracies for image shift
90
QZ
90
accuracy (8)
85
07
80
80
96
70
0.25
0.25
0.25
70
0.5
94
0.5
65
1.0
11
shift intensity
stift intensity
shift Intansity
Figure 10: The impact of the shift type and intensity on the model accuracy for 8 = 1.0 (blue), 5 == 0.5 (green) and = 0.25 (red).
22

---

Published in Transactions on Machine Learning Research (05/2024)
sample
MNIST: Gaussian blur, sample size: 100
MNIST: shift, sample 100
1.0
power estiationn
0.8
Dower estimatiin
Power estimation
0.8
0.6
0.6
0.4
0.4
0.4
0.2
0.0
IV
V1
IV
shift intensity
shift intensity
Shift intensity
MNIST: Gaussian noise, sample size: 100
MNIST: Gaussian blur, sample size: 100
MNIST: Image shift, sample size: 100
1.0
1.0
power estimation
0.8
power estimation
power estimation
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0.0
IV
IV
intensity
intensity
shift intensity
MNIST: Gaussian noise, sample size: 100
MNIST: Gaussian blur, sample size: 100
MNIST: Image shift, sample size: 100
1.0
1.0
power estimation
power estimation
0.8
power estimation
0.8
0.6
0.6
0.4
0.2
0.2
0.0
shift intensity
shift intensity
shift intensity
Figure 11: Power and type I error of the test with MAGDiff (red) w.r.t. the Frobenius norm, used in all other experiments, (top row), the spectral-norm (middle row) and I|s (bottom row) and CV (green) representations w.r.t. the shift intensity for various shift types on the MNIST dataset with 8 0.5, sample size 100, for layer E-1.
23


Reviews:
[{"review_id": "jjjlsOxTwP", "reviewer": "Reviewer_P2yf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is well-written and easy to follow. The proposed method is conceptually simple yet performs well as supported by detailed experiments.\n\nMy concerns about the paper is some required clarification:\n\n1. Due to different types of distribution shift in classification problems (Section 3.1), the notations $\\mathbb{P}_0$ and $\\mathbb{P}_1$ should be clarified. Does $\\mathbb{P}$ denote the joint distribution of covariate $x$ and its label, or only the marginal distribution of $x$? In the context of Section 3.1, it appears to be the joint distribution in order to unify the covariate shift and label shift. However, in Section 4.2, $\\mathbb{P}$ only denotes the marginal distribution of covariates.\n\n2. Is the output space of $\\Psi$ a typo in Section 4.1, where $\\Psi$ outputs a vector in $\\mathbb{R}^N$. Here I believe $N$ is the sample size.\n\n3. In the experiments, what is the precise definition of intensity in different types of shifts? Moreover, these shifts are deployed only on the covariates, corresponding to covariate shift. Is it possible to extend to label shifts? This might connect to my follow up question: In implementing the algorithm, do we need to know the label information in the test data set?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a distribution-shift detection method using pre-trained neural network classifiers. The test statistic is constructed using neural network intermediate layers for extracting an activation pattern. This is different from the existing methods, such as using the classification logits as a test statistic. Experimental results demonstrate the proposed test statistic yield better performance in distribution shift detection: it requires less sample to detect even faint changes that might be difficult for classification logits-based test.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Requested changes are summarized in the previous section.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bDrnzq5812", "reviewer": "Reviewer_vNs6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is relevant to the TMLR community, and shift detection is an important problem.\n2. The experiments consider several datasets and architectures under different settings.\n\nWeaknesses:\n1. The paper is unorganized and misses significant and crucial information. For example:\n- What is Card(Xi) in Section 3.1?\n- What are p-value and alpha (significance level) in Section 3.1? they need to be defined.\n- The authors state in the footnote in Page 3 that the method can also be used for resnet-like architectures. I think that since your experiments are based on resnets, you should clearly define the graph for this case.\n- Section 3.3 is very poorly written. It is not clear what the vertices are, what do they represent and how do they interact with each other. It is also not clear how the weighting matrix W is defined. I think it needs to be re-written and also include a figure that illustrates the discussed graph.\n2. While the experiments show good results compared to CV, it seems to still be a limited comparison. I am not an expert on shift detection methods in NNs but I would assume more methods exist. Why not compare with them? Otherwise, it is hard to quantify the contribution of the method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors suggest MAGDiff, a method to detect covariate distribution shift between training and inference times in neural networks. \nThe method is based on black box shift detection (BBSD) and shows improvement over this method on several benchmarks.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the suggestions in my weaknesses report.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ui0BGGJsWn", "reviewer": "Reviewer_yjoa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\nThe method is very simple and not computationally expensive. These are strong positives for the method to be used. \n\nThe experiments show that the method does result in improved performance compared to some baselines. \n\n**Weaknesses**\n\nI am a little confused about certain aspects that I do not think are reasonable for real-life situations. \n\n1) The access to CS. This should be better highlighted. In real use cases, we would not have access to CS. We would only have the training data and SS. Hence, I think the better comparison would be to replace CS with the training data. (It is because of this I vote no for claims and evidence)\n\n2) The method seems to focus on detecting whether a collection of samples is out of sample. It would be interesting to explore the single sample case. In real-life scenarios, we potentially have a mixture of in-distribution and out-of-distribution samples; hence, being able to detect which samples are made seems the more practical use case.\n\n\n**Questions**\n\nHave the authors explored the classifier's effects on their method's performance? For example, do over-parameterized classifiers performance better or worse? \n\nHave you considered the connection to Neural Collapse? Specially, since you look at the features at the penultimate layer.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a new technique for detecting distribution shifts. In particular, the paper studies the problem where the conditional distribution $p(y|x)$ stays the same, but the distribution $p(x)$ changes from the training to the test period. \n\nTo detect distribution shift, the paper supposes we have a neural network $F$ trained on the training data. Let $W_1, \\ldots, W_\\ell$ be the weight matrices for the layers, and let $x_1, \\ldots, x_{\\ell}$ be the inputs to the layers. Then, the paper defines an activation graph as a matrix $W_k \\cot \\text{diag}(x_{k})$. Then, to determine shifts, the paper computes the mean matrix for each class for the training distribution and then, for a new sample, measures the deviation (using the Frobenius norm) [This might differ slightly from the paper because the definition of $G$ wasn't clear, but this should be equivalent due to the Frobenius norm]\n\nThe paper then uses this as the statistic. It uses the KS test with the Bonferroni correction and a p-value of $0.05$ to determine whether a collection of data points is out of distribution. The paper extensively tests their methods against prior work.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the weakness. At a minimum I think the first weakness should be addressed.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers generally appreciated the simplicity and scalability of the proposed approach, as well as its empirical evaluation.  There were several comments relating to clarity of presentation, but the authors have largely addressed those.  Overall, the committee is in consensus that the submission should be accepted.  I encourage the authors to account for the more open-ended (high level) feedback they received (in particular detection of out-of-distribution samples) in future work, as it may greatly enhance the impact of this research vein.
Decision: Accept (poster)