TIME-LAPSE: U NCERTAINTY S CORING VIA L ATENTS PACE E MBEDDINGS OVER T IME
Anonymous authors
Paper under double-blind review
A BSTRACT
Safe deployment of trained ML models requires determining when they are uncertain of their predictions and refraining from using them for decision-making.
Existing approaches inspect test samples in isolation to estimate their corresponding predictive uncertainty.
However, in the real-world, deployed models typically see test inputs consecutively and predict labels continuously over time during inference.
We propose TIME-LAPSE, an uncertainty scoring framework that examines a sequence of la tent- sp ace e mbeddings over time prior to the current sample to determine its predictive uncertainty.
Speciﬁcally
(a) our spatial uncertainty score estimates uncertainty using distance metrics (Mahalanobis distance) and similarity metrics (cosine similarity) in the latent-space and (b) our temporal uncertainty score determines deviations in correlations over time using representations of past inputs in a non-parametric, sliding-window
based algorithm
We evaluate
TIMELAPSE through the lens of out-of-distribution (OOD detection and dataset shift detection on tasks over diverse domains audio and vision using public datasets and further benchmark our approach on a challenging real world electroencephalograms
EEG dataset for seizure detection
We achieve state of
-the- art results for
OOD detection in the audio and
EEG domain and observe considerable gains in semantically corrected vision benchmarks
We show that TIME-LAPSE is more driven by semantic content compared to other methods i.e. it is more robust to dataset statistics
We also show that
TIME-LAPSE outperforms spatial methods signiﬁcantly through our sequential evaluation framework that emulates real-life drift settings through extensive experiments and ablations.
I NTRODUCTION
Modern machine learning (ML) has seen tremendous success in various tasks across multiple domains (Bojarski et al., 2016; Hinton, 2018; Kreinovich & Kosheleva, 2020; van den Oord et al., 2016), surpassing human performance in many benchmarks (Esteva et al., 2017; Yala et al., 2019; Krizhevsky et al., 2012).
However, deep learning models have been known to fail silently and catastrophically with highly conﬁdent predictions (Nguyen et al., 2015; Goodfellow et al., 2015;
Guo et al., 2017).
Such models assume a closed world scenario, i.e., they assume that variability encountered when deployed in the real-world would be similar to the variability present in their training data.
In practice, they encounter an open world where incoming samples can come from shifted distributions or completely new distributions (Liu et al., 2020).
This behaviour can have severe consequences in mission-critical domains such as healthcare and autonomous driving, where errors can be costly, resulting in injury or even death (Amodei et al., 2016).
Most widely cited models do not come with an uncertainty scoring mechanism to say “I don’t know” or to abstain from prediction (Kompa et al., 2021).
As deployed ML models are used to inform real-world decisions, it is important that they have the ability to understand when they ought to be “unsure”.
A good uncertainty scoring framework must assign higher uncertainty estimates when the network is faced with distributional shifts or new semantic content, i.e., out-of-distribution (OOD) data but must be certain (low uncertainty) and generalize well on learnt distributions yet unseen data (in-distribution or InD data).
The task of identifying when incoming inputs are drawn far from the training distribution is called out-of-distribution (OOD) detection.
At its core, it is a binary classiﬁcation problem, evaluated using measures such as area under the receiver operating curve (AUROC), area under the precision recall curve (AUPR) and false positive rate at 80% true positive rate (FPR80).
Current OOD evaluation schemes hold different datasets to be OOD sources, without considering if the distinctions are semantic in nature (Ahmed & Courville, 2020) leading to methods that are insensitive to semantic overlap and biased to dataset statistics.
Closely related is the task of dataset shift detection (MorenoTorres et al., 2012), crucial for deployed systems.
Dataset shift can occur due to sample selection bias, non-stationary environments, missing values, new concepts evolving over time, etc.
Most ML systems do not evaluate their ability to detect such shifts, presenting a huge gap between models performing well on test sets and models capable of being deployed in the wild.
In this paper, we propose TIME-LAPSE, a predictive uncertainty quantiﬁcation framework using la tent sp ace e mbeddings over time , evaluated through downstream tasks of OOD detection and dataset shift detection.
A key insight in our approach is that most real-world scenarios (such as electroencephalogram (EEG) seizure analysis, healthcare decision making, autonomous driving, etc.) involve consecutive inputs to the model that are likely to be correlated over time.
For example, an obstacle detector deployed in a self-driving car will see images correlated over time.
A seizure detector installed in a neurology clinic will process hours of time-correlated EEG signals.
Even low-risk image classiﬁcation models deployed in search engines over the cloud will see correlated inputs over time when grouped by user ID or location.
Thus, sequentially occurring samples share meaningful semantic correlations.
In TIME-LAPSE (Fig. 1), we compute spatial uncertainty scores based on the following hypotheses:
an encoder with enough inductive bias likely (a) maps OOD inputs “far
" from other InD samples in the latent space under the Mahalanobis distance metric (Lee et al., 2018) and (b) produces dissimilarities between their representations under the cosine similarity metric (Jones & Furnas, 1987).
We compute temporal uncertainty scores by exploiting correlations between lower dimensional representations of consecutive samples over time when examined as a sequence.
A sequence of InD inputs will show more correlation than a sequence with both InD and OOD inputs or a sequence with just OOD inputs.
Recent works limit themselves to OOD detection tasks with evaluations purely on highly curated image benchmarks, but do not extend their tests to more diverse and realistic scenarios required for practical applications (Ren et al., 2019; Ahmed & Courville, 2020).
Such OOD image benchmarks consider different datasets to be OOD sources, neglecting any semantic overlap between their OOD test sets and InD train sets, leading to narrow model capabilities (Yang et al., 2021).
We show that
TIME-LAPSE is more driven by semantic content compared to other techniques, i.e. it is more robust to dataset statistics.
Moreover, current evaluation schemes do not consider different dataset drift settings that could occur in the real-world.
We put forth an additional sequential evaluation framework to evaluate methods under realistic conditions of data drifts and show that TIME-LAPSE outperforms other methods signiﬁcantly.
We demonstrate the efﬁcacy of our proposed method by comparing against multiple baselines over different domains (vision, audio and clinical EEGs).
We achieve state-of-the-art (SOTA) on both audio and EEG datasets while improving performance on semantically corrected image benchmarks.
To the best of our knowledge, we are the ﬁrst to use temporal sequences of latent space embeddings for uncertainty quantiﬁcation and subsequent OOD detection.
We are also the ﬁrst to use deep-learning based uncertainty scoring techniques for EEG analyses.
To summarize, our key contributions are:
• We propose TIME-LAPSE, an uncertainty scoring that uses samples’ similarity in the latent space and their temporal similarity amongst sequential inputs to determine their predictive uncertainty.
• We show that scores from TIME-LAPSE are driven by semantic content and are more robust to dataset statistics whereas popular baselines, benchmarked on standard image datasets, are susceptible to dataset statistics and overlook semantic overlap for OOD detection.
• We benchmark TIME-LAPSE on diverse domains: audio speech classiﬁcation, seizure detection using clinical EEGs and image classiﬁcation.
TIME-LAPSE achieves state-of-the-art (SOTA) in audio tasks, the EEG domain and semantically corrected image benchmarks.
• We propose a sequential evaluation framework for dataset shift detection to evaluate methods under realistic conditions of data drifts and show that TIME-LAPSE outperforms other methods signiﬁcantly.
We hope this evaluation scheme will be adopted for routinely characterizing ML system performance in the wild.
R ELATED W ORK
Uncertainty estimation is a rich ﬁeld with a long history.
Classical techniques include density estimation (Breunig et al., 2000), one-class SVMs (Schölkopf et al., 1999), tree-isolation forests (Liu et al., 2008), etc for one-dimensional data such as time-series However, such techniques scale badly with input dimensionality (Rabanser et al., 2019).
Calibration is a frequentist notion of uncertainty, measured by proper scoring rules like log-loss or Brier scoring (Guo et al., 2017; DeGroot & Fienberg, 1983; Dawid, 1982).
Uncertainty estimation for deep neural networks (NNs)
(Lakshminarayanan et al., 2017) typically use a Bayesian formalism to learn distributions over model weights (Blundell et al., 2015; Malinin & Gales, 2018; Chen et al., 2019; Graves, 2011; Neal, 1996; Welling & Teh, 2011).
However, most Bayesian methods are difﬁcult to train and computationally expensive.
Gal &
Ghahramani (2016) use Monte-Carlo dropout at test-time with an approximate Bayesian interpretation to derive uncertainty estimates.
It is also computationally expensive, requiring k forward passes for each instance during inference.
Quality of uncertainty estimates are commonly evaluated using
OOD detection.
Moreno-Torres et al.
(2012) give a complete overview of closely related topics of distributional shift detection including covariate shift, label shift and concept drift (Gama et al., 2014).
Several works consider semi-supervised techniques for OOD such as outlier exposure methods (Hendrycks et al., 2019a; Ruff et al., 2020), while many assume the supervised setting where the target OOD distribution is known and propose alternative training strategies and auxillary OOD tasks (DeVries & Taylor, 2018; Hendrycks et al., 2019b; Mohseni et al., 2020; Shalev et al., 2018) in a multi-task setting.
In this paper, we consider the more general setting of unsupervised OOD detection where no OOD examples are labelled as such and the encoder has access to only InD training labels.
Other methods use representations from NNs to infer OOD inputs.
Hendrycks & Gimpel (2017) use maximum softmax probabilites (MSP) to detect OOD data.
Liang et al.
(2018) improve this by introducing a temperature parameter to the softmax equation.
Lee et al.
ﬁt class-conditional
Gaussians to intermediate activations and use the Mahalanobis distance to identify OOD samples.
Recently, some works use self-supervision to get better representations to improve OOD detection (Tack et al., 2020; Winkens et al., 2020; Sehwag et al., 2021).
Several methods directly use input likelihoods as the OOD detection score.
However, studies have shown that generative techniques may be overconﬁdent on complex inputs (Nalisnick et al., 2019).
Modiﬁed likelihood scores have been proposed since then (Serrà et al., 2020; Xiao et al., 2020; Choi et al., 2019) including energy-based models (Liu et al., 2020; Du & Mordatch, 2019; Grathwohl et al., 2020) and likelihood ratios (Ren et al., 2019).
Reconstruction-based OOD methods use reconstruction loss as the uncertainty score and primarily use auto-encoders (Zong et al., 2018; Pidhorskyi et al., 2018) or GANs (Schlegl et al., 2017; Deecke et al., 2019; Perera et al., 2019).
Our temporal uncertainty scores builds upon and shares similarity to change point detection methods in time-series data (Aminikhanghahi & Cook, 2017; Kifer et al., 2004) along with signal processing techniques such as Particle and Kalman ﬁltering (van der Merwe et al., 2001; Kalman, 1960).
TIME-LAPSE: O UR U NCERTAINTY S CORING F RAMEWORK
We propose TIME-LAPSE, an uncertainty scoring framework using la tent- sp ace e mbeddings over time .
We consider a multi-class classiﬁcation setting here, though our framework can be extended to other scenarios such as regression, segmentation, etc.
3. P ROBLEM S ETUP
Let X represent our high-dimensional input space, X ⊆ R n .
Let Y = { 0 , 1 , 2 , ..., C − 1 } denote the label space where C is the number of classes.
In standard multi-class classiﬁcation, we learn a classiﬁer f :
X (cid:55)→ Y using a dataset D train (assumed to be sampled from an underlying distribution p ∗ ) such that f ( x ) = p ( Y = y i | x ) , where x ∈ X and y i ∈
Y ∀ i .
The ﬁnal prediction for an unseen input x is given by ˆ y = arg max
y i p ( y i | x ) .
Given a sequence of unseen inputs, x t (where t denotes time) to a classiﬁer during inference, our goal is to output a score s ( x t ) and a selective function g ( x t ) with threshold th ∀ x t such that
[Equation]
or
FLAG
AS OOD
, else
We call the score s ( x t ) the associated uncertainty score of input sample x t
3. TIME-LAPSE F RAMEWORK
We use the learnt classiﬁer f to derive an encoder h : X (cid:55)→ U that maps high-dimensional inputs x ∈ X ⊆ R n onto a latent space U
⊆
R d through its intermediate representations.
Note that h is inherently learnt when f is trained using the InD training dataset D train .
TIME-LAPSE is agnostic to the training method, which can be supervised, self-supervised or unsupervised depending on how the label information in D train is utilized.
3.2. L EARNING S PATIAL U NCERTAINTY S CORES
To extract the spatial uncertainty score of an input test sample’s prediction, we apply two fundamental hypotheses: (a) a well-trained encoder maps InD samples onto a dense region in the latent space but maps OOD inputs outside this region under a distance metric d with high likelihood, and (b) the OOD inputs that get mapped to the latent space do not share similarity with InD inputs under a similarity metric, sim .
To identify spatially distinguishable OOD inputs, we ﬁrst model the extent of InD region D within the ﬁxed-dimensional latent space, using the InD samples.
We extract a coreset by sampling from our
InD training data, D train .
[Equation]
We compute the distance score s dist and the similarity score s sim of the unseen test sample x by comparing its latent representation to that of samples in the coreset using a distance metric d and a similarity metric sim such that:
[Equation]
Finally, the combined spatial uncertainty score is given by
[Equation]
We choose the distance metric d to be the Mahalanobis distance obtained using class-conditional
Gaussians (Lee et al., 2018) without label smoothing (Winkens et al., 2020) where class-wise means µ c and covariance matries Σ c are estimated from the coreset.
[Equation]
As similarity metric sim , we adopt the cosine similarity between the test input’s representation and the coreset.
[Equation]
We note that TIME-LAPSE will accept any d and sim that is most suited for the target domain.
3.2. L EARNING T EMPORAL U NCERTAINTY S CORES
Our hypothesis in extracting our temporal uncertainty scores is that there is value in evaluating samples in the context of other samples instead of viewing them independently.
In our framework, we consider high-dimensional inputs to the model as a sequence (Fig. 1).
We use the representational capabilities of modern encoders and examine them over time instead of directly modeling the incoming high-dimensional, multivariate data stream or treating each sample independently, i.e., we detect when input samples go out-of-distribution by comparing their reduced-dimension scores ( s spatial ) with those of other inputs over time.
We note that when these scores are scalar (as in our case), the problem reduces to change-point detection for a one-dimensional sequence.
We put forth an unsupervised, non-parametric, sliding-window based algorithm that builds on the work done by Kifer et al.
(2004) to generate our temporal uncertainty scores s temporal .
We consider a sequence of r samples in the input data stream denoted by { x 1 , x 2 , ..., x t , ..., x r } .
We obtain the scalar spatial scores for the r samples, { s spatial ( x 1 ) , s spatial ( x 2 ) , ..., s spatial ( x t ) , ..., s spatial ( x r ) } from their representations { h ( x 1 ) , h ( x 2 ) , ..., h ( x t ) , ..., h ( x r ) } as explained in Section 3.2.1.
Given two window sizes w A and w B ( w A , w B < r ), we deﬁne a reference window R W and a sliding window ( S W ) t to be
[Equation]
where t ∈ { 1 , ..., ( r − w A − w B ) } as visualized in Fig.
On choosing an appropriate statistic or distance measure F (e.g. probability odds ratio, Kolmogorov-Smirnov, Kullback-Leibler, Wilcoxon,
Mann-Whitney, etc.)
, we compute the temporal uncertainty score between reference window R W and the t th sliding window ( S W ) t for the t th sample
[Equation]
The null hypothesis H 0 for our setting here is that the two windows (reference and sliding) are similar and come from the same distribution.
We hope to reject H 0 in favour of the alternate hypothesis Hat a signiﬁcance level of 0.05 whenever the inputs go out-of-distribution.
This approach (as in (Kifer et al., 2004)) assumes the data points are generated sequentially by some underlying probability distribution, but otherwise makes no assumptions on the nature of the generating distribution nor does it assume the samples are identically distributed.
E XPERIMENTAL S ETUP
We perform experiments across multiple domains (vision, audio and clinical EEG signals) to evaluate our approach.
We choose the task of image classiﬁcation for the visual domain, spoken word classiﬁcation from audio clips for the audio domain and the challenging, real-world clinical task of seizure detection from EEG signals framed as an EEG clip binary classiﬁcation task for the healthcare domain.
Our code will be made publicly available after publication.
We use a variety of datasets and models in our experiments.
For the audio domain, we use the
Free Spoken Digits Dataset (FSDD) (Jackson et al., 2017) and Google Speech Commands (GSC) dataset (Warden, 2018).
We train an M5 encoder (Dai et al., 2016) with raw audio InD data from spoken digits 0-9 from GSC and evaluate it on unseen InD samples from GSC 0-9 and FSDD 0along with OOD samples from the non-digit spoken words from GSC.
For our seizure tasks, we use
EEG data from Stanford Health Care (SHC) (InD: patient population 20-60 years), Lucile Packard
Children’s Hospital (LPCH) (OOD: patient population <20 years) and the public Temple University
Hospital seizure corpus (TUH) (Shah et al., 2018; Obeid & Picone, 2016)
(OOD: different patient demographics) along with a Dense-Inception encoder as described by Saab et al.
For our vision tasks, we use MNIST as InD and MNIST-like data as OOD and CIFAR10 as InD, SVHN as OOD and CIFAR100 as a semantic OOD evaluation set up.
Details on the data, model training, embedding extraction and temporal framework settings in Appendix B.1 and B.3.
We compare against six baselines: Maximum Softmax Probability (MSP) (Hendrycks & Gimpel, 2017), Predictive Entropy (Ren et al., 2019), KL Divergence with Uniform Distribution (Hendrycks et al., 2019a), ODIN (Liang et al., 2018), Vanilla Mahalanobis distance (Lee et al., 2018) and
Test-time Dropout (Gal & Ghahramani, 2016).
We provide baseline details in Appendix B.
E VALUATION
For all experiments, we evaluate our approach only using samples unseen during training.
Our test sets include unseen samples drawn from the same distribution as the training data along with samples from unseen classes and signiﬁcantly different datasets.
We also follow the setting recommended by Ahmed & Courville (2020) to avoid dataset bias by holding out few classes from a dataset during training and using the held-out classes as new semantic content for evaluation.
We evaluate
OOD detection performance using standard metrics: area under the receiver operating characteristic (AUROC ↑ ) , area under the precision-recall curve (AUPR ↑ ) and the false positive rate at 80% true positive rate (FPR80 ↓ ) as commonly prescribed (Ren et al., 2019).
In practice, it is important to note that what constitutes InD or OOD is driven by context, i.e., it depends on the target task, the encoder capacity, what our model needs to be robust to and what it should detect as outliers.
Standard OOD benchmarks treat different datasets to be OOD resulting in models with narrow capabilities susceptible to dataset bias (Ahmed & Courville, 2020; Yang et al., 2021).
Moreover, they fail to capture any semantic overlap between different datasets.
For example, if a model is trained to classify cats vs dogs and is given a different breed of a dog (high semantic overlap) from a different dataset at test time, it should still be considered InD whereas a mufﬁn would be OOD since it has no semantic overlap with the training data.
In this work, we evaluate OOD detection performance with and without semantic overlap and show that TIME-LAPSE is driven more by semantic content than other baselines.
We further propose a sequential evaluation framework to evaluate OOD detection under realistic conditions of data drifts.
We use the following setup: we evaluate performance metrics such as time to detect the change point, % error across trials, detection accuracy and false positive rate over
N = 1000 trials.
For each trial, we generate a stream of test data from our test sets with change points inserted according to a drift rate parameter r and fraction of OOD points given by the parameter λ .
We vary r and λ to emulate various drift conditions including rapidly changing streams and slowly shifting streams.
We also generate control streams by setting r = 0 to estimate the false positive rate.
R ESULTS 5. OOD D ETECTION P ERFORMANCE ON A UDIO , C LINICAL EEG AND I MAGE DATASETS
We achieve the state-of-the-art (SOTA) with TIME-LAPSE scores on OOD detection in both our audio and EEG-based tasks of spoken digits classiﬁcation and seizure detection compared to our baselines (Table 1) across all metrics: AUROC, AUPR and FPR80.
The AUROC values for TIME-LAPSE scores are 73.9 % and 77.1 % , 9 points and 12 points higher than the strongest baseline, Test-time
Dropout, for both tasks respectively.
In our MNIST experiments, we see TIME-LAPSE performs comparably with Test-Time Dropout.
It has the best AUPR and comparable AUROC and FPR values.
Over CIFAR10 experiments with the SVHN dataset (no semantic overlap with CIFAR10) as
OOD, TIME-LAPSE outperforms other baselines signiﬁcantly.
When CIFAR100 classes are added (considered OOD as in standard benchmarks) to the evaluation, TIME-LAPSE shows the best AUPR score and the second best AUROC and FPR80.
We examine the effect of semantic overlap present in
CIFAR100 classes with the InD classes as well as those of other domains in Section 5.2.
[Table]
[Table]
5. A FFINITY FOR S EMANTIC C ONTENT AND R OBUSTNESS TO D ATASET S TATISTICS
We study the effect of semantic overlap in OOD classes and characterize how TIME-LAPSE compares against other methods in its robustness to dataset statistics over multiple domains.
Audio
In our audio experiments, we train our encoder only on spoken digits 0-9 (forming the InD classes) from the Google Speech Commands (GSC) dataset.
The other 25 classes (Yes, No, Right,
Left, Bird, etc) in GSC have never been encountered by the model and have drastic semantic shifts in their content when compared to the InD classes 0-9 and are considered OOD.
During evaluation, we show the model unseen samples from each of the classes in GSC along with samples from the Free
Spoken Digits Dataset (FSDD) which contains classes 0-9 (same semantic content as InD data but different dataset) and measure OOD detection performance (Table 2).
As an ablation, we examine the performance when we ﬂip the FSDD classes to be OOD data in our evaluation.
We see that AUROC,
AUPR and FPR80 values for most baselines (notably Test-time Dropout and Predictive Entropy) increase signiﬁcantly, whereas they drop for TIME-LAPSE and vanilla Mahalanobis (Table 2).
This shows that our strongest baseline methods have narrow capabilities that assume only data belonging to InD classes and having the same dataset statistics as the training data will be considered as InD. All other data will be rejected as OOD, leading to very high FPRs in practice.
In contrast, TIME-LAPSE is able to identify true InD samples despite differing dataset statistics, i.e. TIME-LAPSE detects
OOD samples based on semantic content than dataset statistics when compared to the other methods.
Vision (CIFAR10)
For our CIFAR10 experiments, we train our encoder only on 7 classes of CIFAR10, leaving out classes Airplane, Bird and Dog.
SVHN data consists of images of numbers and has no semantic overlap with any of the 7 InD classes (Automobile, Cat, Deer, Frog, Horse, Ship, Truck).
To study the effect of semantic overlap, we consider the 100 classes from CIFAR100 along with the 3 left out classes from CIFAR10.
We compare the performance metrics when naively considering all of the above as OOD ignoring any semantic overlap.
We see that baselines perform better while
TIME-LAPSE drops (Table 1).
We analyse class-wise performance and we see that TIME-LAPSE’s performance strongly correlates to semantic content compared to other methods.
For example, consider Class Pickup-Truck and Streetcar from CIFAR100.
While they are disjoint from the InD classes Automobile and Truck in CIFAR10, they shares a high degree of semantic content and should be considered InD. When models encounter samples from classes that have high semantic overlap with InD, they need to consider them InD and not OOD.
Instead, if such samples are considered
OOD, only methods that have narrow model capacities will show high detection performance (Table 3) indicating their dependence on data statistics instead of semantic content.
Vision (MNIST)
We train two different encoders, one trained on all digits 0-9 while the other was trained on { 0 , 1 , 4 , 6 , 7 , 8 , 9 } leaving digits 2, 3 and 5.
Amongst our evaluations, only certain classes from the e-MNIST data shared semantic overlap – letters ‘o’,‘l’,‘i’,‘z’,‘y’,‘s’ and ‘q’ share structural similarity with classes 0, 1, 2, 4, 5 and 9 respectively.
We show results for both encoders with and without the above classes in the OOD evaluations (Table 4)
[Table]
EEGs While discussions on semantic issues on clinical EEGs may be out of scope for this paper
[Table]
we do observe interesting semantic effects in this case as well.
For instance, seizure types unseen by the network will generate higher uncertainty scores.
[Table] [Table]
[Table]
5. D RIFT D ETECTION : S EQUENTIAL E VALUATION R ESULTS
To evaluate the performance under our sequential framework [Table] we deﬁne the detector to have erred if it wrongly concludes an incoming test sample x t to be OOD when it is actually InD or vice-versa.
If the framework detects the change point within its ﬁxed window size, we do not consider it as an error.
We generate data streams of length 10,000 samples changing the distribution every k samples in each trial The distribution changes are simulated by ﬁrst randomly choosing InD or OOD (Bernoulli random variable with probability p ∈ { 0 . 2 , 0 . 5 , 0 . 7 } ) and then randomly drawing the k ∈ { 50 , 100 , 200 , 500 , 1000 , 5000 } samples from the chosen distribution.
We calculate the TIMELAPSE scores using the two-sided Mann-Whitney test (Mann & Whitney, 1947) at a signiﬁcance level of 0 .
for each sample in the stream and run it through our sequential detector with window sizes w A = w B = 25 for vision experiments and w A = w B = 50 for audio and EEG experiments.
We show example plots of datastreams for each domain in the Appendix, Figs. 2, 6, 7, 8, 9.
We perform a 2-cluster KMeans on the generated Mann-Whitney scores to assign InD and OOD predictions.
We then calculate the cumulative error by adding the errors of all samples to calculate detection error.
We repeat this experiment N times ( N = 1000 here) to estimate the detector’s error distribution (EEG distribution given in the Fig.
2, Appendix).
Over 73% of the cumulative errors in our EEG task (over 85% for our audio task) lie within 10% and over 93% of the errors in our EEG task (over 97% for our audio task) lie within 20% indicating our detector’s high performance in comparison to other spatial detectors.
For our vision tasks, the error distribution is much tighter and we are able to get near 99% detection accuracy for over 95% of the time.
5. A BLATIONS : E FFECT OF E NCODER CAPACITY
The proposed TIME-LAPSE framework depends on the encoder being well-trained and having good representational capability.
To study the effects of encoder capacity, we consider our audio task.
We compare two encoders, trained on the same InD data and evaluated with the same OOD data.
The lower capacity encoder, Kymatio, is a two-layer network with a log-scattering layer that extracts scattering coefﬁcients from the inputs as lower-dimensional latent space representations followed by a log-softmax layer for classiﬁcation with 0.75 classiﬁcation accuracy for InD test samples.
The higher capacity encoder is a CNN-based M5 model with 0.85 classiﬁcation accuracy for InD test samples.
As shown in Fig.
10 (Appendix), Kymatio is not able to separate out any of the classes based on the TIME-LAPSE scores whereas M5 is able to correctly separate out semantic InD samples from
OOD samples well.
Thus, the success of TIME-LAPSE is highly dependent on the representational capacity of the model.
To further investigate the effects of the lower capacity model, we train Kymatio on the much smaller
FSDD dataset but with the same InD semantic classes 0-9.
The model reaches 0.97 classiﬁcation accuracy, showing that it is able to express high-dimensional audio signals well.
However, on comparing its TIME-LAPSE scores with those of Kymatio trained on GSC 0-9 (Fig 11, Appendix), we see that the former separates out FSDD InD from GSC InD and
GSC OOD indicating that it is overﬁtting to the dataset statistics whereas the latter is not able to separate out any of them.
Thus,
TIME-LAPSE scores can help in determining both OOD detection capability of an encoder as well as generalization capability over OOD samples.
D ISCUSSION
In this work, we develop a unique uncertainty scoring framework that can identify when incoming inputs fall out-of-distribution with respect to trained encoders and subsequently enable models to abstain from prediction on such inputs.
Our theoretical formulation encompasses all representation-based
OOD detection methods and provides an opportunity to leverage temporal signals in combination.
We show that longitudinal information across time can be leveraged for the challenging task of identifying distributional shifts.
Through our experiments (Figs. 3, 4, 5, Appendix), we verify that temporal uncertainty scores and spatial uncertainty scores are complementary and combining them gives additional information than when used separately.
We proposed a simple sliding-window hypothesis-testing based approach to model temporal uncertainty of the reduced dimension representations and their spatial uncertainty scores.
But TIME-LAPSE can accept any type of spatial scoring and temporal scoring.
We believe that there is scope for developing sophisticated methods that leverage the temporal aspect and leave it open for future work.
Furthermore, alongside works by Ahmed & Courville (2020) and Yang et al. (2021), we motivate the necessity to rethink standard OOD image benchmarks that disregard semantic overlaps.
We note that the task of identifying OOD inputs is highly dependent on context.
For a classiﬁer designed to distinguish between cats and dogs, a tree is OOD whereas all of them would be InD for an object recognition system in a self-driving car.
We believe that showing generalizability of OOD detection methods across domains is key to ensure robust performance in the wild.
It is not sufﬁcient to show better performance over just image OOD benchmarks.
We show that TIME-LAPSE generalizes across 3 diverse domains- vision, audio and healthcare EEG, through our extensive experiments and analyses.
We also propose a sequential evaluation framework to view OOD detection from the lens of dataset drift detection.
To enable safe deployment and periodic monitoring of trained models, we believe that such a sequence-based evaluation scheme is essential.
We hope this evaluation scheme will be adopted for routinely characterizing uncertainty estimation and model performance in the wild.
From our ablation, we see that our method depends strongly on encoder capacity.
For our experiments, we train encoders with labelled InD data in a supervised setting.
Our framework will beneﬁt from more robust representations learnt via self-supervision or semi-supervision.
Also, we note that both the audio and the seizure analyses tasks generate much higher-dimensional inputs when compared to image classiﬁcation tasks.
We hypothesize that model performance is more strongly dependent on semantic content for such high-dimensional inputs when compared to dataset statistics and thus, gains from TIME-LAPSE are more obvious.
In conclusion, we provide a framework that utilizes both latent-space and time to determine predictive uncertainty for deep learning models.
Our method is end-to-end trainable and network agnostic: it (a) does not require exposure to outliers (though can easily be extended to accommodate them if present), (b) does not require modiﬁcations to existing network architectures, and (c) does not require computationally heavy dropout techniques or ensemble passes.
It can thus can be easily adopted to high-risk settings such as clinical workﬂows and autonomous driving.
E THICS S TATEMENT
We strongly believe that uncertainty quantiﬁcation and its associated tasks of out-of-distribution detection and dataset need to be benchmarked and evaluated on diverse datasets and domains aside from standard, well-curated, image datasets to reduce the effect of bias, variability, “easyness” of the task or domain.
To that effect, we have chosen three disparate domains (audio, vision and clinical
EEGs) and associated tasks (spoken word classiﬁcation, image classiﬁcation, seizure detection posed as binary classiﬁcation).
While this is an important and necessary step, we believe our work and the community at large will only beneﬁt from evaluating on more domains, datasets and tasks to ascertain a method’s success and failure modes to ensure model deployment.
We also hope to work with domain experts to validate TIME-LAPSE’s strengths and error modes for more real-world use cases.
All the data used in our work are either publicly available or are used with full IRB approval.
R EPRODUCIBILITY S TATEMENT
Our source code for TIME-LAPSE as well as our baseline implementations will be released to the public via Github upon publication.
The datasets used for our audio and vision tasks are publicly available: FSDD (Jackson et al., 2017), GSC (Warden, 2018), x -MNIST (Lecun et al., 1998;
Cohen et al., 2017; Tarin et al., 2018; Xiao et al., 2017), CIFAR10 & CIFAR100 (Krizhevsky & Hinton, 2009).
The Temple University Hospital EEG Seizure Corputs (TUH) used in our clinical EEG experiments is publicly available (Shah et al., 2018; Obeid & Picone, 2016).
EEG data from Stanford Healthcare (SHC) and Lucile Packard Children’s Hospital (LPCH) contain patient sensitive information and cannot be shared publicly.
Details on the data preprocessing steps and experimental settings are provided in Appendix B.
A PPENDIX A S UPPLEMENTARY T ABLES
[Table]
B D ETAILED E XPERIMENTAL S ETUP B. D ATASETS
Audio For the task of spoken word classiﬁcation of audio recordings, we use the Free Spoken Digits
Dataset (FSDD) (Jackson et al., 2017), consisting of .wav recordings of English digits 0-9, and the
Google Speech Commands (GSC) dataset (Warden, 2018) that contain .wav recordings of 35 English
[Table]
[Table] keywords including the digits 0-9 and non-digits such as “Yes", “No", “Stop", etc.
The two datasets provide a unique setup where digits 0-9 occur in different datasets but belong to the same semantic classes.
EEG Signals For our seizure analyses tasks, we use high-dimensional EEG data from different institutions and age groups: (a) Stanford Healthcare (SHC) with patients in the age group ∼ 20years, (b) Lucile Packard Children’s Hospital (LPCH) with patients under 18 years and (c) publicly available Temple University Hospital seizure corpus (TUH) (Shah et al., 2018; Obeid & Picone, 2016).
12 second or 60 second EEG clips with 19 channels, each sampled at 200Hz, are used for seizure detection.
Vision We use CIFAR-10 (InD), CIFAR-100 (Krizhevsky & Hinton, 2009) and SVHN (Netzer et al., 2011) datasets along with MNIST (Lecun et al., 1998) (InD), FashionMNIST (Xiao et al., 2017), eMNIST (Cohen et al., 2017) and kMNIST (Tarin et al., 2018)
(collectively referred to as x -MNIST) datasets for image classiﬁcation as is typically reported in literature (Hendrycks & Gimpel, 2017;
Lee et al., 2018; Ren et al., 2019).
B. B ASELINES
We use the following baselines to compare against our framework.
We don’t compare with methods that require exposure to outliers to match our framework settings.
MSP : Hendrycks & Gimpel (2017) use the maximum softmax probability (MSP) p (ˆ y | x ) = max k p ( y = k | x ) as the uncertainty score.
OOD inputs tend to have lower MSP than InD data.
Predictive Entropy : Ren et al.
(2019) show that using high entropy of the predicted class distribution − (cid:80) k p ( y = k | x ) log p ( y = k
| x ) as an indicator for OOD inputs is a strong baseline.
KL-divergence with Uniform distribution : Hendrycks et al.
(2019a) use the KL divergence of the softmax predictions to the uniform distribution U , KL ( U || p ( y | x )) to identify OOD inputs.
ODIN :
Liang et al.
(2018) use temperature-scaling and input perturbations to increase the gap between OOD and InD data using MSP.
We ﬁx T = 1000 , (cid:15) = 1 .
4 e − 3 , following their most common setting instead of tuning the parameters with outlier exposure.
Vanilla Mahalanobis : Lee et al.
(2018) use the Mahalanobis distance from the nearest classconditional Gaussian with shared covariance as the uncertainty score.
This approach directly ﬁts in with our spatial scoring method (Section 3.2.1) though we use class-wise covariance matrices.
Test-time Dropout : Gal & Ghahramani (2016) show that Monte Carlo (test-time) dropout to estimate the prediction distribution give good uncertainty estimates.
Note that it is computationally intensive requiring k forward passes of the classiﬁer, with k -fold increase in runtime.
We use k = 10 .
B. M ODEL T RAINING & H YPERPARAMETERS
Image Encoders & Pre-processing
We carry out our vision experiments using CNN-based classiﬁers, i.e. the LeNet architecture (Lecun et al., 1998) when working with x -MNIST data and the
ResNet18 architecture (He et al., 2016) when working with CIFAR10/100/SVHN data.
With the x -MNIST data, we normalize the inputs using the mean and standard deviation calculated from the training set for both training and inference, as is standard.
With the CIFAR10/100/SVHN data, we apply the normalization transform along with augmentation strategies (random crop and horizontal ﬂips) during training and only normalization during inference.
Audio Encoders & Pre-processing
For our audio experiments, we use the M5 classiﬁer (Dai et al., 2016) as our encoder.
We also train a simple two-layer network – a static, normalized, log-scattering layer that extracts scattering coefﬁcients from audio signals (Andreux et al., 2018) followed by a log-softmax layer that generates output probabilities to compare encoder capabilities (Section 5.4).
Both models are trained only on classes 0-9 from the GSC dataset with the rest of the classes as OOD inputs.
In all cases, we learn embeddings directly from raw, one second audio clips (resampled to 8kHz) without making use of any spectrogram features like MFCC (Xu et al., 2005).
EEG Encoders & Pre-processing
For our seizure detection task, we use the Dense-Inception based models (Saab et al., 2020) as encoders.
We use the same data pre-processing strategy as given by
Saab et al.
The models are
trained on 19 channel 12 second or 60 second raw EEG clips from InD datasets, resampled to 200Hz
Additional Details We train all encoders on InD training data using standard weight initializations,
SGD/Adam optimizer and ReLU non-linearities.
We train and tune hyperparameters for our encoders using only InD data samples.
We don’t expose any outlier data to our encoders.
We extract activations from a fully connected (FC) hidden layer before the logits layer to form latent space representations.
We normalize our distance scores and similarity scores before forming the spatial scores in cases where one of them is tightly bound while the other is not.
For our temporal uncertainty scoring, we set window sizes w A , w B = 25 for vision tasks and w A , w B = 50 for audio and EEG tasks based on hyper-parameter tuning.
We choose the reference window by sampling randomly from the training set (or coreset) and using its spatial scores in any order.
We randomly shufﬂe the evaluation set and send in data points sequentially.
We ﬁrst obtain the spatial uncertainty scores for the evaluation set forming a 1D evaluation sequence, on which we form the sliding window sequentially.
We choose the non-parametric, two-sided Mann-Whitney test (Mann & Whitney, 1947) at a signiﬁcance level of 0 .
as our measure F .
We perform two-centered kMeans clustering (MacQueen, 1967) on the temporal score sequence to identify uncertain model predictions and reject corresponding test samples as OOD.
C S EQUENTIAL E VALUATION : V ISUALIZATIONS
Some examples of data streams generated from sampling EEG, audio and vision datasets are shown below (Figs. 2, 3, 4, 5, 6, 7, 8 and 9).
The The X axis indicates the time sequence.
The Y axis gives the uncertainty scores (Mann Whitney Scores are the same as TIME-LAPSE temporal scores).
Using a two cluster KMeans algorithm, change points were detected and individual samples were accepted or rejected as InD.
D S UPPLEMENTARY F IGURES : E FFECT OF E NCODER CAPACITY
We visualize the distributions of TIME-LAPSE spatial scores generated by two encoder architectures (Kymatio and M5) with varying modelling capacity.
We see from Fig.
10 that more modelling capacity an encoder has, the better it is able to separate InD and OOD samples in its latent space.
Note that in Fig.
10, the left plot shows the distribution of scores for InD samples from different datasets in blue and orange and that of all the OOD samples together in green.
In the right plot,
In Fig.
11, we keep the architecture of the encoder ﬁxed, i.e., Kymatio, and vary how well it is trained.
We can two encoders of the same architecture, train one on GSC 0-9 classes and another on FSDD 0-9 classes.
Both the encoders are thus trained on the same InD classes but different datasets.
The model trained on FSDD is able to reach high accuracy on the FSDD test sets whereas the one trained on GSC cannot.
From the plots, we see that the model trained on FSDD actually overﬁts to the data since it is able to distinguish between InD classes from different datasets (Fig.
right).
E Q UALITATIVE A NALYSES
We plot the UMAP (McInnes et al., 2018) embeddings of our raw InD data and their extracted latent space representations from the trained encoders along with those of OOD data.
Fig.
12 shows the plots for our audio GSC dataset.
We see that the raw data are too high-dimensional to be captured according to their class labels by UMAP.
On the other hand, the extracted representations arrange themselves in well-separable clusters showing the representational capacity of the trained encoder.
Most distance-based OOD detectors assume OOD representations will get mapped far away from the InD representations in the latent-space.
This is a good assumption and holds in many cases.
Indeed, the spatial scorer in TIME-LAPSE uses this hypothesis (along with others) to derive its predictive uncertainty score.
However, OOD inputs can get mapped very close to InD inputs in the representational space under most distance metrics.
Fig.
12 (right) shows the overlay of OOD inputs in the latent-space.
Although it is just a 2D projection and the extracted representations have higher dimensions, this ﬁgure still shows that OOD inputs can get mapped near to InD inputs.
In that case, purely distance-based spatial scores lose their power.
TIME-LAPSE looks at multiple hypotheses to determine if samples are OOD.
We ﬁnd that the combination of our similarity score and distance score outperform each score individually.
Moreover, since TIME-LAPSE has a temporal component to it, the temporal scorer acts as an additional constraint to ensure OOD inputs are separable.
Thus, within a few samples, TIME-LAPSE can indicate when samples are going out-of-distribution with much better accuracy.