Original: D EEP B ATCH
Revised: D EEP B ATCH

Original: A CTIVE A NOMALY D ETECTION
Revised: A CTIVE A NOMALY D ETECTION

Original: W ITH D IVERSE Q UERIES
Revised: W ITH D IVERSE Q UERIES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Selecting informative data points for expert feedback can significantly improve the performance of anomaly detection in various contexts, such as medical diagnostics or fraud detection.
Revised: Selecting informative data points for expert feedback can significantly improve the performance of anomaly detection in various contexts, such as medical diagnostics or fraud detection.

Original: In this paper, we determine a set of conditions under which the ranking of anomaly scores generalizes from labeled queries to unlabeled data.
Revised: In this paper, we determine a set of conditions under which the ranking of anomaly scores generalizes from labeled queries to unlabeled data.

Original: Inspired by these conditions, we propose a new querying strategy for batch active anomaly detection that leads to systematic improvements over current approaches.
Revised: Inspired by these conditions, we propose a new querying strategy for batch active anomaly detection that leads to systematic improvements over current approaches.

Original: It selects a diverse set of data points for labeling, achieving high data coverage with a limited budget.
Revised: It selects a diverse set of data points for labeling, achieving high data coverage with a limited budget.

Original: These labeled data points provide weak supervision to the unsupervised anomaly detection problem.
Revised: These labeled data points provide weak supervision to the unsupervised anomaly detection problem.

Original: However, correctly identifying anomalies in the contaminated training data requires an estimate of the contamination ratio.
Revised: However, correctly identifying anomalies in the contaminated training data requires an estimate of the contamination ratio.

Original: We show how this anomaly rate can be estimated from the query set by importance-weighting, removing the associated bias due to the non-uniform sampling procedure.
Revised: We show how this anomaly rate can be estimated from the query set by importance-weighting, removing the associated bias due to the non-uniform sampling procedure.

Original: Extensive experiments on image, tabular, and video data sets show that our approach results in state-of-the-art active anomaly detection performance.
Revised: Extensive experiments on image, tabular, and video data sets show that our approach results in state-of-the-art active anomaly detection performance.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Detecting anomalies in data is a fundamental task in machine learning with applications in various domains, from industrial fault detection to medical diagnosis.
Revised: Detecting anomalies in data is a fundamental task in machine learning with applications in various domains, from industrial fault detection to medical diagnosis.

Original: The main idea is to train a model (such as a neural network) on a data set of “normal” samples to minimize the loss of an auxiliary (e.g., self-supervised) task.
Revised: The main idea is to train a model (such as a neural network) on a data set of “normal” samples to minimize the loss of an auxiliary (e.g., self-supervised) task.

Original: Using the loss function to score test data, one hopes to obtain low scores for normal data and high scores for anomalies (Ruff et al., 2021).
Revised: Using the loss function to score test data, one hopes to obtain low scores for normal data and high scores for anomalies (Ruff et al., 2021).

Original: Oftentimes, the training data is contaminated with unlabeled anomalies, and many approaches either hope that training will be dominated by the normal samples ( inlier priority , Wang et al.
Revised: Oftentimes, the training data is contaminated with unlabeled anomalies, and many approaches either hope that training will be dominated by the normal samples ( inlier priority , Wang et al.

Original: or try to detect and exploit anomalies in the training data (e.g., Qiu et al.
Revised: or try to detect and exploit anomalies in the training data (e.g., Qiu et al.

Original: (2022a)).
Revised: (2022a)).

Original: In some set-ups, expert feedback is available to check if individual samples are normal or should be considered anomalies.
Revised: In some set-ups, expert feedback is available to check if individual samples are normal or should be considered anomalies.

Original: These labels are usually expensive to obtain but are very valuable to guide an anomaly detector during training.
Revised: These labels are usually expensive to obtain but are very valuable to guide an anomaly detector during training.

Original: For example, in a medical setting, one may ask a medical doctor to confirm whether a given image shows normal or abnormal cellular tissue.
Revised: For example, in a medical setting, one may ask a medical doctor to confirm whether a given image shows normal or abnormal cellular tissue.

Original: Other application areas include detecting network intrusions or machine failures.
Revised: Other application areas include detecting network intrusions or machine failures.

Original: As expert feedback is typically expensive, it is essential to find effective strategies for querying informative data points.
Revised: As expert feedback is typically expensive, it is essential to find effective strategies for querying informative data points.

Original: Previous work on active anomaly detection primarily involves domain-specific applications and/or ad hoc architectures, making it hard to disentangle modeling choices from querying strategies (Trittenbach et al., 2021).
Revised: Previous work on active anomaly detection primarily involves domain-specific applications and/or ad hoc architectures, making it hard to disentangle modeling choices from querying strategies (Trittenbach et al., 2021).

Original: This paper aims to disentangle different factors that affect detection accurary.
Revised: This paper aims to disentangle different factors that affect detection accurary.

Original: We theoretically analyze generalization performance under various querying strategies and find that diversified sampling systematically improve over existing popular querying strategies, such as querying data based on their predicted anomaly score or around the decision boundaries.
Revised: We theoretically analyze generalization performance under various querying strategies and find that diversified sampling systematically improve over existing popular querying strategies, such as querying data based on their predicted anomaly score or around the decision boundaries.

Original: Based on these findings, we propose active latent outlier exposure (ALOE): a state-of-the-art active learning strategy compatible with many unsupervised and self-supervised losses for anomaly detection (Ruff et al., 2021;
Revised: Based on these findings, we propose active latent outlier exposure (ALOE): a state-of-the-art active learning strategy compatible with many unsupervised and self-supervised losses for anomaly detection (Ruff et al., 2021;

Original: Qiu et al., 2022a).
Revised: Qiu et al., 2022a).

Original: ALOE draws information from both queried and unqueried parts of the data based on two equally-weighted losses.
Revised: ALOE draws information from both queried and unqueried parts of the data based on two equally-weighted losses.

Original: Its sole hyperparameter—the assumed anomaly rate—can be efficiently estimated based on an importance sampling estimate.
Revised: Its sole hyperparameter—the assumed anomaly rate—can be efficiently estimated based on an importance sampling estimate.

Original: We show on a multitude of data sets (images, tabular data, and video) that ALOE leads to a new state of the art.
Revised: We show on a multitude of data sets (images, tabular data, and video) that ALOE leads to a new state of the art.

Original: In summary, our main contributions are as follows:
Revised: In summary, our main contributions are as follows:

Original: We prove that the ranking of anomaly scores generalizes from labeled queries to unlabeled data under certain conditions that characterize how well the queries cover the data.
Revised: We prove that the ranking of anomaly scores generalizes from labeled queries to unlabeled data under certain conditions that characterize how well the queries cover the data.

Original: Based on this theory, we propose a diverse querying strategy for batch active anomaly detection.
Revised: Based on this theory, we propose a diverse querying strategy for batch active anomaly detection.

Original: We propose ALOE, a new active learning framework compatible with a large number of deep anomaly detection losses.
Revised: We propose ALOE, a new active learning framework compatible with a large number of deep anomaly detection losses.

Original: It trains on both the labeled queries and the unlabeled data and we characterize in Thm.
Revised: It trains on both the labeled queries and the unlabeled data and we characterize in Thm.

Original: 1 how the performance on the queried samples generalizes to the unlabeled training data.
Revised: 1 how the performance on the queried samples generalizes to the unlabeled training data.

Original: We also show how all major hyperparameters in ALOE can be eliminated, making the approach easy to use.
Revised: We also show how all major hyperparameters in ALOE can be eliminated, making the approach easy to use.

Original: To this end, we provide an importance-sampling-based estimate for the rate of anomalies in the data.
Revised: To this end, we provide an importance-sampling-based estimate for the rate of anomalies in the data.

Original: We provide an extensive benchmark for deep active anomaly detection.
Revised: We provide an extensive benchmark for deep active anomaly detection.

Original: Our experiments on image, tabular, and video data provide evidence that ALOE with diverse querying, outperforms existing methods significantly.
Revised: Our experiments on image, tabular, and video data provide evidence that ALOE with diverse querying, outperforms existing methods significantly.

Original: Comprehensive ablations disentangle the benefits of each component.
Revised: Comprehensive ablations disentangle the benefits of each component.

Original: Our paper is structured as follows.
Revised: Our paper is structured as follows.

Original: Section 2 discusses related work in deep active anomaly detection.
Revised: Section 2 discusses related work in deep active anomaly detection.

Original: Section 3 introduces our main algorithm.
Revised: Section 3 introduces our main algorithm.

Original: Section 4 discusses experimental result on image, video, and tabular data.
Revised: Section 4 discusses experimental result on image, video, and tabular data.

Original: Finally, we conclude this work in Section 5.
Revised: Finally, we conclude this work in Section 5.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Deep Anomaly Detection.
Revised: Deep Anomaly Detection.

Original: Many recent advances in anomaly detection are in the area of deep learning Ruff et al.
Revised: Many recent advances in anomaly detection are in the area of deep learning Ruff et al.

Original: One early strategy was to use autoencoder- (Principi et al., 2017;
Revised: One early strategy was to use autoencoder- (Principi et al., 2017;

Original: Zhou and
Revised: Zhou and

Original: Paffenroth, 2017; Chen and Konukoglu, 2018) or density-based (Schlegl et al., 2017; Deecke et al., 2018) models.
Revised: Paffenroth, 2017; Chen and Konukoglu, 2018) or density-based (Schlegl et al., 2017; Deecke et al., 2018) models.

Original: Another pioneering stream of research combines one-class classification (Schölkopf et al., 2001) with deep learning for unsupervised (Ruff et al., 2018) and semi-supervised (Ruff et al., 2019) anomaly detection.
Revised: Another pioneering stream of research combines one-class classification (Schölkopf et al., 2001) with deep learning for unsupervised (Ruff et al., 2018) and semi-supervised (Ruff et al., 2019) anomaly detection.

Original: Many other approaches to deep anomaly detection are self-supervised.
Revised: Many other approaches to deep anomaly detection are self-supervised.

Original: They employ a self-supervised loss function to train the detector and score anomalies (Golan and
Revised: They employ a self-supervised loss function to train the detector and score anomalies (Golan and

Original: El-Yaniv, 2018; Hendrycks et al., 2019; Bergman and Hoshen, 2020; Qiu et al., 2021; Shenkar and
Revised: El-Yaniv, 2018; Hendrycks et al., 2019; Bergman and Hoshen, 2020; Qiu et al., 2021; Shenkar and

Original: Wolf, 2022; Qiu et al., 2022b).
Revised: Wolf, 2022; Qiu et al., 2022b).

Original: Our work resides in the self-supervised anomaly detection category and can be extended to other data modalities if an appropriate loss is provided.
Revised: Our work resides in the self-supervised anomaly detection category and can be extended to other data modalities if an appropriate loss is provided.

Original: While all these methods assume that the training data consists of only normal samples, in many practical applications, the training pool may be contaminated with unidentified anomalies (Vilhjálmsson and Nordborg, 2013; Steinhardt et al., 2017).
Revised: While all these methods assume that the training data consists of only normal samples, in many practical applications, the training pool may be contaminated with unidentified anomalies (Vilhjálmsson and Nordborg, 2013; Steinhardt et al., 2017).

Original: This can be problematic because the detection accuracy typically deteriorates when the contamination ratio increases (Wang et al., 2019).
Revised: This can be problematic because the detection accuracy typically deteriorates when the contamination ratio increases (Wang et al., 2019).

Original: Addressing this, refinement (Zhou and Paffenroth, 2017; Yoon et al., 2021) attempts to cleanse the training pool by removing anomalous samples therein, although they may provide valuable training signals.
Revised: Addressing this, refinement (Zhou and Paffenroth, 2017; Yoon et al., 2021) attempts to cleanse the training pool by removing anomalous samples therein, although they may provide valuable training signals.

Original: As a remedy, Qiu et al.
Revised: As a remedy, Qiu et al.

Original: (2022a) propose to jointly infer binary labels to each datum (normal vs. anomalous) while updating the model parameters of a deep anomaly detector based on outlier exposure.
Revised: (2022a) propose to jointly infer binary labels to each datum (normal vs. anomalous) while updating the model parameters of a deep anomaly detector based on outlier exposure.

Original: Our work also makes the contaminated data assumption and employs the training signal of abnormal data.
Revised: Our work also makes the contaminated data assumption and employs the training signal of abnormal data.

Original: Active Anomaly Detection.
Revised: Active Anomaly Detection.

Original: Active learning for anomaly detection was pioneered in Pelleg and
Revised: Active learning for anomaly detection was pioneered in Pelleg and

Original: Moore (2004).
Revised: Moore (2004).

Original: Most research in active anomaly detection is on shallow detectors.
Revised: Most research in active anomaly detection is on shallow detectors.

Original: Many works query samples that are close to the decision boundary of a one-class SVM (Görnitz et al., 2013; Yin et al., 2018) or a density model (Ghasemi et al., 2011).
Revised: Many works query samples that are close to the decision boundary of a one-class SVM (Görnitz et al., 2013; Yin et al., 2018) or a density model (Ghasemi et al., 2011).

Original: Siddiqui et al.
Revised: Siddiqui et al.

Original: Das et al.
Revised: Das et al.

Original: (2016) propose to query the most anomalous instance, while Das et al.
Revised: (2016) propose to query the most anomalous instance, while Das et al.

Original: (2019) employ a tree-based ensemble to query both anomalous and diverse samples.
Revised: (2019) employ a tree-based ensemble to query both anomalous and diverse samples.

Original: A recent survey compares various aforementioned query strategies applied to one-class classifiers (Trittenbach et al., 2021).
Revised: A recent survey compares various aforementioned query strategies applied to one-class classifiers (Trittenbach et al., 2021).

Original: Recently, deep active anomaly detection has received a lot of attention.
Revised: Recently, deep active anomaly detection has received a lot of attention.

Original: Pimentel et al.
Revised: Pimentel et al.

Original: (2020) query samples with the top anomaly scores for autoencoder-based methods, while Ning et al.
Revised: (2020) query samples with the top anomaly scores for autoencoder-based methods, while Ning et al.

Original: (2022) improve the querying by taking diversity into consideration.
Revised: (2022) improve the querying by taking diversity into consideration.

Original: Tang et al.
Revised: Tang et al.

Original: (2020) use an ensemble of deep anomaly detectors and query the most likely anomalies for each detector separately.
Revised: (2020) use an ensemble of deep anomaly detectors and query the most likely anomalies for each detector separately.

Original: Russo et al.
Revised: Russo et al.

Original: (2020) query samples where the model is uncertain about the predictions.
Revised: (2020) query samples where the model is uncertain about the predictions.

Original: Pang et al.
Revised: Pang et al.

Original: (2021) and Zha et al.
Revised: (2021) and Zha et al.

Original: (2020) propose querying strategies based on reinforcement learning, which requires additional labeled datasets.
Revised: (2020) propose querying strategies based on reinforcement learning, which requires additional labeled datasets.

Original: In contrast, our work does not require a labeled dataset to start active learning.
Revised: In contrast, our work does not require a labeled dataset to start active learning.

Original: Our work is also not comparable to Pelleg and Moore (2004) and Ghasemi et al.
Revised: Our work is also not comparable to Pelleg and Moore (2004) and Ghasemi et al.

Original: (2011), who fit a density model on the raw input data, which is known to be problematic for high-dimensional data (Nalisnick et al., 2018).
Revised: (2011), who fit a density model on the raw input data, which is known to be problematic for high-dimensional data (Nalisnick et al., 2018).

Original: Other querying strategies from the papers discussed above are fairly general and can be applied in combination with various backbone models.
Revised: Other querying strategies from the papers discussed above are fairly general and can be applied in combination with various backbone models.

Original: In this paper we study these methods in the batch active anomaly detection setting.
Revised: In this paper we study these methods in the batch active anomaly detection setting.

Original: For fair comparison, we apply their meta-strategies to the same state-of-the-art backbone model that our proposed method uses in our experiments (see details in Tab. 1 and Sec. 4).
Revised: For fair comparison, we apply their meta-strategies to the same state-of-the-art backbone model that our proposed method uses in our experiments (see details in Tab. 1 and Sec. 4).

Original: Batch Active Learning.
Revised: Batch Active Learning.

Original: Like Hoi et al.
Revised: Like Hoi et al.

Original: (2006a)
Revised: (2006a)

Original: our work considers the batch active anomaly detection setting.
Revised: our work considers the batch active anomaly detection setting.

Original: Here it is assumed that labeling can be performed only once.
Revised: Here it is assumed that labeling can be performed only once.

Original: Either because interacting with the expert for labeling is too expensive or because retraining the model with each additional query is too inefficient (Hoi et al., 2006a).
Revised: Either because interacting with the expert for labeling is too expensive or because retraining the model with each additional query is too inefficient (Hoi et al., 2006a).

Original: The batch setting is widely studied in deep active learning (Sener and Savarese, 2018; Ash et al., 2020; Citovsky et al., 2021; Pinsler et al., 2019;
Revised: The batch setting is widely studied in deep active learning (Sener and Savarese, 2018; Ash et al., 2020; Citovsky et al., 2021; Pinsler et al., 2019;

Original: Hoi et al., 2006b; Guo and Schuurmans, 2007) but is still under-explored in active anomaly detection.
Revised: Hoi et al., 2006b; Guo and Schuurmans, 2007) but is still under-explored in active anomaly detection.

Original:  M ETHODS
Revised:  M ETHODS

Original: A successful method for active anomaly detection will involve a strategy for selecting informative data points to be labeled – these are called the queries – and a training strategy that best exploits the labeled (and unlabeled) data.
Revised: A successful method for active anomaly detection will involve a strategy for selecting informative data points to be labeled – these are called the queries – and a training strategy that best exploits the labeled (and unlabeled) data.

Original: We first review recent deep learning losses for anomaly detection (Sec. 3.1), before proposing a new approach for batch active anomaly detection in Secs.
Revised: We first review recent deep learning losses for anomaly detection (Sec. 3.1), before proposing a new approach for batch active anomaly detection in Secs.

Original: 3.2 to 3.4.
Revised: 3.2 to 3.4.

Original: In Sec. 3.2, we analyze querying strategies for active anomaly detection.
Revised: In Sec. 3.2, we analyze querying strategies for active anomaly detection.

Original: We determine a set of conditions that guarantee that the ranking in terms of the anomaly score provably generalizes from the labeled queries to the unlabeled data.
Revised: We determine a set of conditions that guarantee that the ranking in terms of the anomaly score provably generalizes from the labeled queries to the unlabeled data.

Original: Based on our theory, we derive a diverse querying strategy.
Revised: Based on our theory, we derive a diverse querying strategy.

Original: Once the queries are labeled, we can use the union of the queried data and the unlabeled data to train the model.
Revised: Once the queries are labeled, we can use the union of the queried data and the unlabeled data to train the model.

Original: Our proposed loss is presented in Sec. 3.3.
Revised: Our proposed loss is presented in Sec. 3.3.

Original: This approach has an important hyperparameter α which corresponds to the expected fraction of anomalies in the data.
Revised: This approach has an important hyperparameter α which corresponds to the expected fraction of anomalies in the data.

Original: While previous work has to assume that α is known (Qiu et al., 2022a), active learning presents an opportunity to estimate it.
Revised: While previous work has to assume that α is known (Qiu et al., 2022a), active learning presents an opportunity to estimate it.

Original: The estimate has to account for the fact that the optimal querying strategy derived from our theory in
Revised: The estimate has to account for the fact that the optimal querying strategy derived from our theory in

Original: Sec. 3.2 is not i.i.d..
Revised: Sec. 3.2 is not i.i.d..

Original: In Sec. 3.4, we provide an estimate of α for any stochastic querying strategy.
Revised: In Sec. 3.4, we provide an estimate of α for any stochastic querying strategy.

Original: 3. B ACKGROUND
Revised: 3. B ACKGROUND

Original: Active Anomaly Detection.
Revised: Active Anomaly Detection.

Original: We assume a dataset where each sample is associated with a feature x ∈ R d and a binary anomaly label y .
Revised: We assume a dataset where each sample is associated with a feature x ∈ R d and a binary anomaly label y .

Original: The label y = 0 stands for normal and y = 1 stands for abnormal data.
Revised: The label y = 0 stands for normal and y = 1 stands for abnormal data.

Original: While most samples are normal, the dataset is contaminated with a fraction of α anomalies.
Revised: While most samples are normal, the dataset is contaminated with a fraction of α anomalies.

Original: The goal is to learn a parametric function S ( x ) that associates every input x with a real-valued anomaly score.
Revised: The goal is to learn a parametric function S ( x ) that associates every input x with a real-valued anomaly score.

Original: A high anomaly score is predictive of y = 1 and a low anomaly score is predictive of y = 0 .
Revised: A high anomaly score is predictive of y = 1 and a low anomaly score is predictive of y = 0 .

Original: To train S , one has access to unlabeled data with index set denoted by U and labeled data with index set denoted by Q .
Revised: To train S , one has access to unlabeled data with index set denoted by U and labeled data with index set denoted by Q .

Original: The number of queries corresponds to the labeling budget |Q| .
Revised: The number of queries corresponds to the labeling budget |Q| .

Original: Once S is trained, it can be used to rank new samples according to their anomaly score.
Revised: Once S is trained, it can be used to rank new samples according to their anomaly score.

Original: Supervised Anomaly Detection.
Revised: Supervised Anomaly Detection.

Original: One possibility is to use only the labeled data to train S in a supervised manner (Görnitz et al., 2013; Hendrycks et al., 2018).
Revised: One possibility is to use only the labeled data to train S in a supervised manner (Görnitz et al., 2013; Hendrycks et al., 2018).

Original: It requires two related loss functions with shared parameters θ parameterizing the function S .
Revised: It requires two related loss functions with shared parameters θ parameterizing the function S .

Original: The first loss L θ 0 ( x ) ≡ L 0 ( S ( x ; θ )) only involves normal samples, while L θ 1 ( x ) ≡ L 1 ( S ( x ; θ )) is the loss for the anomalies.
Revised: The first loss L θ 0 ( x ) ≡ L 0 ( S ( x ; θ )) only involves normal samples, while L θ 1 ( x ) ≡ L 1 ( S ( x ; θ )) is the loss for the anomalies.

Original: Options for
Revised: Options for

Original: L θ 0 ( x ) include an autoencoder loss, the Deep SVDD loss, or a neural transformation learning objective (Zhou and Paffenroth, 2017; Ruff et al., 2018; Qiu et al., 2021).
Revised: L θ 0 ( x ) include an autoencoder loss, the Deep SVDD loss, or a neural transformation learning objective (Zhou and Paffenroth, 2017; Ruff et al., 2018; Qiu et al., 2021).

Original: The loss L θ 1 ( x ) for abnormal data, is typically chosen to have a complementary effect 1 .
Revised: The loss L θ 1 ( x ) for abnormal data, is typically chosen to have a complementary effect 1 .

Original: With the labeled queries Q , the supervised loss is
Revised: With the labeled queries Q , the supervised loss is

Original: [Equation]
Revised: [Equation]

Original: Latent Outlier Exposure.
Revised: Latent Outlier Exposure.

Original: Latent outlier exposure (LOE, (Qiu et al., 2022a)) is an unsupervised anomaly detection framework that uses the same loss as Eq.
Revised: Latent outlier exposure (LOE, (Qiu et al., 2022a)) is an unsupervised anomaly detection framework that uses the same loss as Eq.

Original: (1) but treats the labels y as latent variables.
Revised: (1) but treats the labels y as latent variables.

Original: An EM-style algorithm alternates between optimizing the model parameters θ and inferring the labels y .
Revised: An EM-style algorithm alternates between optimizing the model parameters θ and inferring the labels y .

Original: For this, Qiu et al.
Revised: For this, Qiu et al.

Original: (2022a) have to assume that the ranking of data anomaly scores is correct and the contamination ratio α is known.
Revised: (2022a) have to assume that the ranking of data anomaly scores is correct and the contamination ratio α is known.

Original: In this work, we propose active latent outlier exposure (ALOE).
Revised: In this work, we propose active latent outlier exposure (ALOE).

Original: We next present the querying strategy and when the querying strategy leads to correct ranking of anomaly scores (Sec. 3.2), the ALOE loss (Sec. 3.3), and how the hyperparameter α can be eliminated (Sec. 3.4).
Revised: We next present the querying strategy and when the querying strategy leads to correct ranking of anomaly scores (Sec. 3.2), the ALOE loss (Sec. 3.3), and how the hyperparameter α can be eliminated (Sec. 3.4).

Original: 3. A CTIVE Q UERYING FOR A NOMALY D ETECTION
Revised: 3. A CTIVE Q UERYING FOR A NOMALY D ETECTION

Original: The first ingredient of an active anomaly detection method is a querying strategy for selecting informative data points to be labeled.
Revised: The first ingredient of an active anomaly detection method is a querying strategy for selecting informative data points to be labeled.

Original: Several strategies have been proposed in the literature.
Revised: Several strategies have been proposed in the literature.

Original: As a simple baseline, data points can be queried at random (Ruff et al., 2019); however, this oftentimes results in very few queried true anomalies.
Revised: As a simple baseline, data points can be queried at random (Ruff et al., 2019); however, this oftentimes results in very few queried true anomalies.

Original: Görnitz et al.
Revised: Görnitz et al.

Original: (2013) propose to query at the boundary of the normal region, which is the learned hypersphere surface in the feature space.
Revised: (2013) propose to query at the boundary of the normal region, which is the learned hypersphere surface in the feature space.

Original: A more general approach is to query data in the α -quantile of an anomaly detector, but the problem is that one typically does not know which value of α defines the decision boundary.
Revised: A more general approach is to query data in the α -quantile of an anomaly detector, but the problem is that one typically does not know which value of α defines the decision boundary.

Original: An important property of the querying strategy is how well it covers unlabeled data.
Revised: An important property of the querying strategy is how well it covers unlabeled data.

Original: The quality of a querying strategy is determined by the smallest radius δ such that all unlabeled points are within distance δ of one queried sample of the same type.
Revised: The quality of a querying strategy is determined by the smallest radius δ such that all unlabeled points are within distance δ of one queried sample of the same type.

Original: In this paper, we prove that if the queries cover both the normal data and the anomalies well (i.e., if δ is small), a learned anomaly detector that satisfies certain conditions is guaranteed to generalize correctly to the unlabeled data (The exact statement and its conditions will be provided in Thm. 1).
Revised: In this paper, we prove that if the queries cover both the normal data and the anomalies well (i.e., if δ is small), a learned anomaly detector that satisfies certain conditions is guaranteed to generalize correctly to the unlabeled data (The exact statement and its conditions will be provided in Thm. 1).

Original: Based on this insight, we propose to use a querying strategy that is better suited for active anomaly detection than previous work.
Revised: Based on this insight, we propose to use a querying strategy that is better suited for active anomaly detection than previous work.

Original: Theorem 1.
Revised: Theorem 1.

Original: Let Q 0 be the index set of datapoints labeled normal and Q 1 the index set of datapoints labeled abnormal.
Revised: Let Q 0 be the index set of datapoints labeled normal and Q 1 the index set of datapoints labeled abnormal.

Original: Let δ ∈ R + be the smallest radius, such that for each unlabeled anomaly u a and each unlabeled normal datum u n there exist labeled data points x a , a ∈ Q 1 and x n , n ∈ Q 0 , such that u a is within the δ -ball of x a and u n is within the δ -ball around x n .
Revised: Let δ ∈ R + be the smallest radius, such that for each unlabeled anomaly u a and each unlabeled normal datum u n there exist labeled data points x a , a ∈ Q 1 and x n , n ∈ Q 0 , such that u a is within the δ -ball of x a and u n is within the δ -ball around x n .

Original: If a λ s -Lipschitz continuous function S ranks the labeled data correctly, with a large enough margin, i.e. S ( x a ) − S ( x n ) ≥ 2 δλ s , then S ranks the unlabeled points correctly too and S ( u a ) ≥ S ( u n ) .
Revised: If a λ s -Lipschitz continuous function S ranks the labeled data correctly, with a large enough margin, i.e. S ( x a ) − S ( x n ) ≥ 2 δλ s , then S ranks the unlabeled points correctly too and S ( u a ) ≥ S ( u n ) .

Original: Proof.
Revised: Proof.

Original: Since S is λ s -Lipschitz continuous and u a and u n are assumed to be closer than δ to x a and x n respectively, we have S ( x a ) − δλ s ≤ S ( u a ) and − S ( x n ) − δλ s ≤ − S ( u n ) .
Revised: Since S is λ s -Lipschitz continuous and u a and u n are assumed to be closer than δ to x a and x n respectively, we have S ( x a ) − δλ s ≤ S ( u a ) and − S ( x n ) − δλ s ≤ − S ( u n ) .

Original: Adding the inequalities and using the condition S ( x a ) − S ( x n ) ≥ 2 δλ s , yields 0 ≤ S ( x a ) − S ( x n ) − 2 δλ s ≤
Revised: Adding the inequalities and using the condition S ( x a ) − S ( x n ) ≥ 2 δλ s , yields 0 ≤ S ( x a ) − S ( x n ) − 2 δλ s ≤

Original: S ( u a ) − S ( u n ) , which proves S ( u a ) ≥ S ( u n ) .
Revised: S ( u a ) − S ( u n ) , which proves S ( u a ) ≥ S ( u n ) .

Original: An implication of this theorem is that a smaller δ corresponding to a tighter cover of the data leads to better generalization performance.
Revised: An implication of this theorem is that a smaller δ corresponding to a tighter cover of the data leads to better generalization performance.

Original: Given a fixed query budget |Q| , random querying puts too much weight on high-density areas of the data space, while other strategies only query locally, e.g., close to the possible decision boundary between normal and abnormal data.
Revised: Given a fixed query budget |Q| , random querying puts too much weight on high-density areas of the data space, while other strategies only query locally, e.g., close to the possible decision boundary between normal and abnormal data.

Original: Instead, we propose a querying strategy that encourages diversity: k-means++ , the seeding algorithm that initializes diverse cluster centers.
Revised: Instead, we propose a querying strategy that encourages diversity: k-means++ , the seeding algorithm that initializes diverse cluster centers.

Original: It iteratively samples another data point to be added to the query set Q until the labeling budget is reached.
Revised: It iteratively samples another data point to be added to the query set Q until the labeling budget is reached.

Original: Given the existing queried samples, the probability of drawing a sample from the unlabeled set U is proportional to its distance to the closest sample in the query set:
Revised: Given the existing queried samples, the probability of drawing a sample from the unlabeled set U is proportional to its distance to the closest sample in the query set:

Original: [Equation]
Revised: [Equation]

Original: (2) where τ is a temperature parameter controlling the strength of diversity, h ( x i )
Revised: (2) where τ is a temperature parameter controlling the strength of diversity, h ( x i )

Original: =
Revised: =

Original: min x j ∈Q d ( x i , x j ) is the distance of a sample x i to the query set Q .
Revised: min x j ∈Q d ( x i , x j ) is the distance of a sample x i to the query set Q .

Original: For a more meaningful notion of distance, we define the latter in an embedding space as d ( x , x ′ ) =
Revised: For a more meaningful notion of distance, we define the latter in an embedding space as d ( x , x ′ ) =

Original: ∥ ϕ ( x ) − ϕ
Revised: ∥ ϕ ( x ) − ϕ

Original: ( x ′ )
Revised: ( x ′ )

Original: ∥ 2 , where ϕ is a neural feature map.
Revised: ∥ 2 , where ϕ is a neural feature map.

Original: We stress that all deep methods considered in this paper have an associated feature map that we can use.
Revised: We stress that all deep methods considered in this paper have an associated feature map that we can use.

Original: In Supp.
Revised: In Supp.

Original: D.5, we report the time complexity.
Revised: D.5, we report the time complexity.

Original: In Supp.
Revised: In Supp.

Original: A, we empirically validate that diverse querying leads to smaller δ than others and is therefore advantageous for active anomaly detection.
Revised: A, we empirically validate that diverse querying leads to smaller δ than others and is therefore advantageous for active anomaly detection.

Original: 3. A CTIVE L ATENT O UTLIER E XPOSURE (ALOE) L OSS
Revised: 3. A CTIVE L ATENT O UTLIER E XPOSURE (ALOE) L OSS

Original: We next consider how to use both labeled and unlabeled samples in training.
Revised: We next consider how to use both labeled and unlabeled samples in training.

Original: We propose active latent outlier exposure (ALOE).
Revised: We propose active latent outlier exposure (ALOE).

Original: Its loss combines the unsupervised anomaly detection loss of LOE (Qiu et al., 2022a) for the unlabeled data with the supervised loss (Eq.
Revised: Its loss combines the unsupervised anomaly detection loss of LOE (Qiu et al., 2022a) for the unlabeled data with the supervised loss (Eq.

Original: (1)) for the labeled samples.
Revised: (1)) for the labeled samples.

Original: For all queried data (with index set Q ), we assume that ground truth labels y
Revised: For all queried data (with index set Q ), we assume that ground truth labels y

Original: i are available, while for unqueried data (with index set U ), the labels ˜
Revised: i are available, while for unqueried data (with index set U ), the labels ˜

Original: y i are unknown.
Revised: y i are unknown.

Original: Adding both losses together yields
Revised: Adding both losses together yields

Original: L ( θ, ˜ y )
Revised: L ( θ, ˜ y )

Original: = 1 |Q| (cid:88)
Revised: = 1 |Q| (cid:88)

Original: [Equation]
Revised: [Equation]

Original: Similar to Qiu et al.
Revised: Similar to Qiu et al.

Original: (2022a), optimizing this loss involves a block coordinate ascent scheme that alternates between inferring the unknown labels and taking gradient steps to minimize Eq.
Revised: (2022a), optimizing this loss involves a block coordinate ascent scheme that alternates between inferring the unknown labels and taking gradient steps to minimize Eq.

Original: (3) with the inferred labels.
Revised: (3) with the inferred labels.

Original: In each iteration, the pseudo labels ˜
Revised: In each iteration, the pseudo labels ˜

Original: y
Revised: y

Original: i for i ∈
Revised: i for i ∈

Original: U are obtained by minimizing
Revised: U are obtained by minimizing

Original: Eq.
Revised: Eq.

Original: (3) subject to a constraint of (cid:80) i ∈Q
Revised: (3) subject to a constraint of (cid:80) i ∈Q

Original: y i + (cid:80) i ∈U ˜ y
Revised: y i + (cid:80) i ∈U ˜ y

Original: i = αN .
Revised: i = αN .

Original: The constraint ensures that the inferred anomaly labels respect a certain contamination ratio α .
Revised: The constraint ensures that the inferred anomaly labels respect a certain contamination ratio α .

Original: In practice, this constrained optimization problem is solved by using the current anomaly score function S to rank the unlabeled samples and assign the top α -quantile of the associated labels y
Revised: In practice, this constrained optimization problem is solved by using the current anomaly score function S to rank the unlabeled samples and assign the top α -quantile of the associated labels y

Original: i to the value 1 , and the remaining to the value 0 .
Revised: i to the value 1 , and the remaining to the value 0 .

Revised: Thm. 1 applies in ranking the unlabeled samples by the anomaly score function S .

Original: In theory, α could be treated as a hyperparameter, but eliminating hyperparameters is important in anomaly detection.
Revised: In theory, α could be treated as a hyperparameter, but eliminating hyperparameters is important in anomaly detection.

Original: In many practical applications of anomaly detection, there is no labeled data that can be used for validation.
Revised: In many practical applications of anomaly detection, there is no labeled data that can be used for validation.

Original: While Qiu et al.
Revised: While Qiu et al.

Original: (2022a) have to assume that the contamination ratio is given, active learning provides an opportunity to estimate α .
Revised: (2022a) have to assume that the contamination ratio is given, active learning provides an opportunity to estimate α .

Original: In Sec. 3.4, we develop an importancesampling based approach to estimate the contamination ratio from the labeled data.
Revised: In Sec. 3.4, we develop an importancesampling based approach to estimate the contamination ratio from the labeled data.

Original: Estimating this ratio can be beneficial for many anomaly detection algorithms, including OC-SVM (Schölkopf et al., 2001), kNN (Ramaswamy et al., 2000), Robust PCA/Auto-encoder (Zhou and Paffenroth, 2017), and Soft-boundary deep SVDD (Ruff et al., 2018).
Revised: Estimating this ratio can be beneficial for many anomaly detection algorithms, including OC-SVM (Schölkopf et al., 2001), kNN (Ramaswamy et al., 2000), Robust PCA/Auto-encoder (Zhou and Paffenroth, 2017), and Soft-boundary deep SVDD (Ruff et al., 2018).

Original: When working with contaminated data, these algorithms require a decent estimate of the contamination ratio for good performance.
Revised: When working with contaminated data, these algorithms require a decent estimate of the contamination ratio for good performance.

Original: Another noteworthy aspect of the ALOE loss is that it weighs the averaged losses equally to each other.
Revised: Another noteworthy aspect of the ALOE loss is that it weighs the averaged losses equally to each other.

Original: While equal weighting can not be justified from the first principles, we expect our queried samples to be informative, and yet we do not want either loss component to dominate the learning task (assuming that we always query only a small percentage of the data).
Revised: While equal weighting can not be justified from the first principles, we expect our queried samples to be informative, and yet we do not want either loss component to dominate the learning task (assuming that we always query only a small percentage of the data).

Original: In Supp.
Revised: In Supp.

Original: E.5, we empirically show that equal weighting yields the best results among a large range of various weights.
Revised: E.5, we empirically show that equal weighting yields the best results among a large range of various weights.

Original: This provides more weight to every queried data point than to an unqueried one, assuring these goals.
Revised: This provides more weight to every queried data point than to an unqueried one, assuring these goals.

Original: Our equal weighting scheme is also practical because it avoids a hyperparameter.
Revised: Our equal weighting scheme is also practical because it avoids a hyperparameter.

Original: 3. C ONTAMINATION R ATIO E STIMATION .
Revised: 3. C ONTAMINATION R ATIO E STIMATION .

Original: To eliminate a critical hyperparameter in our approach, we estimate the contamination ratio α , i.e., the fraction of anomalies in the dataset.
Revised: To eliminate a critical hyperparameter in our approach, we estimate the contamination ratio α , i.e., the fraction of anomalies in the dataset.

Original: Under a few assumptions, we show that this parameter can be estimated despite of the result of Thm. 1, which requires us to use a biased querying strategy.
Revised: Under a few assumptions, we show that this parameter can be estimated despite of the result of Thm. 1, which requires us to use a biased querying strategy.

Original: We consider the contamination ratio α = E p ( x ) [ 1 a ( x )] , where p ( x ) is the data distribution anda ( x ) ≡ 1 ( y ( x )=1) indicates whether or not the sample is abnormal.
Revised: We consider the contamination ratio α = E p ( x ) [ 1 a ( x )] , where p ( x ) is the data distribution anda ( x ) ≡ 1 ( y ( x )=1) indicates whether or not the sample is abnormal.

Original: Estimating this quantity would be trivial given an unlimited budget of queries.
Revised: Estimating this quantity would be trivial given an unlimited budget of queries.

Original: However, since active learning setups involve limited budgets, we aim to estimate this quantity only based on the queried samples Q used to train the model.
Revised: However, since active learning setups involve limited budgets, we aim to estimate this quantity only based on the queried samples Q used to train the model.

Original: Since the queried data points are not independently sampled, we cannot straightforwardly estimate α based on the empirical frequency of anomalies in Q .
Revised: Since the queried data points are not independently sampled, we cannot straightforwardly estimate α based on the empirical frequency of anomalies in Q .

Original: More precisely, our querying procedure results in a chain of indices Q
Revised: More precisely, our querying procedure results in a chain of indices Q

Original: = { i 1 , i 2 , ..., i |Q| } , where i 1 ∼ Unif(1 : N) , and each conditional distribution
Revised: = { i 1 , i 2 , ..., i |Q| } , where i 1 ∼ Unif(1 : N) , and each conditional distribution

Original: i k | i <k is defined by Eq.
Revised: i k | i <k is defined by Eq.

Original: We will show as follows that this sampling bias can be compensated using importance sampling.
Revised: We will show as follows that this sampling bias can be compensated using importance sampling.

Original: As follows, we will state two assumptions about our querying strategy, followed by a theorem for calculating α .
Revised: As follows, we will state two assumptions about our querying strategy, followed by a theorem for calculating α .

Original: Justifications for the two assumptions will be provided below.
Revised: Justifications for the two assumptions will be provided below.

Original: Assumption 1.
Revised: Assumption 1.

Original: The anomaly scores { S ( x i ) : i ∈ Q } in a query set Q are approximately independently distributed, i.e., their off-diagonal correlations are negligible.
Revised: The anomaly scores { S ( x i ) : i ∈ Q } in a query set Q are approximately independently distributed, i.e., their off-diagonal correlations are negligible.

Original: Assumption 2.
Revised: Assumption 2.

Original: Let q ( x i ) denote the marginal distribution of a queried sample under the querying scheme Q , and let p ( x ) denote the data distribution.
Revised: Let q ( x i ) denote the marginal distribution of a queried sample under the querying scheme Q , and let p ( x ) denote the data distribution.

Original: Then, the anomaly scores S ( x ) are a sufficient statistic of both p ( x ) and q ( x i ) .
Revised: Then, the anomaly scores S ( x ) are a sufficient statistic of both p ( x ) and q ( x i ) .

Original: That is, there exist functions f ( · ) and g ( · ) such that p ( x ) =
Revised: That is, there exist functions f ( · ) and g ( · ) such that p ( x ) =

Original: f ( S ( x )) and q ( x i ) =
Revised: f ( S ( x )) and q ( x i ) =

Original: g ( S ( x i )) for all x and x i , with negligible error.
Revised: g ( S ( x i )) for all x and x i , with negligible error.

Original: Both Assumptions 1 and 2 are only approximations to reality.
Revised: Both Assumptions 1 and 2 are only approximations to reality.

Original: We will provide strong evidence below that they are well-justified working hypotheses.
Revised: We will provide strong evidence below that they are well-justified working hypotheses.

Original: The following theorem is a consequence of them:
Revised: The following theorem is a consequence of them:

Original: Theorem 2.
Revised: Theorem 2.

Original: Given Assumptions 1 and 2, the following statement applies: for a random query Q , the following importance sampling estimator provides an unbiased estimate of the contamination ratio:
Revised: Given Assumptions 1 and 2, the following statement applies: for a random query Q , the following importance sampling estimator provides an unbiased estimate of the contamination ratio:

Original: [Equation]
Revised: [Equation]

Original: The proof is in Supp.
Revised: The proof is in Supp.

Original: B. In practice, we learn f and g using a kernel density estimate in the onedimensional space of anomaly scores.
Revised: B. In practice, we learn f and g using a kernel density estimate in the onedimensional space of anomaly scores.

Original: We set the bandwidth to the average spacing of scores.
Revised: We set the bandwidth to the average spacing of scores.

Original: In conclusion, Thm. 2 allows us to estimate the importance ratio based on a non-iid query set Q .
Revised: In conclusion, Thm. 2 allows us to estimate the importance ratio based on a non-iid query set Q .

Original: Discussion.
Revised: Discussion.

Original: We empirically verified the fact that Thm.
Revised: We empirically verified the fact that Thm.

Original: 2 results in reliable estimates for varying contamination ratios in Supp.
Revised: 2 results in reliable estimates for varying contamination ratios in Supp.

Original: B.4.
Revised: B.4.

Original: Since Assumptions 1 and 2 seem strong, we discuss their justifications and empirical validity next.
Revised: Since Assumptions 1 and 2 seem strong, we discuss their justifications and empirical validity next.

Original: We tested Assumption 1 empirically and found negligible off-diagonal correlations among the anomaly scores in the query set.
Revised: We tested Assumption 1 empirically and found negligible off-diagonal correlations among the anomaly scores in the query set.

Original: In Supp.
Revised: In Supp.

Original: B.2, we show that the absolute off-diagonal coefficient values are significantly smaller than one on CIFAR-10.
Revised: B.2, we show that the absolute off-diagonal coefficient values are significantly smaller than one on CIFAR-10.

Original: We can understand this based on a dimensionality argument.
Revised: We can understand this based on a dimensionality argument.

Original: Since we sample repulsively in a high-dimensional space but project the data onto the one-dimensional space of scores, the resulting correlations will vanish upon the projection step.
Revised: Since we sample repulsively in a high-dimensional space but project the data onto the one-dimensional space of scores, the resulting correlations will vanish upon the projection step.

Original: We tested Assumption 2 empirically in Supp.
Revised: We tested Assumption 2 empirically in Supp.

Original: B.3.
Revised: B.3.

Original: We show that there is a monotonic relationship between S ( x ) and log p ( x ) (or log q ( x ) ).
Revised: We show that there is a monotonic relationship between S ( x ) and log p ( x ) (or log q ( x ) ).

Original: We validated both on 2D synthetic data from Gaussian distribution and on CIFAR-10.
Revised: We validated both on 2D synthetic data from Gaussian distribution and on CIFAR-10.

Original: Scatter plots in Supp.
Revised: Scatter plots in Supp.

Original: B show that S ( x ) is predictive of both p ( x ) and q ( x ) and thus S ( x ) is a sufficient statistic of p ( x ) and q ( x ) .
Revised: B show that S ( x ) is predictive of both p ( x ) and q ( x ) and thus S ( x ) is a sufficient statistic of p ( x ) and q ( x ) .

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We study batch active anomaly detection on standard image benchmarks, medical images, tabular data, and surveillance videos.
Revised: We study batch active anomaly detection on standard image benchmarks, medical images, tabular data, and surveillance videos.

Original: Our extensive empirical study establishes how our proposed method compares to eight recent active anomaly detection methods we implemented as baselines.
Revised: Our extensive empirical study establishes how our proposed method compares to eight recent active anomaly detection methods we implemented as baselines.

Original: We first describe the baselines and their implementations (Tab. 1) and then the experiments on images (Sec. 4.1), tabular data (Sec. 4.2), videos (Sec. 4.3) and finally additional experiments (Sec. 4.4).
Revised: We first describe the baselines and their implementations (Tab. 1) and then the experiments on images (Sec. 4.1), tabular data (Sec. 4.2), videos (Sec. 4.3) and finally additional experiments (Sec. 4.4).

Original: Baselines.
Revised: Baselines.

Original: Most existing baselines apply their proposed querying and training strategies to shallow anomaly detection methods or sub-optimal deep models (e.g., autoencoders (Zhou and Paffenroth, 2017)).
Revised: Most existing baselines apply their proposed querying and training strategies to shallow anomaly detection methods or sub-optimal deep models (e.g., autoencoders (Zhou and Paffenroth, 2017)).

Original: In recent years, these approaches have consistently been outperformed by self-supervised anomaly detection methods (Hendrycks et al., 2019).
Revised: In recent years, these approaches have consistently been outperformed by self-supervised anomaly detection methods (Hendrycks et al., 2019).

Original: For a fair comparison of active anomaly detection frameworks, we endow all baselines with the same self-supervised backbone models also used in our method.
Revised: For a fair comparison of active anomaly detection frameworks, we endow all baselines with the same self-supervised backbone models also used in our method.

Original: By default we use neural transformation learning (NTL) (Qiu et al., 2021) as the backbone model, which was identified as state-of-the-art in a recent independent comparison ofmodels (Alvarez et al., 2022).
Revised: By default we use neural transformation learning (NTL) (Qiu et al., 2021) as the backbone model, which was identified as state-of-the-art in a recent independent comparison ofmodels (Alvarez et al., 2022).

Original: Results with other backbone models are shown in Supp.
Revised: Results with other backbone models are shown in Supp.

Original: E.1.
Revised: E.1.

Original: The baselines are summarized in Tab. 1 and detailed in Supp.
Revised: The baselines are summarized in Tab. 1 and detailed in Supp.

Original: C. They differ in their querying strategies (col. 3) and training strategies (col. 4 & 5): the unlabeled data is either ignored or modeled with a one-class objective.
Revised: C. They differ in their querying strategies (col. 3) and training strategies (col. 4 & 5): the unlabeled data is either ignored or modeled with a one-class objective.

Original: Most baselines incorporate the labeled data by a supervised loss (Eq.
Revised: Most baselines incorporate the labeled data by a supervised loss (Eq.

Original: As an exception, Ning et al.
Revised: As an exception, Ning et al.

Original: (2022) remove all queried anomalies and then train a weighted one-class objective on the remaining data.
Revised: (2022) remove all queried anomalies and then train a weighted one-class objective on the remaining data.

Original: All baselines weigh the unsupervised and supervised losses equally.
Revised: All baselines weigh the unsupervised and supervised losses equally.

Original: The baselines also differ in their querying strategies, summarized below:
Revised: The baselines also differ in their querying strategies, summarized below:

Original: • margin query selects samples deterministically that are the closest to the boundary of the normality region.
Revised: • margin query selects samples deterministically that are the closest to the boundary of the normality region.

Original: We provide the method with the true contamination ratio to help it choose an ideal boundary.
Revised: We provide the method with the true contamination ratio to help it choose an ideal boundary.

Original: • margin diverse query combines margin query with neighborhood-based diversification.
Revised: • margin diverse query combines margin query with neighborhood-based diversification.

Original: It tends to select samples that are not in the k -nearest neighbors of the queried set.
Revised: It tends to select samples that are not in the k -nearest neighbors of the queried set.

Original: Thus samples are ensured to sit at different positions of the normality boundary.
Revised: Thus samples are ensured to sit at different positions of the normality boundary.

Original: • most positive query always selects the top-ranked samples ordered by their anomaly scores.
Revised: • most positive query always selects the top-ranked samples ordered by their anomaly scores.

Original: • positive diverse query combines querying according to anomaly scores with distance-based diversification.
Revised: • positive diverse query combines querying according to anomaly scores with distance-based diversification.

Original: The selection criterion is an equally weighted combination of the two aspects: anomaly score and the minimum Euclidean distance to all previously queried samples.
Revised: The selection criterion is an equally weighted combination of the two aspects: anomaly score and the minimum Euclidean distance to all previously queried samples.

Original: • random query draws samples uniformly among the training set.
Revised: • random query draws samples uniformly among the training set.

Original: • positive random query samples uniformly among the top 50% data ranked by anomaly scores.
Revised: • positive random query samples uniformly among the top 50% data ranked by anomaly scores.

Original: Implementation Details.
Revised: Implementation Details.

Original: In all experiments, we use a NTL (Qiu et al., 2021) backbone model for all methods.
Revised: In all experiments, we use a NTL (Qiu et al., 2021) backbone model for all methods.

Original: Experiments with other backbone models are shown in Supp.
Revised: Experiments with other backbone models are shown in Supp.

Original: E.1.
Revised: E.1.

Original: On images and videos,
Revised: On images and videos,

Original: NTL is built upon the penultimate layer output of a frozen ResNet-152 pre-trained on ImageNet.
Revised: NTL is built upon the penultimate layer output of a frozen ResNet-152 pre-trained on ImageNet.

Original: NTL is trained for one epoch, after which all queries are labeled.
Revised: NTL is trained for one epoch, after which all queries are labeled.

Original: The number of queries |Q| depends
Revised: The number of queries |Q| depends

Original: [Table]
Revised: [Table]

Original: on the experiment and all |Q| queries are collected at once.
Revised: on the experiment and all |Q| queries are collected at once.

Original: The contamination ratio α in ALOE is estimated immediately after the querying step and then fixed for the remaining training process.
Revised: The contamination ratio α in ALOE is estimated immediately after the querying step and then fixed for the remaining training process.

Original: We follow Qiu et al.
Revised: We follow Qiu et al.

Original: (2022a)
Revised: (2022a)

Original: and set ˜ y i = 0 . 5 for inferred anomalies.
Revised: and set ˜ y i = 0 . 5 for inferred anomalies.

Original: This accounts for the uncertainty of whether the sample truly is an anomaly.
Revised: This accounts for the uncertainty of whether the sample truly is an anomaly.

Original: More details are provided in Supp.
Revised: More details are provided in Supp.

Original: D.
Revised: D.

Original: 4. E XPERIMENTS ON I MAGE D ATA
Revised: 4. E XPERIMENTS ON I MAGE D ATA

Original: In this section
Revised: In this section

Original: [Table]
Revised: [Table]

Original: we study ALOE on standard image benchmarks to establish how it compares to eight well-known active anomaly detection baselines.
Revised: we study ALOE on standard image benchmarks to establish how it compares to eight well-known active anomaly detection baselines.

Original: Active learning plays an important role in medical domains where expert labeling is expensive.
Revised: Active learning plays an important role in medical domains where expert labeling is expensive.

Original: Hence, we also study nine medical datasets from Yang et al.
Revised: Hence, we also study nine medical datasets from Yang et al.

Original: We describe the datasets, the evaluation protocol, and finally the results of our study.
Revised: We describe the datasets, the evaluation protocol, and finally the results of our study.

Original: Image Benchmarks.
Revised: Image Benchmarks.

Original: We experiment with two popular image benchmarks: CIFAR-10 and FashionMNIST.
Revised: We experiment with two popular image benchmarks: CIFAR-10 and FashionMNIST.

Original: These have been widely used in previous papers on deep anomaly detection (Ruff et al., 2018; Golan and El-Yaniv, 2018; Hendrycks et al., 2019; Bergman and Hoshen, 2020).
Revised: These have been widely used in previous papers on deep anomaly detection (Ruff et al., 2018; Golan and El-Yaniv, 2018; Hendrycks et al., 2019; Bergman and Hoshen, 2020).

Original: Medical Images.
Revised: Medical Images.

Original: Since medical imaging is an important practical application of active anomaly detection, we also study ALOE on medical images.
Revised: Since medical imaging is an important practical application of active anomaly detection, we also study ALOE on medical images.

Original: The datasets we consider cover different data modalities (e.g., X-ray, CT, electron microscope) and their characteristic image features can be very different from natural images.
Revised: The datasets we consider cover different data modalities (e.g., X-ray, CT, electron microscope) and their characteristic image features can be very different from natural images.

Original: Our empirical study includes all 2D image datasets presented in Yang et al.
Revised: Our empirical study includes all 2D image datasets presented in Yang et al.

Original: (2021) that have more than 500 samples in each class, including Blood, OrganA, OrganC,
Revised: (2021) that have more than 500 samples in each class, including Blood, OrganA, OrganC,

Original: OrganS, OCT, Pathology, Pneumonia, and Tissue.
Revised: OrganS, OCT, Pathology, Pneumonia, and Tissue.

Original: We also include Dermatoscope but restricted to the classes with more than 500 training samples.
Revised: We also include Dermatoscope but restricted to the classes with more than 500 training samples.

Original: Evaluation Protocol.
Revised: Evaluation Protocol.

Original: We follow the community standard known as the “one-vs.-rest” protocol to turn these classification datasets into a test-bed for anomaly detection (Ruff et al., 2018; Golan and
Revised: We follow the community standard known as the “one-vs.-rest” protocol to turn these classification datasets into a test-bed for anomaly detection (Ruff et al., 2018; Golan and

Original: El-Yaniv, 2018; Hendrycks et al., 2019; Bergman and Hoshen, 2020).
Revised: El-Yaniv, 2018; Hendrycks et al., 2019; Bergman and Hoshen, 2020).

Original: While respecting the original train and test split of these datasets, the protocol iterates over the classes and treats each class in turn as normal.
Revised: While respecting the original train and test split of these datasets, the protocol iterates over the classes and treats each class in turn as normal.

Original: Random samples from the other classes are used to contaminate the data.
Revised: Random samples from the other classes are used to contaminate the data.

Original: The training set is then a mixture of unlabeled normal and abnormal samples with a contamination ratio of 10% (Wang et al., 2019; Qiu et al., 2022a).
Revised: The training set is then a mixture of unlabeled normal and abnormal samples with a contamination ratio of 10% (Wang et al., 2019; Qiu et al., 2022a).

Original: This protocol can simulate a “human expert” to provide labels for the queried samples because the datasets provide ground-truth class labels.
Revised: This protocol can simulate a “human expert” to provide labels for the queried samples because the datasets provide ground-truth class labels.

Original: The models are trained on the training data and evaluated on the test data.
Revised: The models are trained on the training data and evaluated on the test data.

Original: The reported results (in terms of AUC % ) for each dataset are averaged over the number of experiments (i.e., classes) and over five independent runs.
Revised: The reported results (in terms of AUC % ) for each dataset are averaged over the number of experiments (i.e., classes) and over five independent runs.

Original: Results.
Revised: Results.

Original: We report the evaluation results of our method (ALOE) and the eight baselines on all eleven image datasets in Tab. 2.
Revised: We report the evaluation results of our method (ALOE) and the eight baselines on all eleven image datasets in Tab. 2.

Original: All methods have a query budget of 20 samples.
Revised: All methods have a query budget of 20 samples.

Original: On all datasets, our proposed method ALOE achieves the best performance and significantly outperforms all baselines by 6 percentage points on average, reaching the state-of-the-art results in active anomaly detection.
Revised: On all datasets, our proposed method ALOE achieves the best performance and significantly outperforms all baselines by 6 percentage points on average, reaching the state-of-the-art results in active anomaly detection.

Original: In addition, we also study detection performance as the query budget increases from 20 to 160 .
Revised: In addition, we also study detection performance as the query budget increases from 20 to 160 .

Original: The results are plotted in Fig.
Revised: The results are plotted in Fig.

Original: The results show that, with a small budget of 20 samples, ALOE (by querying diverse and informative samples) makes better usage of the labeling information than the other baselines and thus leads to better performance by a large margin.
Revised: The results show that, with a small budget of 20 samples, ALOE (by querying diverse and informative samples) makes better usage of the labeling information than the other baselines and thus leads to better performance by a large margin.

Original: As more samples are allowed
Revised: As more samples are allowed

Original: [Table]
Revised: [Table]

Original: to be queried, the performance of almost all methods increases but even for |Q| = 160 queries the added benefit from adding more queries starts to saturate, ALOE still outperforms the baselines.
Revised: to be queried, the performance of almost all methods increases but even for |Q| = 160 queries the added benefit from adding more queries starts to saturate, ALOE still outperforms the baselines.

Original: 4. E XPERIMENTS ON T ABULAR D ATA
Revised: 4. E XPERIMENTS ON T ABULAR D ATA

Original: Many practical use cases of anomaly detection (e.g., in health care or cyber security) are concerned with tabular data.
Revised: Many practical use cases of anomaly detection (e.g., in health care or cyber security) are concerned with tabular data.

Original: For this reason, we study ALOE on several tabular datasets from various domains.
Revised: For this reason, we study ALOE on several tabular datasets from various domains.

Original: We find that it outperforms existing baselines, even with as few as 10 queries.
Revised: We find that it outperforms existing baselines, even with as few as 10 queries.

Original: We also confirmed the fact that our deep models are competitive with classical methods for tabular data in Supp.
Revised: We also confirmed the fact that our deep models are competitive with classical methods for tabular data in Supp.

Original: E.7.
Revised: E.7.

Original: Tabular Datasets.
Revised: Tabular Datasets.

Original: Our study includes the four multi-dimensional tabular datasets from the ODDS repository which have an outlier ratio of at least 30% .
Revised: Our study includes the four multi-dimensional tabular datasets from the ODDS repository which have an outlier ratio of at least 30% .

Original: This is necessary to ensure that there are enough anomalies available to remove from the test set and add to the clean training set (which is randomly sub-sampled to half its size) to achieve a contamination ratio of 10% .
Revised: This is necessary to ensure that there are enough anomalies available to remove from the test set and add to the clean training set (which is randomly sub-sampled to half its size) to achieve a contamination ratio of 10% .

Original: The datasets are BreastW, Ionosphere, Pima, and Satellite.
Revised: The datasets are BreastW, Ionosphere, Pima, and Satellite.

Original: As in the image experiments, there is one round of querying, in which 10 samples are labeled.
Revised: As in the image experiments, there is one round of querying, in which 10 samples are labeled.

Original: For each dataset, we report the averaged F1-score (%) with standard deviations over five runs with random train-test splits and random initialization.
Revised: For each dataset, we report the averaged F1-score (%) with standard deviations over five runs with random train-test splits and random initialization.

Original: Results.
Revised: Results.

Original: Our proposed method ALOE performs best on 3 of 4 datasets and outperforms all baselines by 3.2 percentage points on average, reaching state-of-the-art results in active anomaly detection.
Revised: Our proposed method ALOE performs best on 3 of 4 datasets and outperforms all baselines by 3.2 percentage points on average, reaching state-of-the-art results in active anomaly detection.

Original: [Table]
Revised: [Table]

Original: Diverse querying best utilizes the limited query budget to label the diverse and informative data points, yielding a consistent improvement over existing baselines also on tabular data
Revised: Diverse querying best utilizes the limited query budget to label the diverse and informative data points, yielding a consistent improvement over existing baselines also on tabular data

Original: [Table]
Revised: [Table]

Original: 4. E XPERIMENTS ON V IDEO D ATA
Revised: 4. E XPERIMENTS ON V IDEO D ATA

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Detecting unusual objects in surveillance videos is an important application area for anomaly detection.
Revised: Detecting unusual objects in surveillance videos is an important application area for anomaly detection.

Original: Due to the large variability in abnormal objects and suspicious behavior in surveillance videos, expert feedback is very valuable to train an anomaly detector (we use NTL as the backbone model) in an active manner.
Revised: Due to the large variability in abnormal objects and suspicious behavior in surveillance videos, expert feedback is very valuable to train an anomaly detector (we use NTL as the backbone model) in an active manner.

Original: We study ALOE on a public surveillance video dataset (UCSD Peds1).
Revised: We study ALOE on a public surveillance video dataset (UCSD Peds1).

Original: The goal is to accurately detect abnormal video frames that contain non-pedestrian objects with a limited query budget.
Revised: The goal is to accurately detect abnormal video frames that contain non-pedestrian objects with a limited query budget.

Original: We make the same independent frames assumption as Pang et al.
Revised: We make the same independent frames assumption as Pang et al.

Original: and mix the original training and testing video frames together.
Revised: and mix the original training and testing video frames together.

Original: As in Pang et al.
Revised: As in Pang et al.

Original: (2020), we sub-sample 6800 normal frames and 2914 abnormal frames for training such that the anomaly ratio is 0.3, which corresponds to the original ratio in this dataset.
Revised: (2020), we sub-sample 6800 normal frames and 2914 abnormal frames for training such that the anomaly ratio is 0.3, which corresponds to the original ratio in this dataset.

Original: The other samples are used for testing.
Revised: The other samples are used for testing.

Original: Before running any of the methods, a ResNet pretrained on ImageNet is used to obtain a fixed feature vector for each frame.
Revised: Before running any of the methods, a ResNet pretrained on ImageNet is used to obtain a fixed feature vector for each frame.

Original: We vary the query budget from |Q| = 20 to |Q| = 160 and compare ALOE to all baselines.
Revised: We vary the query budget from |Q| = 20 to |Q| = 160 and compare ALOE to all baselines.

Original: All experiments are repeated five times with different random initializations.
Revised: All experiments are repeated five times with different random initializations.

Original: Results in terms of average
Revised: Results in terms of average

Original: AUC and standard error over the five runs are reported in Fig.
Revised: AUC and standard error over the five runs are reported in Fig.

Original: ALOE consistently outperforms all baselines, especially for smaller querying budgets.
Revised: ALOE consistently outperforms all baselines, especially for smaller querying budgets.

Original: 4. A DDITIONAL E XPERIMENTS
Revised: 4. A DDITIONAL E XPERIMENTS

Original: In Supp.
Revised: In Supp.

Original: E, we provide several additional experiments and ablations demonstrating ALOE’s strong performance and justifying modeling choices.
Revised: E, we provide several additional experiments and ablations demonstrating ALOE’s strong performance and justifying modeling choices.

Original: In summary, we address the following aspects:
Revised: In summary, we address the following aspects:

Original: • Backbone Models.
Revised: • Backbone Models.

Original: Tab.
Revised: Tab.

Original: 6 shows that ALOE is the best performing active learning framework also for the backbone models MHRot (Hendrycks et al., 2019) and Deep SVDD (Ruff et al., 2018).
Revised: 6 shows that ALOE is the best performing active learning framework also for the backbone models MHRot (Hendrycks et al., 2019) and Deep SVDD (Ruff et al., 2018).

Original: • ALOE Loss Components.
Revised: • ALOE Loss Components.

Original: Tab.
Revised: Tab.

Original: 7 shows how different loss components affect performance.
Revised: 7 shows how different loss components affect performance.

Original: • Performance under ˆ α .
Revised: • Performance under ˆ α .

Original: Fig.
Revised: Fig.

Original: 9 shows that ALOE with either true or estimated ratio performs similar given all query budgets.
Revised: 9 shows that ALOE with either true or estimated ratio performs similar given all query budgets.

Original: • Querying Strategies.
Revised: • Querying Strategies.

Original: Fig.
Revised: Fig.

Original: 8 further
Revised: 8 further

Original: confirms the theory of Thm. 1: diverse querying significantly outperforms the other querying strategies in the supervised setting.
Revised: confirms the theory of Thm. 1: diverse querying significantly outperforms the other querying strategies in the supervised setting.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We introduced Active Latent Outlier Exposure: a batch active learning approach for deep anomaly detection.
Revised: We introduced Active Latent Outlier Exposure: a batch active learning approach for deep anomaly detection.

Original: Inspired by a set of conditions that guarantee the generalization of anomaly score rankings from queried data to unqueried data, we proposed to use a diversified querying strategy and a combination of two losses for queried and unqueried samples.
Revised: Inspired by a set of conditions that guarantee the generalization of anomaly score rankings from queried data to unqueried data, we proposed to use a diversified querying strategy and a combination of two losses for queried and unqueried samples.

Original: Based on a simple heuristic for weighting the losses relative to each other and by estimating the unknown contamination rate from queried samples, we were able to make our approach free of its most important hyperparameters.
Revised: Based on a simple heuristic for weighting the losses relative to each other and by estimating the unknown contamination rate from queried samples, we were able to make our approach free of its most important hyperparameters.

Original: We showed on a variety of data sets from different domains that the approach results in a new state of the art in batch active anomaly detection.
Revised: We showed on a variety of data sets from different domains that the approach results in a new state of the art in batch active anomaly detection.

Original: A T HEOREM
Revised: A T HEOREM

Original: In Thm.
Revised: In Thm.

Original: we considered using the fixed-radius neighborhood ( δ -ball) of the queried data as the cover of the whole dataset and mentioned diverse querying has a smaller radius than other querying strategies.
Revised: we considered using the fixed-radius neighborhood ( δ -ball) of the queried data as the cover of the whole dataset and mentioned diverse querying has a smaller radius than other querying strategies.

Original: In this section, we will empirically verify this fact and further illustrate diverse querying leads to good ranking of un-queried data (see also Fig. 8 on test data)
Revised: In this section, we will empirically verify this fact and further illustrate diverse querying leads to good ranking of un-queried data (see also Fig. 8 on test data)

Original: As defined in Thm.
Revised: As defined in Thm.

Original: the radius is the smallest distance that is required for any un-queried sample to be covered by the neighborhood of a queried sample of the same type.
Revised: the radius is the smallest distance that is required for any un-queried sample to be covered by the neighborhood of a queried sample of the same type.

Original: Mathematically, we compute the radius as
Revised: Mathematically, we compute the radius as

Original: [Equation]
Revised: [Equation]

Original: (5) where we adopt the euclidean distance in the feature space for a meaningful metric d .
Revised: (5) where we adopt the euclidean distance in the feature space for a meaningful metric d .

Original: We apply NTL on the first class of CIFAR-10 and F-MNIST dataset.
Revised: We apply NTL on the first class of CIFAR-10 and F-MNIST dataset.

Original: We make queries with different budgets, after which we compute δ by Eq.
Revised: We make queries with different budgets, after which we compute δ by Eq.

Original: We repeat this procedure for 100 times and report the mean and standard deviation in Fig.
Revised: We repeat this procedure for 100 times and report the mean and standard deviation in Fig.

Original: We compared four querying strategies: diverse queries (k-means++ ),
Revised: We compared four querying strategies: diverse queries (k-means++ ),

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: uncertain queries (Mar), positive queries (Pos1), and random queries (Rand1).
Revised: uncertain queries (Mar), positive queries (Pos1), and random queries (Rand1).

Original: It shows that diverse queries significantly lead the smallest radius δ among the compared strategies on all querying budgets.
Revised: It shows that diverse queries significantly lead the smallest radius δ among the compared strategies on all querying budgets.

Original: Next
Revised: Next

Original: [Figure]
Revised: [Figure]

Original: we provide an empirical, overall justification of Thm. 

Revised: we provide an empirical, overall justification of Thm.

Original: (see also Fig.
Revised: (see also Fig.

Original: 8 on test data).
Revised: 8 on test data).

Original: An implication of Thm.
Revised: An implication of Thm.

Original: 1 is that, assuming anomaly scores are fixed, a smaller δ will satisfy the large anomaly score margin ( S ( x a ) − S ( x n ) ) more easily, hence it is easier for S to correctly rank the remaining unlabeled points.
Revised: 1 is that, assuming anomaly scores are fixed, a smaller δ will satisfy the large anomaly score margin ( S ( x a ) − S ( x n ) ) more easily, hence it is easier for S to correctly rank the remaining unlabeled points.

Original: To justify this implication
Revised: To justify this implication

Original: [Figure]
Revised: [Figure]

Original: we need a metric of ranking.
Revised: we need a metric of ranking.

Original: AUC satisfies this requirement as it is alternatively defined as (Mohri et al., 2018, 10.5.2)(Cortes and Mohri, 2003)
Revised: AUC satisfies this requirement as it is alternatively defined as (Mohri et al., 2018, 10.5.2)(Cortes and Mohri, 2003)

Original: [Equation]
Revised: [Equation]

Original: which measures the probability of ranking unlabeled samples u a higher than u n in terms of their scores.
Revised: which measures the probability of ranking unlabeled samples u a higher than u n in terms of their scores.

Original: U =
Revised: U =

Original: U 0 (cid:83) U 1 is the un-queried data indices and U 

Revised: U 0 (cid:83) U 1 is the un-queried data indices and U

Original: and U 1 are disjoint un-queried normal and abnormal data sets respectively.
Revised: and U 1 are disjoint un-queried normal and abnormal data sets respectively.

Original: u a and u n are instances of each kind.
Revised: u a and u n are instances of each kind.

Original: We conducted experiments on
Revised: We conducted experiments on

Original: CIFAR-10 and F-MNIST, where we trained an anomaly detector (NTL) on the queried data forepochs and then compute the AUC on the remaining un-queried data.
Revised: CIFAR-10 and F-MNIST, where we trained an anomaly detector (NTL) on the queried data forepochs and then compute the AUC on the remaining un-queried data.

Original: The results of four querying straties are reported in Fig.
Revised: The results of four querying straties are reported in Fig.

Original: 4, which shows that our proposed diverse querying strategy generalizes the anomaly score ranking the best to the unqueried data among the compared strategies, testifying our analysis in the main paper.
Revised: 4, which shows that our proposed diverse querying strategy generalizes the anomaly score ranking the best to the unqueried data among the compared strategies, testifying our analysis in the main paper.

Original: A consequence is that diverse querying can provide accurate assignments of the latent anomaly labels, which will further help learn a high-quality of anomaly detector through the unsupervised loss term in Eq.
Revised: A consequence is that diverse querying can provide accurate assignments of the latent anomaly labels, which will further help learn a high-quality of anomaly detector through the unsupervised loss term in Eq.

Original: B T HEOREM
Revised: B T HEOREM

Original: In this section, we will empirically justify the assumptions we made in Sec.
Revised: In this section, we will empirically justify the assumptions we made in Sec.

Original: 3.4 that are used to build an unbiased estimator of the anomaly ratio α (Eq.
Revised: 3.4 that are used to build an unbiased estimator of the anomaly ratio α (Eq.

Original: We will also demonstrate the robustness of the estimation under varying α .
Revised: We will also demonstrate the robustness of the estimation under varying α .

Original: B. P ROOF
Revised: B. P ROOF

Original: Proof.
Revised: Proof.

Original: Let A1 and A2 denote Assumptions 1 and 2, respectively.
Revised: Let A1 and A2 denote Assumptions 1 and 2, respectively.

Original: Furthermore, let q ( x 1 , ..., x | Q | ) denote the query distribution, q ( x i ) its marginals, and | Q | the number of queried samples.
Revised: Furthermore, let q ( x 1 , ..., x | Q | ) denote the query distribution, q ( x i ) its marginals, and | Q | the number of queried samples.

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: B. A SSUMPTION
Revised: B. A SSUMPTION

Original: [Figure]
Revised: [Figure]

Original: F-MNIST Class 2 as Normal 0 .0 .0 .0 .0 .1 .
Revised: F-MNIST Class 2 as Normal 0 .0 .0 .0 .0 .1 .

Original: We verify Assumption 1 by showing the correlation matrix in Fig.
Revised: We verify Assumption 1 by showing the correlation matrix in Fig.

Original: 5, where we jointly queriedpoints with diversified querying strategy and repeated 1000 times on two classes of CIFAR-10 and FMNIST.
Revised: 5, where we jointly queriedpoints with diversified querying strategy and repeated 1000 times on two classes of CIFAR-10 and FMNIST.

Original: Then the correlation between each pair of points are computed and placed in the off-diagonal entries.
Revised: Then the correlation between each pair of points are computed and placed in the off-diagonal entries.

Original: For each matrix, we show the average, maximum, and minimum of the off-diagonal terms
Revised: For each matrix, we show the average, maximum, and minimum of the off-diagonal terms

Original: • CIFAR-10 Class 1: -0.001, 0.103, -0.
Revised: • CIFAR-10 Class 1: -0.001, 0.103, -0.

Original: • CIFAR-10 Class 2: -0.001, 0.085, -0.
Revised: • CIFAR-10 Class 2: -0.001, 0.085, -0.

Original: • F-MNIST Class 1: -0.001, 0.081, -0.
Revised: • F-MNIST Class 1: -0.001, 0.081, -0.

Original: • F-MNIST Class 2: -0.005, 0.087, -0.
Revised: • F-MNIST Class 2: -0.005, 0.087, -0.

Original: Which shows the correlations ⟨ S ( x i ) , S ( x j ) ⟩ are negligible, and the anomaly scores can be considered approximately independent random variables.
Revised: Which shows the correlations ⟨ S ( x i ) , S ( x j ) ⟩ are negligible, and the anomaly scores can be considered approximately independent random variables.

Original: B. A SSUMPTION
Revised: B. A SSUMPTION

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: We verify Assumption 2 by checking the correlation between the anomaly score S ( x ) and the data density on one 2D Gaussian-distributed synthetic dataset and one real-world dataset – CIFAR-10.
Revised: We verify Assumption 2 by checking the correlation between the anomaly score S ( x ) and the data density on one 2D Gaussian-distributed synthetic dataset and one real-world dataset – CIFAR-10.

Original: We denote the marginal data distribution by p ( x ) , and denote the marginal distribution of a queried sample by q ( x ) .
Revised: We denote the marginal data distribution by p ( x ) , and denote the marginal distribution of a queried sample by q ( x ) .

Original: For synthetic data, the density p ( x ) is known, and we model q ( x ) with a kernel density estimator.
Revised: For synthetic data, the density p ( x ) is known, and we model q ( x ) with a kernel density estimator.

Original: For CIFAR-10, both data densities p ( x ) and q ( x ) are modeled by Masked Autoregressive
Revised: For CIFAR-10, both data densities p ( x ) and q ( x ) are modeled by Masked Autoregressive

Original: Flow (Papamakarios et al., 2017).
Revised: Flow (Papamakarios et al., 2017).

Original: For the choice of the anomaly score function S , we learn a deep
Revised: For the choice of the anomaly score function S , we learn a deep

Original: SVDD (Ruff et al., 2018) for the synthetic data and a NTL for CIFAR-10 with protocols discussed in
Revised: SVDD (Ruff et al., 2018) for the synthetic data and a NTL for CIFAR-10 with protocols discussed in

Original: Sec. 4.1.
Revised: Sec. 4.1.

Original: Fig. 6 and Fig.
Revised: Fig. 6 and Fig.

Original: 7 plot S ( x ) against log p ( x ) and log q ( x ) respectively.
Revised: 7 plot S ( x ) against log p ( x ) and log q ( x ) respectively.

Original: In Fig.
Revised: In Fig.

Original: 7, we observe that there is a monotonic relationship between S ( x ) and log p ( x ) or log q ( x ) with a R 2 value of 0 .
Revised: 7, we observe that there is a monotonic relationship between S ( x ) and log p ( x ) or log q ( x ) with a R 2 value of 0 .

Original: 41 and
Revised: 41 and

Original: 32 respectively.
Revised: 32 respectively.

Original: In Fig.
Revised: In Fig.

Original: 6, we observe a perfect linear relationship between S ( x ) and log p ( x ) , and a monotonic relationship between S ( x ) and log q ( x ) .
Revised: 6, we observe a perfect linear relationship between S ( x ) and log p ( x ) , and a monotonic relationship between S ( x ) and log q ( x ) .

Original: These plots empirically show that the anomaly score S ( x ) is predictive and a sufficient statistic of both p ( x ) and q ( x ) .
Revised: These plots empirically show that the anomaly score S ( x ) is predictive and a sufficient statistic of both p ( x ) and q ( x ) .

Original: B. C ONTAMINATION R ATIO E STIMATION
Revised: B. C ONTAMINATION R ATIO E STIMATION

Original: [Table]
Revised: [Table]

Original: We estimate the contamination ratio by Eq.
Revised: We estimate the contamination ratio by Eq.

Original: (4) under varying true ratios.
Revised: (4) under varying true ratios.

Original: This part shows the estimated contamination ratio when the query budget is |Q| = 40 .
Revised: This part shows the estimated contamination ratio when the query budget is |Q| = 40 .

Original: The estimations the backbone
Revised: The estimations the backbone

Original: NTL is shown in Tab. 4.
Revised: NTL is shown in Tab. 4.

Original: The first row contamination rate, and datasets using
Revised: The first row contamination rate, and datasets using

Original: Most estimates are withing the error bars and hence accurate.
Revised: Most estimates are withing the error bars and hence accurate.

Original: The estimation error is acceptable as confirmed by the sensitivity study in (Qiu et al., 2022a) which concludes that the LOE approach still works well if the anomaly ratio is mis-specified within 5 percentage points.
Revised: The estimation error is acceptable as confirmed by the sensitivity study in (Qiu et al., 2022a) which concludes that the LOE approach still works well if the anomaly ratio is mis-specified within 5 percentage points.

Original: C B ASELINES D ETAILS
Revised: C B ASELINES D ETAILS

Original: In this section, we describe the details of the baselines in Tab. 1 in the main paper.
Revised: In this section, we describe the details of the baselines in Tab. 1 in the main paper.

Original: For each baseline method, we explain their query strategies and post-query training strategies we implement in our experiment.
Revised: For each baseline method, we explain their query strategies and post-query training strategies we implement in our experiment.

Original: Please also refer to our codebase for practical implementation details.
Revised: Please also refer to our codebase for practical implementation details.

Original: • Rand1.
Revised: • Rand1.

Original: This strategy used by Ruff et al.
Revised: This strategy used by Ruff et al.

Original: (2019) selects queries by sampling uniformly without replacement across the training set, resulting in the queried index set Q
Revised: (2019) selects queries by sampling uniformly without replacement across the training set, resulting in the queried index set Q

Original: = { i q ∼ Unif (1 , · · · , N ) | 1 ≤ q ≤ |Q|} .
Revised: = { i q ∼ Unif (1 , · · · , N ) | 1 ≤ q ≤ |Q|} .

Original: After the querying, models are trained with a supervised loss function based on outlier exposure on the labeled data and with a one-class classification loss function on the unlabeled data,
Revised: After the querying, models are trained with a supervised loss function based on outlier exposure on the labeled data and with a one-class classification loss function on the unlabeled data,

Original: [Equation]
Revised: [Equation]

Original: As in ALOE both loss contributions are weighted equally.
Revised: As in ALOE both loss contributions are weighted equally.

Original: L Rand1 ( θ ) is minimized with respect to the backbone model parameters θ .
Revised: L Rand1 ( θ ) is minimized with respect to the backbone model parameters θ .

Original: • Rand2.
Revised: • Rand2.

Original: The querying strategy of Trittenbach et al.
Revised: The querying strategy of Trittenbach et al.

Original: (2021) samples uniformly among the top 50% data ranked by anomaly scores without replacement.
Revised: (2021) samples uniformly among the top 50% data ranked by anomaly scores without replacement.

Original: This leades to a random set of “positive” queries.
Revised: This leades to a random set of “positive” queries.

Original: After the queries are labeled, the training loss function is the same as L Rand1 ( θ ) (Eq.
Revised: After the queries are labeled, the training loss function is the same as L Rand1 ( θ ) (Eq.

Original: • Mar.
Revised: • Mar.

Original: After training the backbone model for one epoch, this querying strategy by Görnitz et al.
Revised: After training the backbone model for one epoch, this querying strategy by Görnitz et al.

Original: (2013) uses the α -quantile ( s α ) of the training data anomaly scores to define a “normality region”.
Revised: (2013) uses the α -quantile ( s α ) of the training data anomaly scores to define a “normality region”.

Original: Then the |Q| samples closest to the margin s α are selected to be queried.
Revised: Then the |Q| samples closest to the margin s α are selected to be queried.

Original: After the queries are labeled, the training loss function is the same as L Rand1 ( θ ) (Eq.
Revised: After the queries are labeled, the training loss function is the same as L Rand1 ( θ ) (Eq.

Original: Note that in practice we don’t know the true anomaly ratio for the α -quantile.
Revised: Note that in practice we don’t know the true anomaly ratio for the α -quantile.

Original: In all experiment, we provide this querying strategy with the true contamination ratio of the dataset.
Revised: In all experiment, we provide this querying strategy with the true contamination ratio of the dataset.

Original: Even with the true ratio, the “Mar” strategy is still outperformed by ALOE.
Revised: Even with the true ratio, the “Mar” strategy is still outperformed by ALOE.

Original: • Hybr1.
Revised: • Hybr1.

Original: This hybrid strategy, also used by (Görnitz et al., 2013) combines the “Mar” query with neighborhood-based diversification.
Revised: This hybrid strategy, also used by (Görnitz et al., 2013) combines the “Mar” query with neighborhood-based diversification.

Original: The neighborhood-based strategy selects samples with fewer neighbors covered by the queried set to ensure the samples’ diversity in the feature space.
Revised: The neighborhood-based strategy selects samples with fewer neighbors covered by the queried set to ensure the samples’ diversity in the feature space.

Original: We start
Revised: We start

Original: by selecting the data index arg min 1 ≤ i ≤ N ∥
Revised: by selecting the data index arg min 1 ≤ i ≤ N ∥

Original: s i − s α ∥ into Q .
Revised: s i − s α ∥ into Q .

Original: Then the samples are selected sequentially without replacement by the criterion
Revised: Then the samples are selected sequentially without replacement by the criterion

Original: [Equation]
Revised: [Equation]

Original: where the inter-sample distance is measured in the feature space and the number of nearest neighbors is k = ⌈ N/
Revised: where the inter-sample distance is measured in the feature space and the number of nearest neighbors is k = ⌈ N/

Original: |Q|⌉ .
Revised: |Q|⌉ .

Original: We set β = 1 for equal contribution of both terms.
Revised: We set β = 1 for equal contribution of both terms.

Original: After the queries are labeled, the training loss function is the same as L Rand1 ( θ ) (Eq.
Revised: After the queries are labeled, the training loss function is the same as L Rand1 ( θ ) (Eq.

Original: • Pos1.
Revised: • Pos1.

Original: This querying strategy by Pimentel et al.
Revised: This querying strategy by Pimentel et al.

Original: (2020) always selects the top-ranked samples ordered by their anomaly scores, arg max 1 ≤ i ≤ N s i .
Revised: (2020) always selects the top-ranked samples ordered by their anomaly scores, arg max 1 ≤ i ≤ N s i .

Original: After the queries are labeled, the training loss only involves the labeled data
Revised: After the queries are labeled, the training loss only involves the labeled data

Original: [Equation]
Revised: [Equation]

Original: Pimentel et al.
Revised: Pimentel et al.

Original: (2020) use the logistic loss but we use the supervised outlier exposure loss.
Revised: (2020) use the logistic loss but we use the supervised outlier exposure loss.

Original: The supervised outlier exposure loss is shown to be better than the logistic loss in learning anomaly detection models (Ruff et al., 2019;
Revised: The supervised outlier exposure loss is shown to be better than the logistic loss in learning anomaly detection models (Ruff et al., 2019;

Original: Hendrycks et al., 2018).
Revised: Hendrycks et al., 2018).

Original: • Pos2.
Revised: • Pos2.

Original: This approach of (Barnabé-Lortie et al., 2015) uses the same querying strategy as Pos1, but the training is different.
Revised: This approach of (Barnabé-Lortie et al., 2015) uses the same querying strategy as Pos1, but the training is different.

Original: Pos2 also uses the unlabeled data during training.
Revised: Pos2 also uses the unlabeled data during training.

Original: After the queries are labeled, the training loss function is the same as L Rand1 ( θ ) (Eq.
Revised: After the queries are labeled, the training loss function is the same as L Rand1 ( θ ) (Eq.

Original: • Hybr2.
Revised: • Hybr2.

Original: This hybrid strategy by Das et al.
Revised: This hybrid strategy by Das et al.

Original: (2019) makes positive diverse queries.
Revised: (2019) makes positive diverse queries.

Original: It combines querying according to anomaly scores with distance-based diversification.
Revised: It combines querying according to anomaly scores with distance-based diversification.

Original: Hybr2 selects the initial query arg max 1 ≤ i ≤ N s i into Q .
Revised: Hybr2 selects the initial query arg max 1 ≤ i ≤ N s i into Q .

Original: Then the samples are selected sequentially without replacement by the criterion
Revised: Then the samples are selected sequentially without replacement by the criterion

Original: [Equation]
Revised: [Equation]

Original: , x b ) where d ( x i , x j ) =
Revised: , x b ) where d ( x i , x j ) =

Original: || ϕ
Revised: || ϕ

Original: ( x i ) − ϕ ( x j ) || 2 .
Revised: ( x i ) − ϕ ( x j ) || 2 .

Original: We set β = 1 for equal contribution of both terms.
Revised: We set β = 1 for equal contribution of both terms.

Original: After the queries are labeled, Das et al.
Revised: After the queries are labeled, Das et al.

Original: (2019) use the labeled set to learn a set of weights for the components of an ensemble of detectors.
Revised: (2019) use the labeled set to learn a set of weights for the components of an ensemble of detectors.

Original: For a fair comparison of active learning strategies, we use the labeled set to update an individual anomaly detector with parameters θ by optimizing the loss
Revised: For a fair comparison of active learning strategies, we use the labeled set to update an individual anomaly detector with parameters θ by optimizing the loss

Original: [Equation]
Revised: [Equation]

Original: • Hybr3.
Revised: • Hybr3.

Original: This baseline by (Ning et al., 2022) uses the same query strategy as Hybr2, but differs in the training loss function,
Revised: This baseline by (Ning et al., 2022) uses the same query strategy as Hybr2, but differs in the training loss function,

Original: [Equation]
Revised: [Equation]

Original: where w j = 2 σ ( d j ) and ˆ w i = 2 − 2 σ ( d i ) where σ ( · ) is the Sigmoid function and d i =
Revised: where w j = 2 σ ( d j ) and ˆ w i = 2 − 2 σ ( d i ) where σ ( · ) is the Sigmoid function and d i =

Original: c d (cid:0) || ϕ
Revised: c d (cid:0) || ϕ

Original: ( x i ) − c 

Revised: ( x i ) − c

Original: ||
Revised: ||

Original: −
Revised: −

Original: || ϕ
Revised: || ϕ

Original: ( x i ) − c
Revised: ( x i ) − c

Original: || 2 (cid:1) where c 0 is the center of the queried normal samples and c 1 is the center of the queried abnormal samples in the feature space, and c d is the min-max normalization factor.
Revised: || 2 (cid:1) where c 0 is the center of the queried normal samples and c 1 is the center of the queried abnormal samples in the feature space, and c d is the min-max normalization factor.

Original: We make three observations for the loss function.
Revised: We make three observations for the loss function.

Original: First, L Hybr3 ( θ ) filters out all labeled anomalies in the supervised learning part and puts a large weight (but only as large as 2 at most) to the true normal data that has a high anomaly score.
Revised: First, L Hybr3 ( θ ) filters out all labeled anomalies in the supervised learning part and puts a large weight (but only as large as 2 at most) to the true normal data that has a high anomaly score.

Original: Second, in the unlabeled data, L Hybr3 ( θ ) puts smaller weight (less than 1) to the seemingly abnormal data.
Revised: Second, in the unlabeled data, L Hybr3 ( θ ) puts smaller weight (less than 1) to the seemingly abnormal data.

Original: Third, overall, the weight of the labeled data is similar to the weight of the unlabeled data.
Revised: Third, overall, the weight of the labeled data is similar to the weight of the unlabeled data.

Original: This is unlike ALOE, which weighs labeled data |U| / |Q| times higher than unlabeled data.
Revised: This is unlike ALOE, which weighs labeled data |U| / |Q| times higher than unlabeled data.

Original: D I MPLEMENTATION D ETAILS
Revised: D I MPLEMENTATION D ETAILS

Original: In this section, we present the implementation details in the experiments.
Revised: In this section, we present the implementation details in the experiments.

Original: They include an overall description of the experimental procedure for all datasets, model architecture, data split, and details about the optimization algorithm.
Revised: They include an overall description of the experimental procedure for all datasets, model architecture, data split, and details about the optimization algorithm.

Original: D. E XPERIMENTAL P ROCEDURE
Revised: D. E XPERIMENTAL P ROCEDURE

Original: We apply the same experimental procedure for each dataset and each compared method.
Revised: We apply the same experimental procedure for each dataset and each compared method.

Original: The experiment starts with an unlabeled, contaminated training dataset with index set U .
Revised: The experiment starts with an unlabeled, contaminated training dataset with index set U .

Original: We first train the anomaly detector on U for one epoch as if all data were normal.
Revised: We first train the anomaly detector on U for one epoch as if all data were normal.

Original: Then we conduct the diverse active queries at once and estimate the contamination ratio α by the importance sampling estimator Eq.
Revised: Then we conduct the diverse active queries at once and estimate the contamination ratio α by the importance sampling estimator Eq.

Original: Lastly, we optimize the post-query training losses until convergence.
Revised: Lastly, we optimize the post-query training losses until convergence.

Original: The obtained anomaly detectors are evaluated on a held-out test set.
Revised: The obtained anomaly detectors are evaluated on a held-out test set.

Original: D. D ATA S PLIT
Revised: D. D ATA S PLIT

Original: Image Data.
Revised: Image Data.

Original: For the image data including both natural (CIFAR-10 (Krizhevsky et al., 2009) and
Revised: For the image data including both natural (CIFAR-10 (Krizhevsky et al., 2009) and

Original: F-MNIST (Xiao et al., 2017)) and medical (MedMNIST (Yang et al., 2021)) images, we use the original training, validation (if any), and test split.
Revised: F-MNIST (Xiao et al., 2017)) and medical (MedMNIST (Yang et al., 2021)) images, we use the original training, validation (if any), and test split.

Original: When contaminating the training data of one class, we randomly sample images from other classes’ training data and leave the validation and test set untouched.
Revised: When contaminating the training data of one class, we randomly sample images from other classes’ training data and leave the validation and test set untouched.

Original: Specifically for DermaMNIST in MedMNIST, we only consider the classes that have more than 500 images in the training data as normal data candidates, which include benign keratosis-like lesions, melanoma, and melanocytic nevi.
Revised: Specifically for DermaMNIST in MedMNIST, we only consider the classes that have more than 500 images in the training data as normal data candidates, which include benign keratosis-like lesions, melanoma, and melanocytic nevi.

Original: We view all other classes as abnormal data.
Revised: We view all other classes as abnormal data.

Original: Different experiment runs have different randomness.
Revised: Different experiment runs have different randomness.

Original: Tabular Data.
Revised: Tabular Data.

Original: Our study includes the four multi-dimensional tabular datasets from the ODDS repository 2 which have an outlier ratio of at least 30% . .
Revised: Our study includes the four multi-dimensional tabular datasets from the ODDS repository 2 which have an outlier ratio of at least 30% . .

Original: To form the training and test set for tabular data, we first split the data into normal and abnormal categories.
Revised: To form the training and test set for tabular data, we first split the data into normal and abnormal categories.

Original: We randomly sub-sample half the normal data as the training data and treat the other half as the test data.
Revised: We randomly sub-sample half the normal data as the training data and treat the other half as the test data.

Original: To contaminate training data, we randomly sub-sample the abnormal data into the training set to reach the desired 10% contamination ratio; the remaining abnormal data goes into the test set.
Revised: To contaminate training data, we randomly sub-sample the abnormal data into the training set to reach the desired 10% contamination ratio; the remaining abnormal data goes into the test set.

Original: Different experiment runs have different randomness.
Revised: Different experiment runs have different randomness.

Original: Video Data.
Revised: Video Data.

Original: We use UCSD Peds1 3 , a benchmark dataset for video anomaly detection.
Revised: We use UCSD Peds1 3 , a benchmark dataset for video anomaly detection.

Original: UCSD Peds contains 70 surveillance video clips – 34 training clips and 36 testing clips.
Revised: UCSD Peds contains 70 surveillance video clips – 34 training clips and 36 testing clips.

Original: Each frame is labeled to be abnormal if it has non-pedestrian objects and labeled normal otherwise.
Revised: Each frame is labeled to be abnormal if it has non-pedestrian objects and labeled normal otherwise.

Original: Making the same assumption as (Pang et al., 2020), we treat each frame independent and mix the original training and testing clips together.
Revised: Making the same assumption as (Pang et al., 2020), we treat each frame independent and mix the original training and testing clips together.

Original: This results in a dataset of 9955 normal frames and 4045 abnormal frames.
Revised: This results in a dataset of 9955 normal frames and 4045 abnormal frames.

Original: We then randomly sub-sample 6800 frames out of the normal frames and 2914 frames out of the abnormal frames without replacement to form a contaminated training dataset with 30% anomaly ratio.
Revised: We then randomly sub-sample 6800 frames out of the normal frames and 2914 frames out of the abnormal frames without replacement to form a contaminated training dataset with 30% anomaly ratio.

Original: A same ratio is also used in the literature (Pang et al., 2020) that uses this dataset.
Revised: A same ratio is also used in the literature (Pang et al., 2020) that uses this dataset.

Original: The remaining data after sampling is used for the testing set, whose about 30% data is anomalous.
Revised: The remaining data after sampling is used for the testing set, whose about 30% data is anomalous.

Original: Like the other data types, different experiment runs have different randomness for the training dataset construction.
Revised: Like the other data types, different experiment runs have different randomness for the training dataset construction.

Original: D. M ODEL A RCHITECTURE
Revised: D. M ODEL A RCHITECTURE

Original: The experiments involve two anomaly detectors, NTL and multi-head RotNet (MHRot), and three data types.
Revised: The experiments involve two anomaly detectors, NTL and multi-head RotNet (MHRot), and three data types.

Original: NTL on Image Data and Video Data.
Revised: NTL on Image Data and Video Data.

Original: For all images (either natural or medical) and video frames, we extract their features by feeding them into a ResNet152 pre-trained on ImageNet and taking the penultimate layer output for our usage.
Revised: For all images (either natural or medical) and video frames, we extract their features by feeding them into a ResNet152 pre-trained on ImageNet and taking the penultimate layer output for our usage.

Original: The features are kept fixed during training.
Revised: The features are kept fixed during training.

Original: We then train an
Revised: We then train an

Original: NTL on those features.
Revised: NTL on those features.

Original: We apply the same number of transformations, network components, and anomaly loss function L θ 1 ( x ) , as when Qiu et al.
Revised: We apply the same number of transformations, network components, and anomaly loss function L θ 1 ( x ) , as when Qiu et al.

Original: (2022a) apply NTL on the image data.
Revised: (2022a) apply NTL on the image data.

Original: NTL on Tabular Data.
Revised: NTL on Tabular Data.

Original: We directly use the tabular data as the input of NTL.
Revised: We directly use the tabular data as the input of NTL.

Original: We apply the same number of transformations, network components, and anomaly loss function L θ 1 ( x ) , as when Qiu et al.
Revised: We apply the same number of transformations, network components, and anomaly loss function L θ 1 ( x ) , as when Qiu et al.

Original: (2022a) apply NTL on the tabular data.
Revised: (2022a) apply NTL on the tabular data.

Original: MHRot on Image Data.
Revised: MHRot on Image Data.

Original: We use the raw images as input for MHRot.
Revised: We use the raw images as input for MHRot.

Original: We set the same transformations, MHRot architecture, and anomaly loss function as when Qiu et al.
Revised: We set the same transformations, MHRot architecture, and anomaly loss function as when Qiu et al.

Original: (2022a) apply MHRot on the image data.
Revised: (2022a) apply MHRot on the image data.

Original: DSVDD on Image Data.
Revised: DSVDD on Image Data.

Original: For all images (either natural or medical), we build DSVDD on the features from the penultimate layer of a ResNet152 pre-trained on ImageNet.
Revised: For all images (either natural or medical), we build DSVDD on the features from the penultimate layer of a ResNet152 pre-trained on ImageNet.

Original: The features are kept fixed during training.
Revised: The features are kept fixed during training.

Original: The neural network of DSVDD is a three-layer MLP with intermediate batch normalization layers and ReLU activation.
Revised: The neural network of DSVDD is a three-layer MLP with intermediate batch normalization layers and ReLU activation.

Original: The hidden sizes are [1024 , 512 , 128] .
Revised: The hidden sizes are [1024 , 512 , 128] .

Original: D. O PTIMIZATION A LGORITHM
Revised: D. O PTIMIZATION A LGORITHM

Original: [Table]
Revised: [Table]

Original: In the experiments, we use Adam (Kingma and Ba, 2014) to optimize the objective function to find the local optimal anomaly scorer parameters θ .
Revised: In the experiments, we use Adam (Kingma and Ba, 2014) to optimize the objective function to find the local optimal anomaly scorer parameters θ .

Original: For Adam, we set β 1 = 0 .
Revised: For Adam, we set β 1 = 0 .

Original: 9 , β 2 = 0 .
Revised: 9 , β 2 = 0 .

Original: 999 and no weight decay for all experiments.
Revised: 999 and no weight decay for all experiments.

Original: To set the learning rate, training epochs, minibatch size for MedMNIST, we find the best performing hyperparameters by evaluating the method on the validation dataset.
Revised: To set the learning rate, training epochs, minibatch size for MedMNIST, we find the best performing hyperparameters by evaluating the method on the validation dataset.

Original: We use the same hyperparameters on other image data.
Revised: We use the same hyperparameters on other image data.

Original: For video data and tabular data, the optimization hyperparameters are set as recommended by Qiu et al.
Revised: For video data and tabular data, the optimization hyperparameters are set as recommended by Qiu et al.

Original: (2022a).
Revised: (2022a).

Original: We summarize all optimization hyperparameters in Tab. 5.
Revised: We summarize all optimization hyperparameters in Tab. 5.

Original: When training models with ALOE, we resort to the block coordinate descent scheme that update the model parameters θ and the pseudo labels ˜
Revised: When training models with ALOE, we resort to the block coordinate descent scheme that update the model parameters θ and the pseudo labels ˜

Original: y of unlabeled data in turn.
Revised: y of unlabeled data in turn.

Original: In particular, we take the following two update steps iteratively:
Revised: In particular, we take the following two update steps iteratively:

Original: • update θ by optimizing Eq.
Revised: • update θ by optimizing Eq.

Original: (3) given ˜ y fixed; • update ˜ y by sovling the constrained optimization in Sec.
Revised: (3) given ˜ y fixed; • update ˜ y by sovling the constrained optimization in Sec.

Original: 3.3 given θ fixed;
Revised: 3.3 given θ fixed;

Original: Upon updating ˜
Revised: Upon updating ˜

Original: y
Revised: y

Original: , we use the LOE S variant (Qiu et al., 2022a) for the unlabeled data.
Revised: , we use the LOE S variant (Qiu et al., 2022a) for the unlabeled data.

Original: We set the pseudo labels ˜
Revised: We set the pseudo labels ˜

Original: y
Revised: y

Original: by performing the optimization below
Revised: by performing the optimization below

Original: [Equation]
Revised: [Equation]

Original: where ˜ α is the updated contamination ratio of U after the querying round, ˜ α = (cid:0) αN − (cid:80) j ∈Q 1 a ( x j ) (cid:1) / |U| , and α is computed by Eq.
Revised: where ˜ α is the updated contamination ratio of U after the querying round, ˜ α = (cid:0) αN − (cid:80) j ∈Q 1 a ( x j ) (cid:1) / |U| , and α is computed by Eq.

Original: (4) given Q .
Revised: (4) given Q .

Original: The solution is to rank the data by L θ 0 ( x ) − L θ 1 ( x ) and label the top ˜ α data abnormal (equivalently setting ˜ y
Revised: The solution is to rank the data by L θ 0 ( x ) − L θ 1 ( x ) and label the top ˜ α data abnormal (equivalently setting ˜ y

Original: = 0 . 5 ) and all the other data normal (equivalently ˜ y
Revised: = 0 . 5 ) and all the other data normal (equivalently ˜ y

Original: = 0 ).
Revised: = 0 ).

Original: When we compute the Euclidean distance in the feature space, we construct the feature vector of a sample by concatenating all its encoder representations of different transformations.
Revised: When we compute the Euclidean distance in the feature space, we construct the feature vector of a sample by concatenating all its encoder representations of different transformations.

Original: For example, if the encoder representation has 500 dimensions and the model has 10 transformations, then the final feature representation has 10 × 500 = 5000 dimensions.
Revised: For example, if the encoder representation has 500 dimensions and the model has 10 transformations, then the final feature representation has 10 × 500 = 5000 dimensions.

Original: D. T IME C OMPLEXITY
Revised: D. T IME C OMPLEXITY

Original: Regarding the time complexity, the optimization uses stochastic gradient descent.
Revised: Regarding the time complexity, the optimization uses stochastic gradient descent.

Original: The complexity of our querying strategy is O ( KN ) where K is the number of queries and N is the size of the training data.
Revised: The complexity of our querying strategy is O ( KN ) where K is the number of queries and N is the size of the training data.

Original: This complexity can be further reduced to O
Revised: This complexity can be further reduced to O

Original: ( K log N ) with a scalable extension of k-means++
Revised: ( K log N ) with a scalable extension of k-means++

Original: (Bahmani et al., 2012).
Revised: (Bahmani et al., 2012).

Original: E A DDITIONAL E XPERIMENTS AND A BLATION S TUDY
Revised: E A DDITIONAL E XPERIMENTS AND A BLATION S TUDY

Original: The goal of this ablation study is to show the generality of ALOE, to better understand the success of
Revised: The goal of this ablation study is to show the generality of ALOE, to better understand the success of

Original: ALOE
Revised: ALOE

Original: [Table]
Revised: [Table]

Original: and to disentangle the benefits of the training objective and the querying strategy.
Revised: and to disentangle the benefits of the training objective and the querying strategy.

Original: To this end, we applied ALOE to different backbone models and different data forms (raw input and embedding input), performed specialized experiments to compare the querying strategies, to demonstrate the optimality of the proposed weighting scheme in Eq.
Revised: To this end, we applied ALOE to different backbone models and different data forms (raw input and embedding input), performed specialized experiments to compare the querying strategies, to demonstrate the optimality of the proposed weighting scheme in Eq.

Original: [Table]
Revised: [Table]

Original: and to validate the detection performance of the estimated ratio by Eq.
Revised: and to validate the detection performance of the estimated ratio by Eq.

Original: We also compared ALOE against additional baselines including semi-supervised learning frameworks and shallow anomaly detectors
Revised: We also compared ALOE against additional baselines including semi-supervised learning frameworks and shallow anomaly detectors

Original: [Table]
Revised: [Table]

Original: E. R ESULTS WITH O THER B ACKBONE M ODELS
Revised: E. R ESULTS WITH O THER B ACKBONE M ODELS

Original: [Table]
Revised: [Table]

Original: We are interested whether ALOE works for different backbone models.
Revised: We are interested whether ALOE works for different backbone models.

Original: To that end, we repeat part of the experiments in Tab. 2 but using an self-supervised learning model MHRot (Hendrycks et al., 2019) and a one class classification model Deep SVDD (Ruff et al., 2018) as the backbone model.
Revised: To that end, we repeat part of the experiments in Tab. 2 but using an self-supervised learning model MHRot (Hendrycks et al., 2019) and a one class classification model Deep SVDD (Ruff et al., 2018) as the backbone model.

Original: We compare ALOE to two best performing baselines — Hybr1 and Hybr2.
Revised: We compare ALOE to two best performing baselines — Hybr1 and Hybr2.

Original: In this experiment, MHRot and Deep SVDD take different input types: while MHRot takes raw images as input, Deep SVDD uses pre-trained image features.
Revised: In this experiment, MHRot and Deep SVDD take different input types: while MHRot takes raw images as input, Deep SVDD uses pre-trained image features.

Original: We also set the query budget to be |Q| = 20 .
Revised: We also set the query budget to be |Q| = 20 .

Original: We report the results in Tab. 6.
Revised: We report the results in Tab. 6.

Original: It showcases the superiority of ALOE compared to the baselines.
Revised: It showcases the superiority of ALOE compared to the baselines.

Original: On all datasets, ALOE significantly outperforms the two best performing baselines, Hybr1 and Hybr2, thus demonstrating the wide applicability of ALOE across anomaly detection model types.
Revised: On all datasets, ALOE significantly outperforms the two best performing baselines, Hybr1 and Hybr2, thus demonstrating the wide applicability of ALOE across anomaly detection model types.

Original: E.
Revised: E.

Original: D ISENTANGLEMENT OF ALOE
Revised: D ISENTANGLEMENT OF ALOE

Original: the performance drop when NTL and MHRot trained on the contaminated datasets.
Revised: the performance drop when NTL and MHRot trained on the contaminated datasets.

Original: Results of the unsupervised method
Revised: Results of the unsupervised method

Original: LOE are borrowed from Qiu et al.
Revised: LOE are borrowed from Qiu et al.

Original: (2022a).
Revised: (2022a).

Original: [Table]
Revised: [Table]

Original: We disentangle the benefits of each component of ALOE and compare it to unsupervised anomaly detection with latent outlier exposure (LOE)
Revised: We disentangle the benefits of each component of ALOE and compare it to unsupervised anomaly detection with latent outlier exposure (LOE)

Original: (Qiu et al., 2022a), and to supervised active anomaly detection with k-means++ querying strategy.
Revised: (Qiu et al., 2022a), and to supervised active anomaly detection with k-means++ querying strategy.

Original: Both active approaches (k-means++ and ALOE) are evaluated with |Q| = 20 labeled samples.
Revised: Both active approaches (k-means++ and ALOE) are evaluated with |Q| = 20 labeled samples.

Original: The unsupervised approach LOE requires an hyperparameter of the assumed data contamination ratio, which we set to the ground truth value 10% .
Revised: The unsupervised approach LOE requires an hyperparameter of the assumed data contamination ratio, which we set to the ground truth value 10% .

Original: Comparing ALOE to LOE reveals the benefits of the k-means++ active approach 4 ; comparing ALOE to k-means++ reveals the benefits of the unsupervised loss function in ALOE.
Revised: Comparing ALOE to LOE reveals the benefits of the k-means++ active approach 4 ; comparing ALOE to k-means++ reveals the benefits of the unsupervised loss function in ALOE.

Original: Results in Tab.
Revised: Results in Tab.

Original: 7 show that ALOE leads to improvements for both ablation models.
Revised: 7 show that ALOE leads to improvements for both ablation models.

Original: E. C OMPARISONS OF Q UERYING S TRATEGIES
Revised: E. C OMPARISONS OF Q UERYING S TRATEGIES

Original: To understand the benefit of sampling diverse queries with k-means++ and to examine the generalization ability (stated in Thm. 1) of different querying strategies, we run the following experiment:
Revised: To understand the benefit of sampling diverse queries with k-means++ and to examine the generalization ability (stated in Thm. 1) of different querying strategies, we run the following experiment:

Original: We use a supervised loss on labeled samples to train various anomaly detectors.
Revised: We use a supervised loss on labeled samples to train various anomaly detectors.

Original: The only difference between them is the querying strategy used to select the samples.
Revised: The only difference between them is the querying strategy used to select the samples.

Original: We evaluate them on all image data sets we study for varying number of queries |Q| between 20 and 160 .
Revised: We evaluate them on all image data sets we study for varying number of queries |Q| between 20 and 160 .

Original: Results are in Fig.
Revised: Results are in Fig.

Original: On all datasets except OCT, k-means++ consistently outperforms all other querying strategies from previous work on active anomaly detection.
Revised: On all datasets except OCT, k-means++ consistently outperforms all other querying strategies from previous work on active anomaly detection.

Original: The difference is particularly large when only few samples are queried.
Revised: The difference is particularly large when only few samples are queried.

Original: This also confirms that diverse querying generalizes better on the test data than other querying strategies (see additional results in Supp.
Revised: This also confirms that diverse querying generalizes better on the test data than other querying strategies (see additional results in Supp.

Original: A).
Revised: A).

Original: E. A BLATION ON E STIMATED C ONTAMINATION R ATIO
Revised: E. A BLATION ON E STIMATED C ONTAMINATION R ATIO

Original: To see how the estimated ratio affects the detection performance, we compare ALOE to the counterpart with the true anomaly ratio.
Revised: To see how the estimated ratio affects the detection performance, we compare ALOE to the counterpart with the true anomaly ratio.

Original: We experiment on all 11 image datasets.
Revised: We experiment on all 11 image datasets.

Original: In Fig.
Revised: In Fig.

Original: 9, we report the average results for all datasets when querying |Q| = 20 , 40 , 80 , 160 samples.
Revised: 9, we report the average results for all datasets when querying |Q| = 20 , 40 , 80 , 160 samples.

Original: It shows that ALOE with either true ratio or estimated ratio performs similar given all query budgets.
Revised: It shows that ALOE with either true ratio or estimated ratio performs similar given all query budgets.

Original: Therefore, the estimated ratio can be applied safely.
Revised: Therefore, the estimated ratio can be applied safely.

Original: This is very important in practice, since in many applications the true anomaly ratio is not known.
Revised: This is very important in practice, since in many applications the true anomaly ratio is not known.

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: E. A BLATIONS ON W EIGHTING S CHEME
Revised: E. A BLATIONS ON W EIGHTING S CHEME

Original: We make the implicit assumption that the averaged losses over queried and unqueried data should be equally weighted (Eq.
Revised: We make the implicit assumption that the averaged losses over queried and unqueried data should be equally weighted (Eq.

Original: That means, if a fraction ϵ of the data is queried, every queried data point weights 1 /ϵ as much as an unqueried datum.
Revised: That means, if a fraction ϵ of the data is queried, every queried data point weights 1 /ϵ as much as an unqueried datum.

Original: As a consequence, neither the queried nor the unqueried data points can dominate the result
Revised: As a consequence, neither the queried nor the unqueried data points can dominate the result

Original: [Figure]
Revised: [Figure]

Original: To test whether this heuristic is indeed optimal
Revised: To test whether this heuristic is indeed optimal

Original: [Figure]
Revised: [Figure]

Original: we added a scalar prefactor to the supervised loss in Eq.
Revised: we added a scalar prefactor to the supervised loss in Eq.

Original: (3) (the first term) and reported the results on the CIFAR-10 and F-MNIST datasets with different query budgets (Fig. 10).
Revised: (3) (the first term) and reported the results on the CIFAR-10 and F-MNIST datasets with different query budgets (Fig. 10).

Original: A weight < 1 corresponds to downweighting the queried term, while a weight > 1 corresponds to upweighting it.
Revised: A weight < 1 corresponds to downweighting the queried term, while a weight > 1 corresponds to upweighting it.

Original: We use the same experimental setup and backbone (NTL) as in the paper.
Revised: We use the same experimental setup and backbone (NTL) as in the paper.

Original: The results are shown in Fig.
Revised: The results are shown in Fig.

Original: We see that the performance degrades both upon down-weighting (0.01, 0.1) or up-weighting (10.0) the queried samples.
Revised: We see that the performance degrades both upon down-weighting (0.01, 0.1) or up-weighting (10.0) the queried samples.

Original: In contrast, equal weighting yields optimal results.
Revised: In contrast, equal weighting yields optimal results.

Original: E. C OMPARISONS WITH S EMI - SUPERVISED L EARNING F RAMEWORKS
Revised: E. C OMPARISONS WITH S EMI - SUPERVISED L EARNING F RAMEWORKS

Original: ALOE exploits the unlabeled data to improve the model performance.
Revised: ALOE exploits the unlabeled data to improve the model performance.

Original: This shares the same spirit of semi-supervised learning.
Revised: This shares the same spirit of semi-supervised learning.

Original: We are curious about how a semi-supervised learning method performs in our active anomaly detection setup.
Revised: We are curious about how a semi-supervised learning method performs in our active anomaly detection setup.

Original: To this end, we adapted an existing semi-supervised learning framework FixMatch (Sohn et al., 2020a) to our setup and compared with our method in Fig.
Revised: To this end, we adapted an existing semi-supervised learning framework FixMatch (Sohn et al., 2020a) to our setup and compared with our method in Fig.

Original: As follows, we will first describe the experiment results and then state the adaptation of FixMatch to anomaly detection we made.
Revised: As follows, we will first describe the experiment results and then state the adaptation of FixMatch to anomaly detection we made.

Original: FixMatch, as a semi-supervised learning algorithm, regularizes the image classifier on a large amount of unlabeled data.
Revised: FixMatch, as a semi-supervised learning algorithm, regularizes the image classifier on a large amount of unlabeled data.

Original: The regularization, usually referred to consistency regularization, requires the classifier to have consistent predictions on different views of unlabeled data, thus improves
Revised: The regularization, usually referred to consistency regularization, requires the classifier to have consistent predictions on different views of unlabeled data, thus improves

Original: the
Revised: the

Original: [Figure]
Revised: [Figure]

Original: classifier’s performance.
Revised: classifier’s performance.

Original: FixMatch generates various data views through image augmentations followed by Cutout (DeVries and Taylor, 2017).
Revised: FixMatch generates various data views through image augmentations followed by Cutout (DeVries and Taylor, 2017).

Original: We noticed that, although FixMatch focuses on making use of the unlabeled data, its performance is highly affected by the quality of the labeled data subset.
Revised: We noticed that, although FixMatch focuses on making use of the unlabeled data, its performance is highly affected by the quality of the labeled data subset.

Original: We investigated two variants depending on how we acquire the labeled data.
Revised: We investigated two variants depending on how we acquire the labeled data.

Original: One is the original semi-supervised learning setting, i
Revised: One is the original semi-supervised learning setting, i

Original: [Figure]
Revised: [Figure]

Original: e., assuming the labeled data is a random subset of the whole dataset.
Revised: e., assuming the labeled data is a random subset of the whole dataset.

Original: The other one utilizes the same diversified data querying strategy k-means++ as ALOE to acquire the labeled part.
Revised: The other one utilizes the same diversified data querying strategy k-means++ as ALOE to acquire the labeled part.

Original: In Fig.
Revised: In Fig.

Original: 11, we compared the performance of the two variants with ALOE.
Revised: 11, we compared the performance of the two variants with ALOE.

Original: It shows that, on natural images CIFAR10 for which FixMatch is developped, while the original
Revised: It shows that, on natural images CIFAR10 for which FixMatch is developped, while the original

Original: FixMatch with random labeled data is still outperformed by ALOE, FixMatch with our proposed querying strategy k-means++ has a comparable performance with ALOE.
Revised: FixMatch with random labeled data is still outperformed by ALOE, FixMatch with our proposed querying strategy k-means++ has a comparable performance with ALOE.

Original: However, such advantage of FixMatch diminishes for the gray image dataset F-MNIST, where both variants are beat by ALOE on all querying budgets.
Revised: However, such advantage of FixMatch diminishes for the gray image dataset F-MNIST, where both variants are beat by ALOE on all querying budgets.

Original: In addition, the FixMatch framework is restrictive and may not be applicable for tabular data and medical data, as the augmentations are specially designed for natural images.
Revised: In addition, the FixMatch framework is restrictive and may not be applicable for tabular data and medical data, as the augmentations are specially designed for natural images.

Original: FixMatch is designed for classification.
Revised: FixMatch is designed for classification.

Original: To make it suit for anomaly detection, we adapted the original algorithm 5 and adopted the following procedure and loss function
Revised: To make it suit for anomaly detection, we adapted the original algorithm 5 and adopted the following procedure and loss function

Original: Label all training data as normal and train the anomaly detector for one epoch; 2.
Revised: Label all training data as normal and train the anomaly detector for one epoch; 2.

Original: Actively query a subset of data with size |Q| , resulting in Q and the remaining data U ; 3.
Revised: Actively query a subset of data with size |Q| , resulting in Q and the remaining data U ; 3.

Original: Finetune the detector in a supervised manner on non-augmented Q for 5 epochs; 4.
Revised: Finetune the detector in a supervised manner on non-augmented Q for 5 epochs; 4.

Original: Train the detector with the FixMatch loss Eq.
Revised: Train the detector with the FixMatch loss Eq.

Original: (7) on augmented {U , Q} until convergence.
Revised: (7) on augmented {U , Q} until convergence.

Original: We denote weak augmentation of input x by α ( x ) and the strong augmentation by A ( x ) .
Revised: We denote weak augmentation of input x by α ( x ) and the strong augmentation by A ( x ) .

Original: The training objective function we used is
Revised: The training objective function we used is

Original: [Equation]
Revised: [Equation]

Original: where pseudo labels ˜
Revised: where pseudo labels ˜

Original: y
Revised: y

Original: i = 1 ( S ( α ( x i )) > q 0 . 05 )) and q n is the n -quantile of the anomaly scores { S ( α ( x i ))
Revised: i = 1 ( S ( α ( x i )) > q 0 . 05 )) and q n is the n -quantile of the anomaly scores { S ( α ( x i ))

Original: } i ∈U .
Revised: } i ∈U .

Original: In the loss function, we only use the unlabeled samples with confidently predicted pseudo labels.
Revised: In the loss function, we only use the unlabeled samples with confidently predicted pseudo labels.

Original: This is controlled by the indicator function 1 ( S ( α ( x i )) <
Revised: This is controlled by the indicator function 1 ( S ( α ( x i )) <

Original: q 0 .
Revised: q 0 .

Original: 7 or S ( α ( x i ))
Revised: 7 or S ( α ( x i ))

Original: > q 0 . 05 ) .
Revised: > q 0 . 05 ) .

Original: We apply this loss function for mini-batches on a stochastic optimization basis.
Revised: We apply this loss function for mini-batches on a stochastic optimization basis.

Original: We also extend the semi-supervised learning methods using non-parametric algorithms to our active anomaly detection framework.
Revised: We also extend the semi-supervised learning methods using non-parametric algorithms to our active anomaly detection framework.

Original: We applied k -nearest neighbors and Gaussian process for inferring the latent anomaly labels (Iscen et al., 2019; Li et al., 2018)
Revised: We applied k -nearest neighbors and Gaussian process for inferring the latent anomaly labels (Iscen et al., 2019; Li et al., 2018)

Original: because these algorithms are unbiased in the sense that if the queried sample size is large enough, the inferred latent anomaly labels approach to the true anomaly labels.
Revised: because these algorithms are unbiased in the sense that if the queried sample size is large enough, the inferred latent anomaly labels approach to the true anomaly labels.

Original: For these baselines, we also queried a few labeled data with k-means++
Revised: For these baselines, we also queried a few labeled data with k-means++

Original: -based diverse querying strategy
Revised: -based diverse querying strategy

Original: and then annotate the unqueried samples with k-nearest neighbor classifier or Gaussian process classifer queried
Revised: and then annotate the unqueried samples with k-nearest neighbor classifier or Gaussian process classifer queried

Original: Both methods become ablations of ALOE.
Revised: Both methods become ablations of ALOE.

Original: We compare ALOE with them on CIFAR-10 and F-MNIST under various query budgets and report their results in Fig.
Revised: We compare ALOE with them on CIFAR-10 and F-MNIST under various query budgets and report their results in Fig.

Original: On both datasets, ALOE improves over the variant of using only queried samples for training.
Revised: On both datasets, ALOE improves over the variant of using only queried samples for training.

Original: On F-MNIST, ALOE outperforms all ablations clearly under all query budgets, while on CIFAR-10, ALOE outperforms all ablations except for FixMatch when query budget is low.
Revised: On F-MNIST, ALOE outperforms all ablations clearly under all query budgets, while on CIFAR-10, ALOE outperforms all ablations except for FixMatch when query budget is low.

Original: In conclusion, ALOE boosts the performance by utilizing the unlabeled samples properly, while other labeling strategies are less effective
Revised: In conclusion, ALOE boosts the performance by utilizing the unlabeled samples properly, while other labeling strategies are less effective

Original: [Figure]
Revised: [Figure]

Original: E. M ORE C OMPARISONS
Revised: E. M ORE C OMPARISONS

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Comparisons to kNN (Ramaswamy et al., 2000)
Revised: Comparisons to kNN (Ramaswamy et al., 2000)

Original: We compared against kNN in two ways.
Revised: We compared against kNN in two ways.

Original: First we confirmed that our baseline backbone model NTL is competitive with kNN, which is shown to have a strong performance on tabular data (Shenkar and Wolf, 2022).
Revised: First we confirmed that our baseline backbone model NTL is competitive with kNN, which is shown to have a strong performance on tabular data (Shenkar and Wolf, 2022).

Original: To this end, NTL has been shown to yield 95.7% AUC on clean CIFAR-10 data, see Shenkar and Wolf, 2022, Table 1 column 1.
Revised: To this end, NTL has been shown to yield 95.7% AUC on clean CIFAR-10 data, see Shenkar and Wolf, 2022, Table 1 column 1.

Original: In contrast, Qiu et al.
Revised: In contrast, Qiu et al.

Original: (2022a) reported 96.2% AUC in Table 2, which is very close.
Revised: (2022a) reported 96.2% AUC in Table 2, which is very close.

Original: Second, we tested the performance of the kNN method on our corrupted training data set.
Revised: Second, we tested the performance of the kNN method on our corrupted training data set.

Original: We gave kNN the advantage of using the ground truth contamination ratio (otherwise when under-estimating this value, we saw the method degrade severely in performance).
Revised: We gave kNN the advantage of using the ground truth contamination ratio (otherwise when under-estimating this value, we saw the method degrade severely in performance).

Original: KNN has two key hyperparameters: the number of nearest neighbors k and the assumed contamination ratio of the training set.
Revised: KNN has two key hyperparameters: the number of nearest neighbors k and the assumed contamination ratio of the training set.

Original: The method uses this assumed contamination ratio when fitting to define the threshold on the decision function.
Revised: The method uses this assumed contamination ratio when fitting to define the threshold on the decision function.

Original: In our experiments, we tried multiple values of k and reported the best testing results.
Revised: In our experiments, we tried multiple values of k and reported the best testing results.

Original: Although the ground truth anomaly rate is unknown and our proposed methods don’t have access to it, we gave kNN the competitive advantage of knowing the ground truth contamination ratio.
Revised: Although the ground truth anomaly rate is unknown and our proposed methods don’t have access to it, we gave kNN the competitive advantage of knowing the ground truth contamination ratio.

Original: We studied the same tabular data sets as in our paper: BreastW, Ionosphere, Pima, and Satellite.
Revised: We studied the same tabular data sets as in our paper: BreastW, Ionosphere, Pima, and Satellite.

Original: We used the same procedure for constructing contaminated data outlined in our paper, where the contamination ratio was set to 10%.
Revised: We used the same procedure for constructing contaminated data outlined in our paper, where the contamination ratio was set to 10%.

Original: The results are summarized in Tab. 8.
Revised: The results are summarized in Tab. 8.

Original: We adopted PyOD’s implementation of kNN 6 and set all the other hyperparameters to their default values (“method, radius, algorithm, leaf_size, metric, p, and metric_params”).
Revised: We adopted PyOD’s implementation of kNN 6 and set all the other hyperparameters to their default values (“method, radius, algorithm, leaf_size, metric, p, and metric_params”).

Original: We repeated the experiments 10 times and reported the mean and standard deviation of the F1 scores in Tab. 8.
Revised: We repeated the experiments 10 times and reported the mean and standard deviation of the F1 scores in Tab. 8.

Original: We find that our active learning framework outperforms the kNN baseline.
Revised: We find that our active learning framework outperforms the kNN baseline.

Original: In more detail, the F1 scores for different values of k are listed below, where k
Revised: In more detail, the F1 scores for different values of k are listed below, where k

Original: = 1 , 2 , 5 , 10 , 15 , 20 , respectively:
Revised: = 1 , 2 , 5 , 10 , 15 , 20 , respectively:

Original: • BreastW: 84.3 ± 7.6, 86.5 ± 3.1, 89.9 ± 3.9, 90.7 ± 3.4, 92.5 ± 2.1, 91.9 ± 1.
Revised: • BreastW: 84.3 ± 7.6, 86.5 ± 3.1, 89.9 ± 3.9, 90.7 ± 3.4, 92.5 ± 2.1, 91.9 ± 1.

Original: • Ionosphere: 88.1 ± 1.3, 87.6 ± 2.6, 84.5 ± 3.9, 75.2 ± 2.5, 70.4 ± 3.6, 67.4 ± 3.
Revised: • Ionosphere: 88.1 ± 1.3, 87.6 ± 2.6, 84.5 ± 3.9, 75.2 ± 2.5, 70.4 ± 3.6, 67.4 ± 3.

Original: • Pima: 34.4 ± 3.6, 32.3 ± 3.4, 36.9 ± 6.4, 40.5 ± 4.7, 35.3 ± 3.6, 35.5 ± 4.
Revised: • Pima: 34.4 ± 3.6, 32.3 ± 3.4, 36.9 ± 6.4, 40.5 ± 4.7, 35.3 ± 3.6, 35.5 ± 4.

Original: • Satellite: 51.0 ± 1.1, 53.5 ± 0.7, 54.7 ± 1.3, 57.4 ± 1.8, 59.3 ± 1.3, 61.1 ± 2.
Revised: • Satellite: 51.0 ± 1.1, 53.5 ± 0.7, 54.7 ± 1.3, 57.4 ± 1.8, 59.3 ± 1.3, 61.1 ± 2.

Original: Comparisons to Gradient Diversity Querying Strategy (BADGE) (Ash et al., 2020)
Revised: Comparisons to Gradient Diversity Querying Strategy (BADGE) (Ash et al., 2020)

Original: We compared against a popular active learning method, BADGE (Ash et al., 2020), which is a diversity-driven active learning method that exploits sample-wise gradient diversity.
Revised: We compared against a popular active learning method, BADGE (Ash et al., 2020), which is a diversity-driven active learning method that exploits sample-wise gradient diversity.

Original: We start with observing that
Revised: We start with observing that

Original: BADGE doesn’t work well for anomaly detection in Fig.
Revised: BADGE doesn’t work well for anomaly detection in Fig.

Original: 12, where we only replaced the objects that k-means++ works on in ALOE with gradients demanded in BADGE (Ash et al., 2020) while keeping all other settings fixed.
Revised: 12, where we only replaced the objects that k-means++ works on in ALOE with gradients demanded in BADGE (Ash et al., 2020) while keeping all other settings fixed.

Original: This variant is referred to as "Gradient Diversity" while ours is denoted by "Representation Diversity".
Revised: This variant is referred to as "Gradient Diversity" while ours is denoted by "Representation Diversity".

Original: Fig.
Revised: Fig.

Original: 12 shows the performance of Gradient Diversity is outperformed by a large margin, failing in querying informative samples as our Representation Diversity.
Revised: 12 shows the performance of Gradient Diversity is outperformed by a large margin, failing in querying informative samples as our Representation Diversity.

Original: To understand which part of BADGE breaks for anomaly detection tasks, we check the gradients used by BADGE in an anomaly detection model.
Revised: To understand which part of BADGE breaks for anomaly detection tasks, we check the gradients used by BADGE in an anomaly detection model.

Original: Before that, we start with describing how BADGE works.
Revised: Before that, we start with describing how BADGE works.

Original: BADGE is developed for active learning in classification tasks.
Revised: BADGE is developed for active learning in classification tasks.

Original: Given a pre-trained classifier, it first predicts the most likely label ˆ y (pseudo labels) for the unlabeled training data x .
Revised: Given a pre-trained classifier, it first predicts the most likely label ˆ y (pseudo labels) for the unlabeled training data x .

Original: These pseudo labels are then used to formulate a cross entropy loss l CE ( x , ˆ y ) .
Revised: These pseudo labels are then used to formulate a cross entropy loss l CE ( x , ˆ y ) .

Original: BADGE computes every data point’s loss function’s gradient to the final layer’s weights as the data’s representation.
Revised: BADGE computes every data point’s loss function’s gradient to the final layer’s weights as the data’s representation.

Original: Upon active querying, a subset of data are selected such that their representations are diverse.
Revised: Upon active querying, a subset of data are selected such that their representations are diverse.

Original: In particular, the gradient to each class-specific weight W k is ∇ W k l CE ( x , ˆ y ) =
Revised: In particular, the gradient to each class-specific weight W k is ∇ W k l CE ( x , ˆ y ) =

Original: ( p k − 1 (ˆ y = k ))
Revised: ( p k − 1 (ˆ y = k ))

Original: ϕ
Revised: ϕ

Original: ( x ) where p k is the predicted probability of being class k and ϕ ( x ) is the output of the penultimate layer.
Revised: ( x ) where p k is the predicted probability of being class k and ϕ ( x ) is the output of the penultimate layer.

Original: Proposition 1 of Ash et al.
Revised: Proposition 1 of Ash et al.

Original: (2020) shows the norm of the gradient with pseudo labels is a lower bound of the one with true labels.
Revised: (2020) shows the norm of the gradient with pseudo labels is a lower bound of the one with true labels.

Original: In addition, note that the gradient is a scaling of the penultimate layer output.
Revised: In addition, note that the gradient is a scaling of the penultimate layer output.

Original: The scaling factor describes the predictive uncertainty and is upper bounded by 1.
Revised: The scaling factor describes the predictive uncertainty and is upper bounded by 1.

Original: Therefore, the gradients are informative surrogates of the penultimate layer output of the network, as shown by the inequality
Revised: Therefore, the gradients are informative surrogates of the penultimate layer output of the network, as shown by the inequality

Original: [Equation]
Revised: [Equation]

Original: However, these properties are associated with the softmax activation function usage.
Revised: However, these properties are associated with the softmax activation function usage.

Original: In anomaly detection, models and losses are diverse and are beyond the usage of softmax activation outputs.
Revised: In anomaly detection, models and losses are diverse and are beyond the usage of softmax activation outputs.

Original: Hence the gradients are no longer good ways to construct active queries.
Revised: Hence the gradients are no longer good ways to construct active queries.

Original: For example, the supervised deep
Revised: For example, the supervised deep

Original: SVDD (Ruff et al., 2019) uses the contrasting loss l ( x , y )
Revised: SVDD (Ruff et al., 2019) uses the contrasting loss l ( x , y )

Original: =
Revised: =

Original: y/ ( Wϕ ( x ) − c ) 2 +(1 − y )
Revised: y/ ( Wϕ ( x ) − c ) 2 +(1 − y )

Original: ( Wϕ ( x ) − c )to compact the normal sample representations around center c .
Revised: ( Wϕ ( x ) − c )to compact the normal sample representations around center c .

Original: However, the gradient ∇ W l ( x , y ) = (cid:0) 2(1 − y )
Revised: However, the gradient ∇ W l ( x , y ) = (cid:0) 2(1 − y )

Original: ( Wϕ ( x ) − c ) − 2 y
Revised: ( Wϕ ( x ) − c ) − 2 y

Original: ( Wϕ ( x ) − c ) − 3 (cid:1) ϕ
Revised: ( Wϕ ( x ) − c ) − 3 (cid:1) ϕ

Original: ( x ) is not a bounded scaling of ϕ ( x ) any more, thus not an informative surrogate of point x .
Revised: ( x ) is not a bounded scaling of ϕ ( x ) any more, thus not an informative surrogate of point x .

Original: E. NTL AS A U NIFIED B ACKBONE M ODEL
Revised: E. NTL AS A U NIFIED B ACKBONE M ODEL

Original: In Section 4 of the main paper, we have empirically compared ALOE to active-learning strategies known from various existing papers, where these strategies originally were proposed using different backbone architectures (either shallow methods or simple neural architectures, such as autoencoders).
Revised: In Section 4 of the main paper, we have empirically compared ALOE to active-learning strategies known from various existing papers, where these strategies originally were proposed using different backbone architectures (either shallow methods or simple neural architectures, such as autoencoders).

Original: However, several recent benchmarks have revealed that these backbones are no longer competitive with modern self-supervised ones (Alvarez et al., 2022).
Revised: However, several recent benchmarks have revealed that these backbones are no longer competitive with modern self-supervised ones (Alvarez et al., 2022).

Original: For a fair empirical comparison of ALOE to modern baselines, we upgraded the previously proposed active-learning methods by replacing their
Revised: For a fair empirical comparison of ALOE to modern baselines, we upgraded the previously proposed active-learning methods by replacing their

Original: We motivate our choice of NTL as unified backbone in our experiments as follows.
Revised: We motivate our choice of NTL as unified backbone in our experiments as follows.

Original: Fig.
Revised: Fig.

Original: 13 shows the results of ten shallow and deep anomaly detection methods (Qiu et al., 2022a; Deecke et al., 2018;
Revised: 13 shows the results of ten shallow and deep anomaly detection methods (Qiu et al., 2022a; Deecke et al., 2018;

Original: Ruff et al., 2018; Golan and El-Yaniv, 2018; Hendrycks et al., 2019; Tax and Duin, 2004; Liu et al., 2008; Diederik P. Kingma, 2014; Makhzani and Frey, 2015; Sohn et al., 2020b) on the CIFAR one-vs.-rest anomaly detection task.
Revised: Ruff et al., 2018; Golan and El-Yaniv, 2018; Hendrycks et al., 2019; Tax and Duin, 2004; Liu et al., 2008; Diederik P. Kingma, 2014; Makhzani and Frey, 2015; Sohn et al., 2020b) on the CIFAR one-vs.-rest anomaly detection task.

Original: NTL performs best (by a large margin) among the compared methods, including many classic backbone models known from the active anomaly detection literature (Trittenbach et al., 2021;
Revised: NTL performs best (by a large margin) among the compared methods, including many classic backbone models known from the active anomaly detection literature (Trittenbach et al., 2021;

Original: Ruff et al., 2019; Görnitz et al., 2013; Das et al., 2019; Pimentel et al., 2020;
Revised: Ruff et al., 2019; Görnitz et al., 2013; Das et al., 2019; Pimentel et al., 2020;

Original: Ning et al., 2022; Barnabé-Lortie et al., 2015).
Revised: Ning et al., 2022; Barnabé-Lortie et al., 2015).

Original: -of-the-art for tabular anomaly detection
Revised: -of-the-art for tabular anomaly detection

Original: [Table]
Revised: [Table]

Original: An independent benchmark comparison of 13 methods (including nine deep methods proposed in 2018–2022) (Alvarez et al., 2022) recently identified NTL as the leading anomaly-detection method on tabular data.
Revised: An independent benchmark comparison of 13 methods (including nine deep methods proposed in 2018–2022) (Alvarez et al., 2022) recently identified NTL as the leading anomaly-detection method on tabular data.

Original: In their summary, the authors write: ’NeuTraLAD, the transformation-based approach, offers consistently above-average performance across all datasets.
Revised: In their summary, the authors write: ’NeuTraLAD, the transformation-based approach, offers consistently above-average performance across all datasets.

Original: The data-augmentation strategy is particularly efficient on small-scale datasets where samples are scarce.’.
Revised: The data-augmentation strategy is particularly efficient on small-scale datasets where samples are scarce.’.

Original: Note that the latter is also the scenario where active learning is thought to be the most promising.
Revised: Note that the latter is also the scenario where active learning is thought to be the most promising.

Original: We show the results from Alvarez et al.
Revised: We show the results from Alvarez et al.

Original: (2022) in Tab. 9.
Revised: (2022) in Tab. 9.