Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: #### Problem Statement

This paper considers the effect of bias in anomaly detection. The anomaly detection setup is this: A model is trained on a $1- \alpha, \alpha$ mixture of normal and anomalous examples . The model returns an anomaly score $s(x)$ for each point $x$. We then chose a threshold $\tau$ which ensures a bound on the false  positive rate, while maximizing the number of true anomalies that are labelled $1$. 

There are two different sources of bias that the paper considers:

1. Bias coming from not having sufficient samples. Here the paper proves theoretical results quantifying the convergence in terms of the quantile functions and experiments confirming these bounds.

2. Bias from the data, where the set of anomalies available at training is not representative of the test distribution of anomalies. This is more of a domain adaptation problem. Here they present experimental results showing that the behavior of algorithms can in some settings be worse than if there were no anomalies at all.


 ### Pros

1. The paper is easy to read and scholarly. The definitions are precise, the literature survey is thorough. The process of choosing a scoring threshold that they describe is folklore, but it is nice that they formalize it and prove guarantees about it. 

2. The experiments related to bias in scoring resulting from bias in the training data are interesting. They have a class of normal points("Tops" in FashionMNIST), and experiment with different choices of anomalies, those are are visually  similar (eg. shirts) and those that are dissimilar (eg. shoes). They find that having only "similar" anomalies while training can detract from the ability  to find "dissimilar" anomalies at test, while improving on the ability to find "similar" anomalies. But having "dissimilar" anomalies even at training does not seem to affect other classes adversely.

### Cons


1. The sample complexity bounds are what you would expect given known bounds on the convergence rate of empirical quantiles to the true quantiles. 

2. In the experiments, I found the setup to be a little artificial. In particular, in the "similar" setting, when the model is only given "tops" as normal and "shirts" as abnormal, it is not too surprising that the model is good at distinguishing "top"-like things,  but not so good as distinguishing boots from tops. This is an extreme case of data bias where you only have several distinct kids of anomalies but only one kind is available for training.  A more convincing experiment might be to use different mixture weights for the anomaly groups (like boots, shirts, dresses) at train and test time. It would also be nice to use more "real-world" datasets, say from the [ODDS dataset.](http://odds.cs.stonybrook.edu/)

3. The terminology of unsupervised and semi-supervised that this paper uses is not standard (although this lakc of standardization might be the fault of the area in general, rather than this paper in particular). The [survey by Chandola and Banerjee](http://cucis.ece.northwestern.edu/projects/DMS/publications/AnomalyDetection.pdf) uses semi-supervised for what this paper calls unsupervised (where we only have normal examples at training time). Unsupervised there refers to the setting where one has no labels at all. This class would not a have a name under this paper's current taxonomy. Given that the survey has over 8000 citations by now, I strongly suggest the authors adopt that terminology. 


### Summary 

As I see it, the paper has two distinct contributions: studying the biasing effect of limited samples and skewed data.  While I liked both aspects of the paper, I can't say either by itself is a particularly strong or surprising result. But they fit well together, and bring rigor to an area which could use more of it. I lean towards accepting the paper. 

review_title: Weak accept

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper studies the potential bias in deep semi-supervised anomaly detection. The bias is evaluated in terms of TPR rate given a fixed FPR rate. It uses the anomaly scores output by unsupervised anomaly detectors as a benchmark to examine the relative scoring bias in deep semi-supervised anomaly detectors. It further studies the finite sample rate for this type of scoring bias. This type of bias is verified using some synthetic and real-world datasets. The empirical results also show the potential impact of this bias on several anomaly detectors.

Overall, the paper is well written and studies an important problem in anomaly detection. A number of theoretical and empirical results are presented to justify the arguments.

However, there are some major issues. Particularly, the three claimed contributions are weak and/or built upon some inappropriate foundations. The first contribution is something partially or fully demonstrated in several exiting work  (Pang et al. (2019); Daniel et al. (2019); Yamanaka et al. (2019); Ruff et al. (2020b;a)). It is a quite straightforward phenomenon. Using the labeled anomaly data to train and validate the trained model of course results in some sort of inductive bias. This is a fundamental assumption in supervised learning like classification. In anomaly detection, such bias is often desired in the sense that we want our detectors to identify anomalies similar to those labeled anomaly data. By contrast, there can be novel types of anomalies that can be very different from the known anomalies. Those novel anomalies cannot be detected if fitting only to the known anomalies. Therefore, this conclusion is straightforward. Further, as anomaly detection has this type of crucial difference compared to classification, the idea of maximizing the TPR rate given a fixed FPR rate using a small labeled anomaly data, or the formulation of semi-supervised anomaly detection as a general supervised learning problem,  is ill-posed. This lets me doubt the importance of this work.

I also have major concerns over the definition of relative scoring bias. Why is it reasonable to use a unsupervised anomaly detector to serve as an approximator of the optimal anomaly detector? It does not make much sense to me. However, this is the fundamental assumption of the whole work. 

Some other issues. (1) The title is also inappropriate. It should explicitly limit the scope to deep anomaly detection. This is what this work is about. (2) There are inconsistent  results in terms of the trending in tables 2-4 and tables 5-7, such as Deep SAD. Why would this happen? Also, the performance of each detector on individual datasets varies significantly. This should also be a key factor to be carefully considered before drawing any conclusions. Additionally, what is the standard deviation of these results? if it is large, we may also need to consider the contribution of this factor in the 'bias'.
review_title: Good work but have major issues in the fundamental assumptions

final_score: rating: 7: Good paper, accept
review_content: 
**UPDATE**

I acknowledge that I have read the author responses as well as the other reviews. I appreciate the clarifications and improvements made during the rebuttal phase, which I think have further strengthened this work.

I find the key contributions of this work to be (i) demonstrating that recent methods that include labeled anomalies into training can suffer from unfavorable biases, and (ii) providing a framework for a theoretical analysis of this setting.

Though I see that the presented results are somewhat what one would expect, to my knowledge such an analysis hasn't been carried out in the existing literature.

Since weak forms of supervision (here few labeled anomalies) appears to be a promising research direction for anomaly detection, I find this critical and rigorous analysis to be worth circulating the community.

For these reasons, I would keep my recommendation to accept this work (score: 7)

#####


**Summary**

This paper studies biases in semi-supervised anomaly detection which is the setting where in addition to mostly nominal data a few labeled anomalies are also available for training. A theoretical framework for the semi-supervised setting is introduced that is based on binary classification which formulates the objective of a scorer as seeking to maximize the detection recall (true positive rate (TPR)) at a given target false positive rate (FPR). Using this framework, a relative scoring bias is derived that enables to assess the relative performance difference between unsupervised and semi-supervised detectors. Furthermore, finite sample rates are derived for this relative scoring bias, which subsequently are also validated empirically via synthetic simulations. Finally, an empirical evaluation that includes six recent state-of-the-art deep anomaly detection methods (Deep SVDD, Deep SAD, HSC, AE, SAE, and ABC) is presented on Fashion-MNIST, Statlog (Landsat Satellite), and ImageNet that demonstrates and highlights scenarios where the bias of a labeled (unrepresentative) anomaly set can be useful, but also harmful for anomaly detection performance.


**Pros**
+ The paper presents a novel theoretical PAC framework for analyzing and understanding bias in semi-supervised anomaly detection. The framework extends a previous classification-based view on anomaly detection [4] to the semi-supervised setting.
+ Deep semi-supervised anomaly detection methods [2, 5, 7] that aim to include and learn from labeled anomalies is a timely topic of high practical relevance.
+ The experimental evaluation demonstrates that including labeled anomalies might introduce an unfavorable bias that can decrease detection performance, which is an important insight.
+ The paper has a clear structure and is easy to follow.

**Cons**
- Some related work is missing, especially previous classification-based views on anomaly detection [8].
- There are some questions left open (see below).
- The current manuscript includes some (minor) typos that should be fixed.


**Recommendation**

I recommend to accept this paper.

The paper presents a well-motivated and useful theoretical framework for the timely and relevant semi-supervised anomaly detection setting. The arguments and derivations are technically correct. To my knowledge, this is also the first instance of a finite sample complexity bound on the scoring bias for this setting. The theoretical claims are validated through simulations and tested on real-world datasets in a scientifically rigorous manner. An important message of the analysis is that including labeled anomalies can introduce a bias that can be harmful for anomaly detection performance. In this regard, I think the paper also covers important ground for future analysis and towards building semi-supervised models that are unbiased.


**Questions**

(1) How does the presented view compare to well-known previous classification-based views [8]?

(2) ‘for $\xi$, it also converges to a certain level.’ Specifically the level predicted by the bound of Theorem 3? 

(3) How did you stabilize maximizing the reconstruction error for labeled anomalies in SAE? I suspect optimizing this objective is 
unstable and prone to blow up.

(4) Scenario 2 would make a compelling case for using Outlier Exposure [1]. Did you conduct such experiments similar to [6]?

(5) At the end of Section 3, the empirical TPR estimate should have $s(x_j) > \tau$ in the indicator function, correct?

(6) In the infinite sample case in Section 4.1, do you refer to the Glivenko–Cantelli theorem when citing Parzen (1980)?


**Additional feedback and ideas for improvement**
- Include Outlier Exposure [1] in the experimental analysis.
- There exists further recent related work on biases in anomaly detection that observes that detectors may correctly detect anomalies, but based on wrong (spurious) features [3]. This should be added to the list of works studying biases in anomaly detection.


**Minor Comments**
1. The last paragraph in the Introduction, in which the contributions are listed, is a bit repetitive after the preceding paragraphs.
2. In Section 4.1, $F_0(t)$ and $F_a(t)$ should have $s(x) \leq t$ in their definition to be consistent, right?
3. The notation for the number of anomalous training samples mixes $m$ and $n_1$.
4. Proposition 1: ‘[...], the relative scoring bias *is* [...]’
5. In Section 4.1, after Proposition 1, the $\text{TPR}(s', \tau')$ function is missing parentheses.
6. Corollary 2: ‘Let $q$ be a fixed target FPR. [...] Then, the relative scoring bias is [...]’
7. Note that $\Phi$ denotes the cdf of the standard Gaussian.
8. Finite sample case: ‘[...], where we follow the convention to assume *that the anomaly data* amounts to [...]’
9. Theorem 3: The ‘-’ in the cdf superscripts should be ‘-1’. 10. There are spaces missing after ‘i.i.d.’ in the text.


#####

**References**

[1] D. Hendrycks, M. Mazeika, and T. G. Dietterich. Deep anomaly detection with outlier exposure. In ICLR, 2019.

[2] D. Hendrycks, M. Mazeika, S. Kadavath, and D. Song. Using self-supervised learning can improve model robustness and uncertainty. In NeurIPS, pages 15637–15648, 2019.

[3] J. Kauffmann, L. Ruff, G. Montavon, and K.-R. Müller. The Clever Hans effect in anomaly detection. arXiv preprint arXiv:2006.10609, 2020.

[4] S. Liu, R. Garrepalli, T. Dietterich, A. Fern, and D. Hendrycks. Open category detection with PAC guarantees. In ICML, volume 80, pages 3169–3178, 2018.

[5] G. Pang, C. Shen, and A. van den Hengel. Deep anomaly detection with deviation networks. In KDD, pages 353–362, 2019.

[6] L. Ruff, R. A. Vandermeulen, B. J. Franks, K.-R. Müller, and M. Kloft. Rethinking assumptions in deep anomaly detection. arXiv preprint arXiv:2006.00339, 2020.

[7] L. Ruff, R. A. Vandermeulen, N. Görnitz, A. Binder, E. Müller, K.-R. Müller, and M. Kloft. Deep semi-supervised anomaly detection. In ICLR, 2020.

[8] I. Steinwart, D. Hush, and C. Scovel. A classification framework for anomaly detection. Journal of Machine Learning Research, 6(Feb):211–232, 2005.

review_title: Review AnonReviewer1

final_score: rating: 6: Marginally above acceptance threshold
review_content: ======================
Additional reviews
======================
The authors have resolved some concerns, especially the explanation/justification of the experimental results in section 5.
As I have commented in the initial review, this paper provides suggestive experimental results (the effects of training anomaly dataset) for future anomaly detection research.
Therefore, I have decided to raise my rating from 5 to 6.
======================

Summary:
- This paper investigates the effects of biased training anomaly datasets on anomaly detection problems.
Specifically, this paper proposes the relative scoring bias, which is the difference of TPRs of two anomaly detectors when the FPR is below a certain value, to model the effects of a biased training anomaly dataset.
In addition, this paper also presents the finite sample rates for estimating the relative scoring bias.
This paper empirically analyzes the effect of a biased training anomaly dataset on the detection performance.

Pros:
- Investigating the effect of training anomaly datasets on anomaly detection is important and useful in anomaly detection studies.
- This paper establishes the finite sample rates for estimating the relative scoring bias for semi-supervised anomaly detection.
- Empirical evaluation results (especially, the results in Section 5) are interesting.

Cons:
- Evaluation for the convergence of the relative scoring bias/FPRs was not conducted with the real-world datasets.

Detailed comments and questions:
- As the paper mentioned, semi-supervised anomaly detection, which uses labeled anomaly data as well as normal/unlabeled data for training, has become a promising approach to improve the performance of anomaly detectors. 
Since all types of anomaly data are difficult to collect due to the rarity of anomalies, it is typical to use biased anomaly datasets for learning the semi-supervised anomaly detectors. 
Therefore, analyzing how/when the biased training anomaly dataset affects performance is important/useful in this research area. 

- Although the results in Section 5 are really interesting/useful, can the authors explain why the phenomenon of results 5 occurs? That is, why performance on test data dissimilar to the training anomaly data (do not) becomes worse when anomaly and normal training data are similar (dissimilar)?  

- The paper only evaluated the convergence of the relative scoring bias/FPRs on the simple synthetic dataset. Can the authors investigate it using the real-world datasets to empirically validate the Theorem 3?

- Figure 2 is a bit difficult to see. In particular, it is difficult to compare the variation of $\xi$ at $\alpha=0.2$ between $\alpha=0.01$ when $n=10000$.

Reasons for Score:
- To validate the theoretical results empirically, I would like to see the experimental results about the convergence of the relative scoring bias/FPRs with real-world datasets.
Although my current rating is 5, I would like to raise my score when the above questions are resolved.
Since I'm not familiar with the PAC framework, I cannot judge the novelty of theoretical results (Theorem 3) well. Therefore, my confidence is 2.

Minor commetns:
- There are some typos:
 - p4. Algorithm 1: FPR (recall) -> TPR (recall)
 - below Eq. (3.5) in p4. $s(x_j) \leq \tau $ ->  $s(x_j) \geq \tau $
 - p8. Fashion-MNISST -> Fashion-MNIST

review_title: Important investigation of the effect of biased anomaly datasets in semi-supervised anomaly detection. A more deep analysis of the theoretical and experimental results would improve the quality of this work.

Meta-review: This paper studies the effect of anomaly detection using supervised learning with non-representative abnormal examples on the TPR of the anomaly detection model. Experiments demonstrate that when the abnormal examples presented in the training set are not representative of the abnormal examples in the target distribution, this can lead to bias in estimating the TPR. 

Pros: 
- This paper considers an important issue of tuning anomaly detection models in the absence of representative anomalies.
- The paper is well written and easy to read.
Cons:
- The analysis and experiments provided in the paper are not surprising, and repeat, although perhaps in more detail, known effects of learning with a non-representative training sample. 
Decision: Reject