Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: After reading all the reviews and the comments, I feel more positive about the paper. I appreciate the feedback of the Authors and I have decided to increase the rating.

============================

The paper proposes using Robust Subspace Recovery in combination with an autoencoder (and possibly GANs) for anomaly detection. The encoder maps input data to the latent space of dimensionality D, which then is linearly projected to a subspace of dimensionality d (d < D). The projection of the latent space then goes to a decoder that reconstructs the input.
A transformation matrix A is trained jointly with the autoencoder. Two additional terms are added to the loss: one to encourage the subspace of A^TA to approximate the latent space z and the second one to force it to be an orthogonal projector.
 
The paper claims to generalize the existing RSR framework to the nonlinear case. However, the linear RSR is applied to the latent space of the autoencoder. In addition to that, all the following discussion and proofs are limited to the linear case. 

Since the proposed method is using RSB as it’s core part, and claims to be a non-linear extension of it, it would be crucial to have a comparison with RSB, at least on those experimental setups, where high-level features are used (Tiny Imagenet with ResNET features, Reuters-21578, and 20 Newsgroups). However, there is no such comparison.

Since autoencoders can potentially learn any, arbitrary entangled latent space, it is not clear why outliers should necessarily have such embedding that is outside of the learned subspace. In the case of the original RSR it happens due to the dimensionality reduction by the orthogonal projector. However, autoencoders already perform dimensionality reduction at each layer down to the bottleneck layer.

The matrix A and the parameters of the AE are trained jointly. So, it can be seen that two processes can occur:
- The AE in order to minimize the reconstruction error would learn such latent space z, that would fit into the subspace of A^TA, so that projection \tilde z =Az doesn’t cause data loss.
- The AE in order to minimize the reconstruction error would learn such A, so that the subspace that z approximates is the best possible.

It is not clear, which of the two cases would take place. If the first one would dominate, then it is not clear if such method would have any discriminating capabilities.
 
My point is mainly that the presented work is not really a generalization of RSR as it claims to be, but rather it is just using RSR on a leaned embedding of the data. 

Some citations are missing, as well as it is missing a comparison to some state-of-the art methods such as OCNN ‘Robust, Deep and Inductive Anomaly Detection’ ECML 2017;   ‘Adversarially Learned One-Class Classifier for Novelty Detection’ CVPR 2017; DSVDD ‘Deep one-class classification.’ ICML, 2018; ODIN  ‘Enhancing The Reliability  Of Out-of-distribution Image Detection  In Neural Networks’ ICLR 2018; ‘Generative Probabilistic Novelty Detection with Adversarial Autoencoders’ NeurIPS 2018.

review_title: Official Blind Review #1

final_score: rating: 8: Accept
review_content: This paper adapts the concept of Robust Subspace Recovery (RSR) as a layer in an auto-encoder model for anomaly detection. A loss function is proposed that combines reconstruction error and a regularizer that enforces robustness against outliers. The reconstruction error expresses the accuracy of the nonlinear dimensionality reduction imposed by the autoencoder. The regularizer is the sum of absolute deviations from the latent subspace that represents a linear structure robust against outliers. An alternative procedure is applied where the loss terms are applied iteratively during training. Once trained, the reconstruction error is used directly for anomaly detection with a threshold. The AUC is used as a performance measure. The method is compared against 6 other methods (LOF, OCSVM, IF, DESBM, GT, DAGMM). The setting is fully unsupervised, meaning that the training data contains various amounts of anomalies, and the results are parametrized with the amount of corruption. The results show that the proposed approach outperforms the other methods in most cases, especially for larger amounts of corruption. An ablation study compares the approach with auto-encoder-only and a non-alternating gradient descent (fixed factors for each part of the loss function) and shows that the alternating method outperfroms all by a wide margin.


PROS:

* A novel approach to fully unsupervised anomaly detection that beats the state of the art.

* The RSR layer is a simple fully connected layer and the loss function is simple to calculate, making the approach computationally efficient.

* A pseudo-code algorithm is provided in the appendix, which should help reproducibility. 

* The paper is well written and the math is clearly laid out.

* The result benchmarks are sufficiently exhaustive in both the methods that are compared and the datasets used.

* The ablation study is informative and shows the effect of the regularization term of the loss function as well as the effect of alternating the gradient descent with the separate losses.



CONS:

* There is a serious problem in the results (Figure 1) as the AP curves show better scores for larger corruption factors. Are the AP-score graphs flipped ? Please explain.

* The AUC and AP scores need to be defined. 

* The results should include the case where the training data is not contaminated with outliers (c=0). This would correspond to the semi-supervised scenario and it would be very interesting to see how the method compares to DAGMM and GT which are build for that scenario.

* It would be interesting to see the effect of varying the subspace dimension. The authors chose 10 for all experiments, why is this number chosen, what would be the effect of choosing a smaller one ? This is a key parameter as it defines the structure of the projection subspace. Should this parameter be systematically tuned for each dataset ?


Overall this is a good paper proposing a novel approach to fully unsupervised anomaly detection with state-of-the art results.

review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: This paper proposes to use the robust subspace recovery layer (RSR) in the autoencoder model for unsupervised anomaly detection.
This paper is well written overall. Presentation is clear and it is easy to follow.
The proposed approach is a simple combination of existing approaches.
Although its theoretical analysis with respect to the performance of anomaly detection is limited, experiments show that the proposed method is effective and superior to the existing anomaly detection methods.

I have the following comments:
- Parameter sensitivity should be examined.
    The proposed method has the number of parameters including \lambda_1, \lambda_2, and parameters in neural networks.
    Since parameter tuning is fundamentally difficult in the unsupervised setting, the sensitivity of the proposed method with respect to changes of such parameters should be examined.
- Since the efficiency is also an important issue for anomaly detection methods, runtime comparison would be interesting.
- It would be also interesting whether the proposed method is also effective for non-structured data, where a dataset is given as just a set of (real-valued) feature vectors, with its comparison to the standard anomaly detection methods such as LOF and iForest.

review_title: Official Blind Review #3

Meta-review: Three reviewers have assessed this paper and they have scored it as 6/6/6/6 after rebuttal. Nonetheless, the reviewers have raised a number of criticisms and the authors are encouraged to resolve them for the camera-ready submission.
Decision: Accept (Poster)