Venue: ICLR 2018 conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper presents a new technique for anomaly detection where the dimension reduction and the density estimation steps are jointly optimized. The paper is rigorous and ideas are clearly stated. The idea to constraint the dimension reduction to fit a certain model, here a GMM, is relevant, and the paper provides a thorough comparison with recent state-of-the-art methods. My main concern is that the method is called unsupervised, but it uses the class information in the training, and also evaluation. I'm also not convinced of how well the Gaussian model fits the low-dimensional representation and how well can a neural network compute the GMM mixture memberships.

1. The framework uses the class information, i.e., “only data samples from the normal class are used for training”, but it is still considered unsupervised. Also, the anomaly detection in the evaluation step is based on a threshold which depends on the percentage of known anomalies, i.e., a priori information. I would like to see a plot of the sample energy as a function of the number of data points. Is there an elbow that indicates the threshold cut? Better yet it would be to use methods like Local Outlier Factor (LOF) (Breunig et al., 2000 – LOF:Identifying Density-based local outliers) to detect the outliers (these methods also have parameters to tune, sure, but using the known percentage of anomalies to find the threshold is not relevant in a purely unsupervised context when we don't know how many anomalies are in the data).
2. Is there a theoretical justification for computing the mixture memberships for the GMM using a neural network? 
3. How do the regularization parameters \lambda_1 and \lambda_2 influence the results?
4. The idea to jointly optimize the dimension reduction and the clustering steps was used before neural nets (e.g., Yang et al., 2014 -  Unsupervised dimensionality reduction for Gaussian mixture model). Those approaches should at least be discussed in the related work, if not compared against.
5. The authors state that estimating the mixture memberships with a neural network for GMM in the estimation network instead of the standard EM algorithm works better. Could you provide a comparison with EM?
6. In the newly constructed space that consists of both the extracted features and the representation error, is a Gaussian model truly relevant? Does it well describe the new space? Do you normalize the features (the output of the dimension reduction and the representation error are quite different)? Fig. 3a doesn't seem to show that the output is a clear mixture of Gaussians.
7. The setup of the KDDCup seems a little bit weird, where the normal samples and anomalies are reversed (because of percentage), where the model is trained only on anomalies, and it detects normal samples as anomalies ... I'm not convinced that it is the best example, especially that is it the one having significantly better results, i.e. scores ~ 0.9 vs. scores ~0.4/0.5 score for the other datasets.
8. The authors mention that “we can clearly see from Fig. 3a that DAGMM is able to well separate ...” - it is not clear to me, it does look better than the other ones, but not clear. If there is a clear separation from a different view, show that one instead. We don't need the same view for all methods. 
9. In the experiments the reduced dimension used is equal to 1 for two of the experiments and 2 for one of them. This seems very drastic!

Minor comments:

1. Fig.1: what dimension reduction did you use? Add axis labels.
2. “DAGMM preserves the key information of an input sample” - what does key information mean?
3. In Fig. 3 when plotting the results for KDDCup, I would have liked to see results for the best 4 methods from Table 1, OC-SVM performs better than PAE. Also DSEBM-e and DSEBM-r seems to perform very well when looking at the three measures combined. They are the best in terms of precision.
4. Is the error in Table 2 averaged over multiple runs? If yes, how many?

Quality – The paper is thoroughly written, and the ideas are clearly presented. It can be further improved as mentioned in the comments.

Clarity – The paper is very well written with clear statements, a pleasure to read.

Originality – Fairly original, but it still needs some work to justify it better.

Significance – Constraining the dimension reduction to fit a certain model is a relevant topic, but I'm not convinced of how well the Gaussian model fits the low-dimensional representation and how well can a neural network compute the GMM mixture memberships. 

review_title: Clear paper on joint optimization of dimension reduction and density estimation, more work needed on the justification and the experimental setup.

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Summary

This applications paper proposes using a deep neural architecture to do unsupervised anomaly detection by learning the parameters of a GMM end-to-end with reconstruction in a low-dimensional latent space. The algorithm employs a tailored loss function that involves reconstruction error on the latent space, penalties on degenerate parameters of the GMM, and an energy term to model the probability of observing the input samples.

The algorithm replaces the membership probabilities found in the E-step of EM for a GMM with the outputs of a subnetwork in the end-to-end architecture. The GMM parameters are updated with these estimated responsibilities as usual in the M-step during training.

The paper demonstrates improvements in a number of public datasets. Careful reporting of the tuning and hyperparameter choices renders these experiments repeatable, and hence a suitable improvement in the field. Well-designed ablation studies demonstrate the importance of the architectural choices made, which are generally well-motivated in intuitions about the nature of anomaly detection.

Criticisms

Based on the performance of GMM-EN, the reconstruction error features are crucial to the success of this method. Little to no detail about these features is included. Intuitively, the estimation network is given the latent code conditioned and some (probably highly redundant) information about the residual structure remaining to be modeled.

Since this is so important to the results, more analysis would be helpful. Why did the choices that were made in the paper yield this success? How do you recommend other researchers or practitioners selected from the large possible space of reconstruction features to get the best results?

Quality

This paper does not set out to produce a novel network architecture. Perhaps the biggest innovation is the use of reconstruction error features as input to a subnetwork that predicts the E-step output in EM for a GMM. This is interesting and novel enough in my opinion to warrant publication at ICLR, along with the strong performance and careful reporting of experimental design.


review_title: Strong anomaly detection by end-to-end architecture that predicts GMM parameters with tailored loss function

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: 1. This is a good paper, makes an interesting algorithmic contribution in the sense of joint clustering-dimension reduction for unsupervised anomaly detection
2. It demonstrates clear performance improvement via comprehensive comparison with state-of-the-art methods
3. Is the number of Gaussian Mixtures 'K' a hyper-parameter in the training process? can it be a trainable parameter?
4. Also, it will be interesting to get some insights or anecdotal evidence on how the joint learning helps beyond the decoupled learning framework, such as what kind of data points (normal and anomalous) are moving apart due to the joint learning  
review_title: The paper presents a joint deep learning framework for dimension reduction-clustering, leads to competitive anomaly detection

Meta-review:  + Empirically convincing and clearly explained application: a novel deep learning architecture and approach is shown to significantly outperform state-of-the-art in unsupervised anomaly detection.
 - No clear theoretical foundation and justification is provided for the approach
 - Connexion and differentiation from prior work on simulataneous learning representation and fitting a Gaussian mixture to it would deserve a much more thorough discussion / treatment.

Decision: Accept (Poster)