Paper:
In one-class-learning tasks, only the normal case can be modeled with data, whereas the variation of all possible anomalies is too large to be described sufficiently by samples. Thus, due to the lack of representative data, the wide-spread discriminative approaches cannot cover such learning tasks, and rather generative models, which attempt to learn the input density of the normal cases, are used. However, generative models suffer from a large input dimensionality (as in images) and are typically inefficient learners. We propose to learn the data distribution more efficiently with a multi-hypotheses autoencoder. Moreover, the model is criticized by a discriminator, which prevents artificial data modes not supported by data, and which enforces diversity across hypotheses. This consistency-based anomaly detection (ConAD) framework allows the reliable identification of outof-distribution samples. For anomaly detection on CIFAR-10, it yields up to 3.9% points improvement over previously reported results. On a real anomaly detection task, the approach reduces the error of the baseline models from 6.8% to 1.5%.

Review:
Summary
-------
The paper proposes a technique to make generative models more robust by making them consistent with the local density. It is hypothesized that robust model will be able to detect out-of-distribution samples better and improve anomaly detection.
Main comments
-------------
1. The proposed technique adds additional regularizers to the GAN loss that, in effect, state that the best hypothesis under a WTA strategy should have a high likelihood under the discriminator 'D'. This is an interesting idea and certainly a reasonable thing to try. As stated in the abstract, the generative models are inefficient; it is likely that additional structure enforced by the regularizer helps in improving the efficiency.
2. The objective in GANs is to infer the underlying distribution correctly and so far it has been found that their accuracy is heavily dependent on both the architecture as well as the computational complexity (they may improve with more training, but maybe not consistently). Therefore, it becomes hard to compare the three architectures in Figure 2 since they are all different. A more rigorous comparison would try to keep as many pieces of the architecture the same as possible so that ConAD can be compared with 'all other things being same'. Some experiments seem to follow this idea such as 'MDN+ConAD-{2, 4, 8, 16}' in Table 2. But in these experiments the addition of ConAD offers a mild improvement and even degrades for the maximum number of hypothesis (i.e., 16).
3. Page 2, para 2, last two lines: "For simplicity, imagine an ... the real distribution."
The argument is not clear. It seems too trivial and almost like a straw man argument.
4. Page 4: "In anomaly detection, this is difficult since there is no anomalous data point contained in the training dataset."
This is not true in real-world applications where most data is contaminated with anomalies. This is part of the challenge in anomaly detection.
The above also applies to the following on page 6: "During model training, only data from the normal data class is used..."
5. Page 5: "...D minimizes Eq. 3": Should be 'maximizes' since the reference is to the log likelihood of real data (or, add a negative sign).
6. Eq. 4: The last component should be negative since we trying to maximize the likelihood of the best hypothesis under WTA (right?).
7. Table 1: The datasets are not real anomaly detection datasets (too high proportion of 'anomalies') Moreover, the number of datasets is insufficient for rigor.
8. Section 5.4: "With our framework ConAD, anomaly detection performance remains competitive or better even with an increasing number of hypotheses available."
Section 6: "... and alleviates performance breakdown when the number of hypotheses is increased."
This is not entirely supported by the results in Tables 2, 3, and also 4 and 5 of supplement. The results for ConAD - {2, 4, 8, 16} are not consistently increasing.
Since experiments are very few (and not real-world for anomaly detection task) because of which the observations cannot be generalized.
9. Page 4 (minor) in two places: "one-to-mapping" -> "one-to-many mapping"
10. Page 5 (minor): "chap. 3" -> "section 3"