Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper presents an anomaly detection technique based on EBM (energy based model). The proposed technique is supposed to help adapt to new tasks faster with just a few rounds of training.
strengths_and_weaknesses: The proposed algorithm is a combination of several existing techniques. A fundamental contribution is hard to identify.


The paper is very hard to read and also has numerous typos such as:
Line 11: associates => associate
Line 47: learn => learns
Line 146: pixe => pixel
Line 178: SigShink => SigShrink
Line 245: auto-encode => auto-encoder
Line 249, 251: adapt => adapting


Figure 1 does not really have labeled parts a, b. It also is not very intuitive or informative in explaining parts of the algorithm, e.g., line 135 refers to Fig (a) in relation to the dictionary D. But no additional clarity about D is offered by reference to this figure.


The construction of task-specific dictionary D has not been explained very well. In general it would help to present the content in Sections 3.1 and 3.2 in a more intuitive manner.


It certainly helps that the paper does an ablation study (Sec 4.3) and shows the benefits of the various parts of the algorithm.

=============
Update after author rebuttal:

Thanks to the authors for addressing my concerns. I am revising my scores.
--------------------
questions: Could the authors elaborate more clearly on how the task-specific dictionary is created?
limitations: N/A
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a framework for building an anomaly detector which can be adapted to new tasks at inference time with a few examples from the new task. This is in-contrast to standard anomaly detection methods which had to be rebuild for each new task. For example, a standard autoencoder can be trained for normal tasks(like video surveillance)  and anything with a large reconstruction error can be labeled as anomalous. Now if the scene or camera changes then the autoencoder will have to be re-trained. This paper proposes a three tier framework where the trained anomaly detector can be adapted at inference time.
strengths_and_weaknesses: Strengths:

(i) Paper clearly defines the three tasks G1, G2 and G3 which makes it easy to comprehend.

(ii) The idea of using intermediate examples during the LD process as anomalies and using them to train an EBM is novel. Similarly the idea of borrowing ideas from sparse autoencoders and "re-training" it with a few examples makes the detector adaptive.

(iii) The results are quite impressive. For example, in Table 1, the proposed method is even competitive with "upper-bound" where the autoencoders (or other methods) are trained with many samples from each category.

Weakness:

(i) My slight concern is that the whole framework might be too complicated to be actually deployed.
questions: How well does the method do with so called "co-variate shift". The video surveillance example might be an example of co-variate shift ?
limitations: No potential negative societal impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work presents a new algorithm for image-based anomaly detection. The algorithm relies upon the contrastive-divergence method for training energy based models and it presents a differentiable "sparse-coding layer" which seems to be novel. Beyond the algorithmic innovations the paper also presents practical techniques for improving the speed of generating samples from an EBM for images. This work also presents experiments for few-shot learning their proposed algorithm across new domains. 

The basic idea is to use sparse-coding as a way of approximating whether a novel sample is similar to in-distribution samples or not. The innovation lies in implementing sparse-coding in a differentiable manner that lets them train the entire architecture in an end-to-end manner.
strengths_and_weaknesses: This paper presents a new algorithm for image anomaly detection with good empirical performance and some innovative ideas. I believe the results/techniques could be quite useful in other contexts where researchers want to perform nearest-neighbor decomposition in a differentiable manner. 

However the techniques in the paper sometimes seem to be put together a little haphazardly and the paper does not do a good job of clearly explaining the contribution of many of the important parts. For example, a lot of space is devoted in the paper on episodic training as a way of enabling few-shot learning. However it isn't clear from the experiments that how important is the episodic meta-learning for few-shot learning. The biigest help for few-shot learning comes from the sparse-coding layer which doesn't have any learnable parameters.  For example if instead of breaking up the training into $I$ separate episodes where each episode was one task, if we simply trained in one episode on all the data what will be the performance loss ?

Also the paper sometimes makes claims that are a hard to understand/verify. For example, the paper claims that proposed method is the "first to allow [adaptation] to new tasks with a single forward pass without any training". However even a simple process such as finding the distance to the 1-nearest neighbor allows some level of adaptation in a single forward pass.

--- After the author response -- 

the authors replied back with more evidence about the importance of meta-learning which seems sufficient, updating my rating now

questions: Please see the questions mentioned in the "Strengths and Weaknesses" section.
limitations: Yes the authors adequately discusses the limitations and potential negative societal impact of their work.
review_title: null

Meta-review: metareview: The final consensus from three reviewers knowledgeable in the field was that the paper makes an interesting contribution in the area of anomaly detection. The empirical results were seen as particularly impressive, and the treatment of intermediate samples from Langevin dynamics as abnormal was also seen as offering some novelty. My own assessment is a bit more qualified than the reviewers: while the empirical results are certainly nice, the approach itself seems a somewhat complex combination of ideas from the literature. (This complexity was also noted in one of the reviews.) While we uphold the reviewers' verdict, we encourage the authors to spend a bit more time (perhaps in Sec 2) drawing out some higher-level insights of the proposed approach, and whether there might be simpler alternatives that could also work well.
confidence: Certain
Decision: Accept