Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper considers outlier detection in a few-shot setting with a small samples size in the target domain. 

They propose to combine self-supervised learning with two loss terms, one loss term to encourage larger similarities between in domain samples, and a second loss term to encourage smaller similarities between in domain samples and simulated outliers.

They use a mahalanobis distance using normalized features to score for outlierness, without using a mean term to subtract from the features.

They go ahead to demonstrate results on four industrial defect datasets and provide an ablation study on the terms used.

strength_and_weaknesses: Strengths:
*Easy to read
*evaluation on industry datasets as opposed to simplistic standard image benchmarks
*Mostly an improvement over Cutpaste, showing as insight limited usefulness of negative examples as created by CutPaste for certain real datasets.
*Ablation study

weaknesses: 
*straightforward extension with limited novelty
*limited usefulness of the second proposed loss term
*improvement over cutpaste on three datasets due to omission of the NP term, effectively demonstrating the gain due to the term encouraging clustering between positive samples.

clarity,_quality,_novelty_and_reproducibility: The paper is well readable. The theoretical novelty is limited, effectively showing the usefulness of the positive pair loss term. On the experimental side the use of datasets is a plus as well as the ablation study.
summary_of_the_review: Effectively the paper shows the usefulness of self-supervised finetuning with an added positive pair loss.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper is focused on the few-shot anomaly detection problem. It presents a method that combines contrastive learning for adapting a pre-training model to the target domain, a cross-instance positive pair loss, an option to incorporate negative pair loss, and density-based anomaly detection. They show some promising experimental results for few-shot anomaly detection on several public datasets.
strength_and_weaknesses: Strengths: 
1. The proposed idea of applying contrastive learning for adaptation seems interesting. 
2. The experimental results on 4 different datasets are reported with promising results.

Weaknesses:
1. The paper is not well written. It is difficult to understand the main idea of the paper. How is the adaptation of a pre-trained model done with contrastive learning? Why doers the proposed training method provide good model adaptation? In addition, many important delaits for the implementation are missing. They didn't provide the network architecture of the proposed model.

2. The novelty proposed in this paper is limited. All the componenets used in this paper are not new. The idea of extending the contrastive training to model adaptation is not clear. 

3. The notations used in describing the proposed method are not always defined or explained in the paper. For example, what are online encoder and target encoder in Figure 1 and eq. 2?

4. The experimental comparisons used in this paper should include comparison with more recent SOTA methods, such as 
CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization
via Conditional Normalizing Flows, WACV 2022
Registration based Few-Shot Anomaly Detection, ECCV 2022

5. The experiment protocols should follow the previous experiments published in the previous papers, such as those in Registration based Few-Shot Anomaly Detection, ECCV 2022, for a fair comparison.

6. For the denisity-based anomaly score for anomaly detection given in eq. (5), the covariance matrix could be signular if there is not enough samples in the convariance matrix estimation in eq. (4). There is no discussion on how to solve the problem.


clarity,_quality,_novelty_and_reproducibility: Clarity: 
As described in the previous section, the paper is not very clear in the presentation of the proposed method, especially some idea, notations, the network architecture.

Quality:
The paper still has some room for imprtovement interms of novelty, presentation and experimental justification.

Novelty:
Limited for the reasons given in the previous section.

Reproducibility:
Diifculty very reproduce since many details are missing in the paper.
summary_of_the_review: The paper still has room for improvement in terms of paper presentation, technical novelty, and experimental justification. Please see the comments detailed in the previous sections.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes contrastive training and transfer learning applied to the problem of industrial defect identification via "anomaly detection". In particular, the negative pair loss is introduced as novelty in the paper.
strength_and_weaknesses: Strenghts:
- The approach shows good empirical results in 4 datasets
- The ideas for tweaking the learning using across distance positive pair and the negative pair loss may be useful in some scenarios beyond anomaly detection applied to industrial defect 
- Many experiments are provided to investigate the behaviour of the method and the proposed ideas
- Using CutPaste as a way to synthesize negative instances is a good insight.

Weaknesses
- All new ideas are somewhat already proposed before. Although the authors put together all those with success for the application at hand, that makes the paper less novel
- The backbone may be deterrent to the final results. Maybe using a stronger backbone such as ViT-Dino could improve results of all baselines and diminish the amount of improvement claimed by the method.
- In terms of the application the paper does not provide a lot of insight. That is, why the proposed approaches benefit the application? Why does the defects are easier to spot after the representations are learnt with constrastive learning? Why this application is a good downstream task?
- The batchsize may play a significant role in the learning process, but this is not mentioned or studied in the paper.


clarity,_quality,_novelty_and_reproducibility: The paper is clear and well written overall. I don't see any huge problem with reproducibility

In terms of the problem, the datasets have a large number of defects in testing: usually in inference time anomaly detection tasks have much less anomalies than "normal" data. It would be interesting to see how the model behaves under a more "realistic", unbalanced, distribution.

All new ideas are somewhat already proposed before. Although the authors put together all those with success for the application at hand, that makes the paper less novel,
summary_of_the_review: The paper is well written and has some clever ideas, but there is no significant insight on the application, no reasons or insights are provided on why the methods work in the anomaly detection setting. Also, there is not much novelty on the main proposed methods.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper focuses on few-shot anomaly detection where only a few normal samples are available in training. The new method leverages contrastive learning to transfer pre-trained from the source domain to target domains supported by few-shot samples. In addition, the instance positive pair loss is used to tight up normal samples, while incorporating negative pair loss to separate the anomaly and normal samples (with prior and generated samples). Last, the multivariate Gaussian and density method are used for final detection.
strength_and_weaknesses: Strength:
* One of the major strengths of this paper is the extensive experiments on six benchmarks and convincing results.
* Another strength is the idea of adapting pre-trained networks to target domains through contrastive learning under few-shot AD.

Weakness:
* It would be helpful to provide visualization after the outputs fit the multivariate Gaussian. This may further demonstrate if positive/negative pairs are grouped/separated as expected.
* Some experiment details are missing, e.g., how many normal images were used in Sec. 4.5. 
3* Some competitive methods such as TDG (as mentioned in the paper) could be added to the experiments on cifar10-c and cifar100-c.
* The state-of-the-art CSI [1] can run under few-shot setting and could be evaluated here, too. Note most key steps in this paper are very similar to CSI, so it is strongly recommended to compare with CSI in few-shot tests.
* It should be noted that in Fig. 1 after negative augmentation, the image however did not change to an anomaly, as expected.

[1] Jihoon, et al; CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances. 
clarity,_quality,_novelty_and_reproducibility: The paper is clear in general but could be better illustrated with visualization of multivariate Gaussian for additional interpretation and insights.

The novelty of this paper is limited in the sense that most key steps are very similar to the SOTA method CSI, although using domain adaptation for few-shot setting is somehow interesting and useful in most applications.

The implementations are well-elaborated and can be reproduced.

summary_of_the_review: In brief, this paper presents an interesting few-shot AD method, which includes two steps: domain adaptation for pre-trained networks, and contrastive learning via building positive- and negative sample pairs. Evaluations are extensive and promising. However, the major concern is the novelty and its difference and comparison with the SOTA method such as CSI.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The submission looks into the problem of few-shot anomaly detection. The approach it proposes first trains the model on a large source dataset and then uses a combination of contrastive learning (BYOL), cross-instance loss on pairs of positive examples, and a loss on pairs of positive and synthetic negative examples to adapt the model to the target domain. Anomaly detection is performed by fitting a multivariate Guassian distribution on the few positive examples, and the anomaly score is obtained through the Mahalanobis distance.

The proposed approach is evaluated on four industrial defect identification datasets (MVTec, AITEX, Magnetic Tile Defects, and SemiCon) and two datasets with synthetic common corruptions (CIFAR10-C, CIFAR100-C). It is compared against several baselines and is claimed to achieve state-of-the-art performance.

Reviewers generally feel positive about the submission's clarity; some reviewers raised questions on experimental details which were answered by the authors in the rebuttal. Most reviewers note the proposed approach's promising performance (bC75, gtEo, n46i). Some reviewers pointed out missing recent baselines (bC75 on CFLOW-AD, n46i on TDG and CSI), and the authors addressed this in the latest updated submission. Reviewers are split on acceptance: Reviewer LkYy is leaning towards acceptance, Reviewer n46i went from leaning towards rejection to leaning towards acceptance after the rebuttal, and Reviewer gtEo is leaning towards rejection.

The main outstanding concern shared by all reviewers is technical novelty. Reviewer LkYy points out that the proposed approach is a straightforward application of contrastive learning, but they also note that the observation on the uneven effectiveness of the negative pair term in CutPaste is of interest to the community. Reviewer gtEo is concerned that the submission is an application of existing ideas to a new problem. Reviewer n46i notes that the proposed approach shares design elements with CSI. In their response, the authors highlight the fact that contrastive learning has not been applied to few-shot anomaly detection before and clarify the similarities and differences of their proposed approach with CSI. Following the authors' response, Reviewers LkYy and gtEo remain concerned that the proposed approach is too straightforward of an application of existing ideas and too thin on technical novelty.
justification_for_why_not_higher_score: There is a concern remaining that the proposed approach is too straightforward of an application of existing ideas and too thin on technical novelty.
Decision: Reject