Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: 1. The authors proposed to add noisy data into the training set of unsupervised anomaly detection and created a new task setting.
2. The authors introduced a denoising mechanism and investigated 3 sampling strategies for distinguishing between noisy and clean (normal) data.
3. The experimental results are superior in the new setting.

strengths_and_weaknesses: Strengths：
1.	It is interesting to create a new task that adds noisy data to normal training images for simulating real scenes.
2.	The authors made their motivation and statement clear.
3.	The authors conducted many experiments to verify their proposed methods.
Weaknesses:
1.	The novelty of the proposed method is weak. It seems to be a modified version of previous work: PatchCore, by adding a denoising process to the memory bank. 
2.	The setting of ‘Overlap’ experiment is somewhat confusing and meaningless. It is unfair for methods to train and test on the same (overlap) data. 
3.	In the experiments on ‘No overlap’ setting, little performance drops were seen on state-of-the-art methods, and the comparison between the proposed methods (98.6) and PatchCore (98.4) can not show the benefits of the new approach.

questions: 1.	Explanations of the weakness mentioned are important.
2.	It is confusing in Fig.2. The explanation of notions: ’N, H, W, C’ is missing. The arrow under ‘outlier factors of all selected patches’ are misleading, which is started from all samples including noisy ones.
3.	The authors mentioned on Page 6 Line 197: ‘The soft weights, i.e., local outlier factors, have considered the local relationship around the nearest node.’ An analysis of the neighbor's influence is needed.
4.	The English statement needed to be modified. On Page3 Line 94. ‘too long a training stage limits its usage.’ is confusing.

limitations: None
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The authors propose a robust unsupervised anomaly detection model, SoftCore, that explicitly detects outliers in the training data and selects an anomaly-free coreset as a memory bank. 
This memory bank is used as a reference of normality to detect novel test anomalies. The training noise discriminator that detects training contamination is based on the well-known density-based method: LOF [1]. 
Experiments that are conducted on the MVTec dataset show that SoftCore outperforms some baselines.


References: 
[1] Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. Lof: identifying density-based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pages 93–104, 2000.

[2] Gong, Dong, Lingqiao Liu, Vuong Le, Budhaditya Saha, Moussa Reda Mansour, Svetha Venkatesh and Anton van den Hengel. “Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection.” 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2019): 1705-1714.

[3] Liu, Tongliang and Dacheng Tao. “Classification with Noisy Labels by Importance Reweighting.” IEEE Transactions on Pattern Analysis and Machine Intelligence 38 (2016): 447-461.

[4] Alghushairy, Omar, Raed Alsini, Terence Soule and Xiaogang Ma. “A Review of Local Outlier Factor Algorithms for Outlier Detection in Big Data Streams.” Big Data Cogn. Comput. 5 (2021): 1.

[5] Pang, Guansong, Cheng Yan, Chunhua Shen, Anton van den Hengel and Xiao Bai. “Self-Trained Deep Ordinal Regression for End-to-End Video Anomaly Detection.” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020): 12170-12179.

[6] Liu, Boyang, Ding Wang, Kaixiang Lin, Pang-Ning Tan and Jiayu Zhou. “RCA: A Deep Collaborative Autoencoder Approach for Anomaly Detection.” IJCAI : proceedings of the conference 2021 (2021): 1505-1511 .

[7] Zhou, Chong and Randy Clinton Paffenroth. “Anomaly Detection with Robust Deep Autoencoders.” Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2017): n. pag.
strengths_and_weaknesses: The paper is not particularly well written, and it contains a relatively large number of grammatical mistakes. The main points, however, are comprehensible.
The ideas presented here are not particularly original. The idea of reweighting the training data according to their anomalousness degree has been done before (e.g. some recent references [2,3]). Using LOF for the anomaly detection of noisy data is by no means new (e.g. [4]).

Strengths
	• The comparison to other methods shows the advantage of the method when it comes to noisy training data. However, it would be beneficial to see the results on more challenging datasets.

Weaknesses:
	• The overall idea is not novel, while the specific of the methods suggested here are. The idea of reweighting the training data according to their anomalousness degree has been done before (e.g.  recent references [2,3,4]). 
	• To construct the memory bank (i.e., the reference of the normality), the authors propose to reject “the top τ% patches with the highest outlier scores”. Do they implicitly assume that the ratio of the noise in the training data is known in advance? This assumption is strong in a real-world environment where the ratio of the contamination may vary according to the collected training data. 
	• Experiments on other challenging state-of-the-art datasets are desirable.
	• Limitations are not thoroughly discussed (e.g., the complexity).

Clarity:
	• The paper is not particularly well written. But, the main points are comprehensible.
	• The notations of the formulas are sometimes confusing, which makes it difficult to follow. For example, in line 193, “w” denotes the anomaly weight. However, w denotes the batch position at line 140. 

Relation to prior work:
The authors state that “we are the first one to study the abnormal detection with noisy data”. However, many studies have been proposed in this research field: e.g., [5,6,7]

Reproducibility:
	• The conducted experiments are well described. 
	• It would be helpful if the entire code would be made accessible in case the paper is accepted.
questions: -
limitations: The limitations of this method are not discussed
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper sets forth a memory-based unsupervised AD method for anomaly detection. The novelty lies in efficiently denoising the data at the patch level. To this end, noise discriminators are utilized to generate outlier scores for patch-level noise elimination before corset construction. Compared to the existing algorithms, the proposed algorithm is relatively well performed.
strengths_and_weaknesses: Strengths:
+ A patch-level denoising strategy for coreset memory bank sounds reasonable and intersting
+ Setup a practical setting for unsupervised anomaly detection
+ Experimental results adequate

Weakness:
- No test on real noisy dataset is provided
- No computational analysis is provided
questions: I'd like to authors to respond to the two points in the weakness. 
limitations: not mentioned.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper addresses a practical setting of Anomaly Detection, namely AD with noisy data. The authors propose a novel AD algorithm named SoftCore to enhance model performance and noise robustness. By using a patch-level selection strategy, the model can reweight the coreset samples and wipe off the noisy image patch of noisy samples.
strengths_and_weaknesses: Strengths:
+ The paper is well-structured, and easy to follow.
+ This paper explores the impact of noise data on anomaly detection tasks, which has highly practical significance.
+ The proposed task setup is reasonable, and the authors have conducted fundamental experiments to explain insights and analyze the impact of noise on AD tasks.
+ From the experimental results, the effectiveness of the proposed method is verified. 

Weaknesses:
- There are some grammatical errors: Line 156, $W_(i)$ should be $W_{(i)}$; Line 268, LOF achieve -> achieved.
- The mathematical notation should be improved. For example, the symbol of sample variance $\sum_{h,w}$ is confusing in Eq(2) and (3).
- The critical threshold τ should be given more explanations.

questions: Please refer to the weakness
limitations: Yes.
review_title: null

Meta-review: metareview: Can the authors please remove the word "SoftCore" from the title and manuscript as this word has negative connotations.
confidence: Less certain
Decision: Accept