From: ICLR.cc/2023/Conference/Paper4203/Authors
title: Submission Withdrawn by the Authors

From: ICLR.cc/2023/Conference/Paper4203/Authors
title: Response to Reviewer 3zrL 
comment: Thank you for the constructive comments. We agree with your assessment and modify based on your feedback. We review the English language errors and modify the abstract to provide a clearer explanation. If something is still missing, please comment it and we will revise and supplement it.

**Additional experiments for larger contamination ratios**

We appreciate your perspective. It improves quality and yields better experimental results. On the MNIST dataset, RVAE-ABFA still outperforms heterogeneous loss. However, additional experiments show that when the contamination ratio is greater than 20%, heterogeneous loss performs the best.

**Effect of q depending on the contamination ratio**

The comment was also very helpful. The results according to the contamination ratio are updated in Table 3. It can be confirmed that high robustness can be achieved even when using soft rejection and heterogeneous loss alone, and the best performance can be obtained when using both. In addition, to evaluate the influence of q and $t_s$ alone, it would be good to refer to Appendix A and B.

We appreciate your feedback. We hope our response is satisfactory to you, and if you have any more questions, we will be pleased to answer them. We hope the answers contribute to updating your score.

From: ICLR.cc/2023/Conference/Paper4203/Authors
title: Respone to Reviewer kK5a
comment: Thank you for the constructive comments. We agree with your assessment and modify based on your feedback. 

**Datasets in different domains**

We agree with you that more domain datasets are required to establish generic applicability. We conducted additional experiments on four tabular datasets (Arrhythmia, Thyroid, KDDCUP, and KDDCUP-Rev) for general applicability, and the results are shown in Appendix F. The effect of heterogeneous loss is less significant in the case of a 10% contaminated Arrhythmia and Thyroid dataset. It does, however, produce comparable results on a 20% contaminated dataset and outperforms a 30% contamination ratio. It produces the finest results for KDDCUP and KDDCUP-Rev.

**Problem of Figure 4**

To make it easier to evaluate the experiments, we provide two results: Figure 4 to analyze the trends of models and Table 2 to show the details of the important experiment results.

**Artificial setup**

One vs rest setup is used in a lot of research to evaluate the models in the field of anomaly detection. However, the setup of training, validation, and testing data look arbitrary.  Therefore, there are experiments in Appendix E that use the whole data (do not control the anomaly/normal proportions in the test dataset, 90% of data in training are used for training, and the rest for validation). Heterogeneous loss still shows significant improvements on contaminated datasets.

**Comment for Algorithm format**

Thank you for your feedback. The Algorithm was promptly updated.

We appreciate your feedback. We hope our response is satisfactory to you, and if you have any more questions, we will be pleased to answer them. We hope the answers contribute to updating your score.

From: ICLR.cc/2023/Conference/Paper4203/Authors
title: Response to Reviewer urGg
comment: Thank you for the constructive comments. We agree with your assessment and modify based on your feedback. 

**Explanation of idea, motivation, and experiments**

We added a section (Section 3) to describe the motivation behind our idea. In practice, the normal and abnormal distributions of anomaly score distributions on contaminated datasets overlap. It illustrates that it cannot handle all anomalies when the contamination ratio assumed is identical to the real one as seen in Figure 1b (even after removing 10% of the data, there are still 6.7% abnormalities in the training data). As a result, we suggest a rejection method that suppresses anomaly training at the expense of normal by increasing the contamination ratio assumption, as well as a heterogeneous loss for minimizing the normal sacrifice.

The third paragraph of Section 5.3 explains why hetero loss provides higher performance. The pseudo-Huber loss has limited robustness since it only produces low gradients for high losses and is dependent on the quantity of loss. LOE trains to diverge approximately 10% of potential anomalies, resulting in the deterioration on clean datasets. It also has constraints when the actual contamination ratio exceeds the expected contamination ratio. Heterogeneous loss can outperform LOE since it can cover more anomalies.


**Novelty**

Our main novelty is aggressive rejection, which assumes a 50% contamination ratio to tackle a high number of potential anomalies, and heterogeneous loss function, which provides dynamic gradients to reduce normal sample rejection. Section 5.5 discusses the impact of each component, aggressive rejection (soft rejection), and heterogeneous loss function. Although GA loss utilizes negative loglikelihood to determine an alpha (samples in a mini batch use the same loss function), heterogeneous loss employs a separate alpha for each potential anomaly to train more normal and less abnormal samples.

**Experiment Results**

Because we employ conventional models, they may underperform when compared to robust models. We believe it can outperform robust models when they are replaced by more powerful models.

For DSVDD on F-MNIST dataset, LOE outperforms heterogeneous loss when the contamination ratio is 10%. Because heterogeneous loss distinguishes between normal and abnormal samples based on learning speed, LOE trains to diverge the normal loss function of anomalies (minimize abnormal loss function which can be a reciprocal of normal loss function). However, LOE should assume a low contamination ratio, thus heterogeneous loss outperforms as the contamination ratio rises.

**Sensitivity of $s_q$**

The feedback is beneficial in improving the paper's quality. We conduct further experiments to study the sensitivity of $ s_q$ depending on a contamination ratio and report our observations in Appendix A. When $s_q$ is set to 0.5, it produces the best results, particularly for DSVDD with a 20% contaminated dataset.

**Details of Experiment Setups**

We describe the details of model architecture and experiment setups in Appendix D.

We appreciate your feedback. We hope our response is satisfactory to you, and if you have any more questions, we will be pleased to answer them. We hope the answers contribute to updating your score.

From: ICLR.cc/2023/Conference/Paper4203/Reviewer_urGg
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper presents a heterogeneous loss function with aggressive rejection strategy for training an anomaly detection (AD) model from  contaminated data. Experimental results showed that the AD model trained with the proposed heterogeneous loss function with aggressive rejection outperformed those trained with existing robust loss functions.
strength_and_weaknesses: Strengths:
1. A novel heterogeneous loss function based on extension of GA loss function is proposed.

2. Promising experimental results are shown to demonstrate the robustness of the AD models trained by using the proposed heterogeneous loss function compared to those with traditional robust loss functions.

Weaknesses:
1. The idea and motivation of the proposed heterogeneous loss function is not well explained. The authors should provide better explanation to explain the main idea behind the design of this new loss function. It is not clear why it can provide better performance than the traditional robust loss functions for the case of training from contaminated data.

2. The novelty is limited since the main novelty is just the proposed heterogeneous loss function, which is based on an extension of the GA loss. 

3. The experimental results only show incremental improvement on AD tasks on 3 public datasets. In some cases, the proposed loss function does not provide the best accuracy.

4. How sensitive is the selection of threshold s_q for aggressive rejection in the proposed training strategy? All the experiments with training from contaminated data in this paper are set with 20% contaminated data. This information could be highly dependent on the selection of s_q. I suggest they should perform experiments with different percentages of data contamination.

clarity,_quality,_novelty_and_reproducibility: Clarity: Some implementation details are not provided in the paper, such as the model architectures for the AD models and the experiment settings.

Novelty is somewhat limited as described above.

The paper quality can be further improved in terms of paper presentation, novelty, and experimental justification.


summary_of_the_review: In summary, I think the paper still has room for improvement as detailed above and the contribution for this paper is not significant enough to be accepted at ICLR.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4203/Reviewer_kK5a
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The paper presents a robust loss function for optimizing mean-square based models for image datasets.
strength_and_weaknesses: 1. The paper is very narrow in scope. Even the title should be made specific to image data.

2. From Figure 4, the results for comparison with robust models are quite mixed. For demonstrating general applicability, more datasets should be (from domains other than image) should be included.

3. Section 4.1 Datasets and setup: The setup is quite artificial (1 vs rest, lots of training and validation data, 30% test data being anomalous, etc.) More realistic setting without too much control over the anomaly/normal proportions should be used.

4. While the data setup is not optimal, the algorithm setup is a strength. Specifically, I like the plots in Figure 3 where the performance of the loss functions can be easily compared in a fair manner.

5. Minor issue: line 2 of Algorithm 1 should be '...Mini-batch x subset of X...' instead of '...Mini-batch x element of X...'
clarity,_quality,_novelty_and_reproducibility: The robust loss function is novel and looks useful in practice. The paper is well-written and the illustrations of the loss function (Figure 2) and the algorithmic format of presentation (Algorithm 1) are quite helpful.
summary_of_the_review: The paper presents an useful loss function and should be strengthened with more real-world datasets which are not artificially modified to such an extent that they loose their natural properties.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4203/Reviewer_3zrL
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: This paper presents a new meth0d for anomaly detection (AD) using neural network models where the training data is contaminated with abnormal examples.
The approach is based on an "aggressive" rejection strategy based on the loss distribution (empirical CDF), where the threshold is set such that most anomalies are in the rejected set but also a large amount of normal examples.
For this rejected set, a heterogeneous robust loss function (GA loss) is proposed where the parameter alpha is determined based on the modified z score of each training batch, which amounts to a sort of "soft" rejection.
The approach is evaluated on several image datasets using different CNN architectures, and it is shown that the approach outperforms other robust AD algorithms.


strength_and_weaknesses: Strengths:
- The approach seems quite generic and can be applied to different models
- Experimental validation

Weaknesses:
- Presentation is lacking in various aspects



clarity,_quality,_novelty_and_reproducibility: The paper is overall clear and the contributions are original. Unfortunately, there a numerous English language mistakes, which degrade the overall quality of the paper and make it less readable. But this could be fixed by carefully proofreading..

- According to Figure 4 the proposed approach outperforms RVAE-ABFA only for a contamination ratio larger than 15-20% and not for MNIST. Also for CIFAR-10 and DSVDD the heterogeneous loss outperforms MSE only for a contamination ratio of 20%. It would have been interesting to evaluate the methods for even larger contamination ratios to see if this is a general trend.
- Table 3 and Figure 5 show the influence of the parameters q and ts on the AD performance to some extent. However, the influence of the parameter q is not shown with respect to different levels of contamination.

summary_of_the_review: Overall, the paper presents an original algorithm for the difficult problem of anomaly detection when training data is contaminated.
The experimental results are more or less convincing. Only the presentation presents some minor issues (language mistakes, e.g. the abstract is not really comprehensible).


correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: Not applicable
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold