Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: TAUFE, a novel regularizer has been presented to handle OOD. It rigorously validates its performance on three tasks, classification, regression, and a mix of them, on CIFAR-10, CIFAR-100, ImageNet, CUB200, and CAR datasets with promising results.
main_review: Novelty:
The paper overall is a novel take on OOD problem. TAUFE, a novel regularizer claims to deactivate undesirable features using OOD examples in the feature extraction layer and thus removes the dependency on the task-specific softmax layer. 

Strengths:
1. Code is given zipped as well as URL for verification.
2. Apart from classification tasks, where OOD has started showing some grip, this work also takes care of regression and a mix of them.
3. The experiments and validation on standard datasets creates a good impression wrt results.
4. Mathematical formulation and comparison with softmax layer is sound.

Weakness:
1. there was ample scope to add finer details of implementation and ablation studies in the supplementary section.
2. Ablation on self to understand the pitfalls where method is failing and identifying the corner and failure cases will have made the paper better as well as laid path for future work in need to fill gaps.
3. "deactivates all undesirable features " - is this term not too bold to say?

Relevance to NeurIPS:
Very relevant, aligned to hot area of research "out-of-distribution" handling.

Clarity:
The paper is well written.

Related work Comparison:
There is substantial comparison with other works and standard datasets.

Effort: In terms of thought process, experiments effort is there. However some effort on analysis and corner case / failure identification would have made this a very good effort.

Suggestions:
1. Probably, varying the types of data of the datasets (datatype) might help in getting deeper in the problem as a generalization tool.]
2. Some explanation needed what is meant by undesirable features and desirable features? Can a bias on desirable features help in drawing the boundary line? Is there a possibility of a membership function based boundary line?
3. How will these work in adverserial examples?

limitations_and_societal_impact: Not applicable here.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes a simple regularization that encourages logits to be close to zero on a designated out-distribution during training. The authors show that this improves accuracy across various tasks and datasets.
main_review: Strengths:
The proposed method is quite simple and the results in the paper indicate that it can be very effective at boosting accuracy. The method is also task-agnostic which means it can be applied beyond just, e.g. image classification. Also, the authors have used several different training out-distributions which gives more information about the dependence on the choice of this distribution than many other papers in the field. The inclusion of few-shot results is also interesting.


Weaknesses:
A major issue with this paper is the weakness of their baseline. Even when only using a ResNet18 on CIFAR10 a plain trained classifier can attain 95.0% using AutoAugment. Similarly, an outlier exposure (OE being, as far as I can tell, identical to the OAT that they compare to) trained model can achieve 95.5% when using the same 80mTiny dataset as out-distribution as the authors. On CIFAR100 (77.4% and 77.3%, respectively). In other words, their improvement beyond their baseline is much smaller than their gap to a simple model trained using AutoAugment. Would their results continue to hold when using stronger baselines?

Another issue is that they only compare the classification performance in terms of accuracy. Outlier-aware methods are typically designed to improve out-of-distribution detection. Does their method perform better or worse than OE/OAT on this task? Even if it is not the main focus of the paper, these results need to at least be discussed and included in the appendix.

Since the focus of the paper is so heavily on improving the performance on the original task, I also think that they need to include baselines from the semi-supervised literature (for example, [1], but hopefully also more recent works).


Clarity:
The clarity of the paper needs much improvement. Especially experimental details are not fully laid out, e.g. line 236 "best values are obtained via a grid search." What was the grid? How were the validation sets chosen? What was the performance metric that they optimized for?

I also find Table 1 a little unclear. The caption states "Average cosine similarity between each classe’s activation on CIFAR-10 for Standard, OAT,and TAUFE". I understand this to mean that the cosine similarity between the features of samples within the same class was computed and that those results were averaged across classes. If I understand this correctly, then I do not understand how this is a bad thing. Higher similarity within classes does not seem like an issue. The accompanying discussion in the main text seems to indicate that the authors are instead talking about the cosine similarities between the different classes. How is this computed exactly? The mean of each class is computed and those are compared? Or the cosine similarities with samples from other classes are computed and then averaged? 

The authors also don't explain what ImageNet-10 is. They write in line 222 that it consists of "randomly selected classes". Which classes? This needs to be in the appendix.


Reproducibility:
I very much appreciate that the authors provided their code in the submission.


Additional feedback:
The definitions of desirable and undesirable features should probably be modified slightly to be called rho-desirable or rho-undesirable. This is especially important in Eq. 2 where F_out is defined in terms of a rho, that is not explicitly given. The authors clearly meant to assume that rho is the same for both definitions but the way it is written now is sloppy. Also, the discussion below it should maybe point out, that "a feature vector [...] could be a mixture of desirable and undesirable features" but that it could very well contain neither.

Another point that I would appreciate some clarification on is in line 240 where the authors state that they only sample N points from the out-distribution - even in the full-shot regime. This appears somewhat non-standard so I would have liked some justification for this choice in the experimental setup.

[1] There are many consistent explanations of unlabeled data: Why you should average; Ben Athiwaratkun, Marc Finzi, Pavel Izmailov, Andrew Gordon Wilson; at ICLR'19
limitations_and_societal_impact: I am not quite sure why the authors chose to move their limitations section into the appendix. They clearly still had plenty of space on the final page and the page limit was extended to 9 pages in part to allow for this sort of discussion in the main paper. I recommend moving this back into the main paper.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper describes a simple yet effective approach to deactivate the undesirable features using out-of-distribution data. By shrinking the undesirable features from feature extraction layers instead of task-specific layers using L2 penalty, this method is task agnostic. Authors demonstrated and compared performance with task-specific approaches on classification, regression and mixed tasks using several benchmark data sets. 
main_review: The key idea of this paper is to leverage L2 penalty to shrink the undesirable features extracted from OOD examples towards zero by assuming OOD and ID examples share such undesirable features. Compared with the recently published task-specific OAT approach, this method is task-agnostic yet simple to implement. Since L2 penalty shrinks the correlated features as groups, can authors provide some analysis to show the TAUFE approach de-activates groups of undesirable features but not much on the groups of desirable features? Authors provide Figure 2 illustration with some descriptions from lines 180-186. It would be great if authors can give in-depth analysis to show L2 penalty indeed shrinks undesirable feature groups without deactivating too much on desirable feature groups.

Related comment is on empirical analysis of the claimed properties using experiments. Authors successfully show TAUFE renders the activation of ID classes more distinguishable. However, it is necessary to empirically show that the undesirable feature groups have indeed been deactivated whereas not as much for the desirable feature groups. For example, authors might visualize the examples in desirable and undesirable feature spaces, respectively. Even if the method is demonstrated as task-agnostic analytically and empirically as requested, it can be OOD-dependent, i.e., the undesirable feature deactivation would depend on the choice of OOD examples. As such, the generalizability can be limited to the selected OOD examples from training only. 

The reviewer is willing to raise the rating if authors successfully address the major concerns as mentioned above.
 
Minor comments:
Can authors also report experimental results for standard method using both ID and OOD examples?  
TAUFE does not seem to outperform OAT across all settings. It is also not clear whether OAT has been appropriately optimized for its performance. 

------------- New comments after reading authors' rebuttal
Authors have successfully addressed my concerns by providing an in-depth analysis to show the TAUFE approach deactivates the groups of undesirable features but not much on the groups of desirable features. As such, I raise my review rating to be positive. 

limitations_and_societal_impact: N/A
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: Problem: DNN predictions may get influenced by the undesirable features that are irrelevant to the task at hand, which causes poor  generalization. 
Solution: Existing work showed calibration on out-of-distribution (OOD) samples can remove the contribution of undesirable features in classification. The caveat is its applicability to feature extraction. The paper proposes TAUFE, a novel regularizer that deactivates all the undesirable features in OOD examples from the feature-extraction layer, thereby removing the dependency on the task specific softmax layer.
Results: TAUFE outperforms state-of-the-art methods on the standard benchmarks for classification, regression and the mix of the two.

main_review: Distinction from state-of-the-art:
The proposed approach clearly identifies the gap between the existing techniques to be applied for task agnostic feature extraction and the lack of flexibility and feature entanglement. 

Pros:
The paper is well written, easy to understand and digest the concepts.

The very idea of task agnostic training of a deep model for feature extraction and then doing the tasks specific predictions is interesting.

The paper accurately highlights how deep models use easy features in images, video, object detection, etc, more often they can be irrelevant, in which case the proposed method can be more useful for accurate predictions.

Cons:

Since data augmentation seems to produce unbiased examples (as explained in [17]), how does TAUFE fare against those methods? Maybe why not compare against MixUp like [8] and show the effectiveness, at least for classification if not for regression?

Will TAUFE withstand the adversarial attacks? Maybe this is too much in a single paper, but it is worth throwing some light into the performance of TAUFE for adversarial attacks.

limitations_and_societal_impact: Limitations are unknown performance for adversarial attacks. 
In fact the authors should do a better job explaining the limitations and the societal impact.
review_title: null

Meta-review: This paper proposes a simple regularisation that encourages features to be close to zero on a pre-specified OOD training set.

Reviewers think the proposed approach is novel, simple, and effectively demonstrated on a variety of experiments. An ethical issue has been raised on the 80mTiny dataset used in experiments, for which the authors promise to remove it. So it comes down to the question whether the remaining result is significant enough to demonstrate the claims, for which the reviewers do not see it as an issue.

Because of this issue with the 80mTiny dataset, the paper is being marked as conditionally accept.

Reviewers raise another concern on missing comparison to semi-supervised learning approaches, and the authors agree to add in experiments in revision. Label/logit smoothing and feature denoising have also been extensively discussed in adversarial robustness literature, it would improve the paper if the authors can evaluate the proposed approach on adversarial examples.

Also I would suggest to openly discuss the limitation of the proposed approach in the main text rather than in appendix.

----

UPDATE: Upon reviewing the revision, the decision has been updated to accept.
Decision: Accept (Poster)