Task-Agnostic Undesirable Feature Deactivation Using Out-of-Distribution Data
Authors: Dongmin Park
Hwanjun Song
Minseok Kim
Jae-Gil Lee
Venue: NeurIPS 2021 Poster (2021)
Keywords: Out-of-Distribution
Regularization
Task-Agnostic

Abstract: A deep neural network (DNN) has achieved great success in many machine learning tasks by virtue of its high expressive power. However, its prediction can be easily biased to undesirable features, which are not essential for solving the target task and are even imperceptible to a human, thereby resulting in poor generalization. Leveraging plenty of undesirable features in out-of-distribution (OOD) examples has emerged as a potential solution for de-biasing such features, and a recent study shows that softmax-level calibration of OOD examples can successfully remove the contribution of undesirable features to the last fully-connected layer of a classifier. However, its applicability is confined to the classification task, and its impact on a DNN feature extractor is not properly investigated. In this paper, we propose Taufe, a novel regularizer that deactivates many undesirable features using OOD examples in the feature extraction layer and thus removes the dependency on the task-specific softmax layer. To show the task-agnostic nature of Taufe, we rigorously validate its performance on three tasks, classification, regression, and a mix of them, on CIFAR-10, CIFAR-100, ImageNet, CUB200, and CAR datasets. The results demonstrate that Taufe consistently outperforms the state-of-the-art method as well as the baselines without regularization. 

Task-Agnostic Undesirable Feature Deactivation Using Out-of-Distribution Data
Dongmin Park', Hwanjun Song?, MinSeok Kim', Jae-Gii Leel*
1 KAIST, NAVER Al Lab Republic of Korea
{dongminpark, minseokkin,
ac.kr, hwanjun hwwnjun. songenavercorpp.com
Abstract
A deep neural network (DNN) has achieved great success many machine learning tasks by virtue of its high expressive power. However, its prediction can be easily biased undesirable features, which are not essential for solving the target task and are even imperceptible Leveraging plenty of undesirable features in out-of out-o--distribution (OOD) examples has emerged as potential solution for de-biasing such features, and a recent study shows that softmax-level calibration OOD examples can successfully remove the contribution of undesirable features to the last fully-connecto layer of classifier. However, its applicability is confined to the classification task, and its impact on DNN feature extractor is not properly investigated. In this paper, propose TAUFF, a novel regularizer that deactivates many undesirable features using OOD examples in the feature extraction layer and thus removes the dependency the task-specific softmax layer. To show the task-agnostic nature of TAUFE, we rigorously validate its performance on three tasks, classification, regression, and a mix of them, on CIFAR-10, CIFAR- 100, ImageNet, CUB200, and CAR datasets. The results demonstrate that TAUFE consistently outperforms the state-of-the-art method as well as the baselines without regularization.
human, thereby resulting in poor generalization.
1 Introduction
Undesirable features, which are informally defined as those not relevant to target task, frequently appear in training data; for example, the background is an undesirable feature for classifying animals in images. The undesirable features are mainly caused by the statistical bias in in-distribution training data. In fact, many undesirable features are statistically correlated with labels, even though they are unnecessary and sometimes even harmful for the target task [1]; for example, the "desert" background feature is correlated with "camels" because the camels frequently appear in desert. However, such undesirable features (e.g., desert background) rather yield unreliable predictions because they are easily shifted other unseen data (e.g., images of the camels on the road).
Meanwhile, deep neural networks (DNNs) are known to overly capture any high-frequency data components which are even imperceptible vulnerability of DNNs that can totally overfit to random labels or adversarial examples owing to their extremely high capacity [3, 4, 6]. Accordingly. DNNs are easily biased toward the undesirable features well, thereby often showing unsatisfactory generalization to unseen examples [7]. Thus, it very important prevent overfitting to the undesirable features.
human [2, 3]. This property attributed to the
In this regard, a few research efforts have been devoted to remove the negative influence of undesirable features by leveraging out-of-distrien distribbiitinn (OOD) data [8, 9]. Under the assumption that in-distribution and OOD data share undesirable features, OOD data is treated as a useful resource to alleviate the aforementioned undesirable bias. Notably, recent study [8] proposed sofimax-level calibration,
"Correpooniig author.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

In-distribution Examples
Out-of-distribttion Examples
"Task-agnostic"
"Task-specific"
Softmax Prob.
Xin
Softmax Prob.
Xtn
Classification Task
CNN
FC
CNN
FC
Uniform Prob.
Box IoU
Classification Task
Zero
Regression Task
Xood
Xood
Activations
(a) Softmax-level calibration.
(b) Feature-level calibration (TAUFF).
Figure 1: Comparison of softmax-level and feature-level calibrations.
which assigns uniform softmax probabilities to all possible labels for all examples in OOD data. Although this approach shows decent de-biasing performance in the classification task, the softmax- level calibration has two limitations:
The softmax-level calibration is designated only for the classification task. However, the bias toward undesirable features occurs in numerous tasks, such as object localization and bounding box regression. Therefore, a flexible, task-agnostic approach is required for easily supporting other downstream tasks too.
Lack of Flexibility:
Feature Entanglement: Even desirable features can be entangled with undesirable ones by assigning the uniform softmax probability invariably to all possible labels for OOD examples. Thus, the negative influence of the undesirable features not perfectly removed because they still remain and affect the activation of desirable features (See $ 3.3 for details).
In this paper, we propose novel task task-agnostic and feature- levell calibration method, called TAUFE Undesirable Feature dEactivation), which explicitly forces a model to produce zero values for many undesirable features in OOD examples. Differently from the softmax-level calibration that regularizes the classification layer (Figure ((a)). TAUFE exploits the penultimate layer right before the classification layer and deactivates activation only for OOD examples (Figure (b)). Thus, TAUFE is applicable to any task that requires another task-specific layer other than the classification layer, and the undesirable features are removed in the feature level without feature entanglement. The superiority of the proposed feature-level calibration over the seftmax-level calibration is proven by theoretical and empirical analysis of the feature activation of the penultimate layer.
To validate its general efficacy, we conducted extensive experiments through three tasks: (i) image classification for classification; (ii) bounding box regression for regression; and (iii) weakly supervised object localization (WSOL) for a mix of them. We tested multiple pairs of in-distribution and OOD data: CIFAR-10, CIFAR-100, 100, ImageNet. CUB200, and CAR for in-distribution; and SVHN, LSUN, and P6aces365 for OOD. The experiment results demonstrate that TAUFE consistently outperforms the softmax-level calibrator [8] by up to 9.88% for classification and by up to 8.03% for the mix of classification and regression.
Our main contributions are summarized as follows:
1. We propose a simple yet effective method, TAUFE, to deactivate undesirable features in learning, which is easily applicable any standard learning task with recent DNNs.
2. We provide insight on how feature-level and softmax-level calibration differently affect feature extraction by thcoretic and empirical analysis on the penultimate layer activation.
3. We validate the task-agnostic nature of TAUFE through three tasks and show its performance advantage over the state-of-the- method.
2 Background and Related Work
Negative Impact of Undesirable Features. DNNs tend to overly capture all available signals from training data even when they are not essential for solving a given task [2. 3]. The occurrence of the undesirable features and their negative impact have been recently witnessed in various types of learning tasks. In image classification, classification model often uses background or texture features as an undesirable shorteut for making a prediction instead of using the intrinsic shape of a target class [3, 7J. In object detection, a detector model casily overfits to the background features for

---

localizing target objects in scene [10, 11]. In video action recognition, recognition model often relies on static cues in single frame rather than temporal actions over consecutive frames [12. 13].
In natural language processing (NLP) tasks, language model often makes its predictions based on frequent but meaningless words instead of using semantically meaningful words [14].
Connection with Adversarial Examples. DNNs are casily deceived by adversarial perturbations of the inputs, so-called adversarial examples [15]. Differently from standard learning, the undesirable features are maliciously added and then make the model incur more errors. In addition, it is widely known that such adversarial perturbations are transferable even from different domains [16]; that is, an adversarial attack can drastically degrade the generalization capability of the classifier without knowing its internals [16]. To remedy this problem, the use of OOD examples has gained great attention in that they enhance the robustness against the adversarial examples by preventing the model from overfitting to the undesirable features [8].
Removing Undesirable Feature Contribution. Numerous studies have attempted to prevent the overfitting to the undesirable features in standard supervised learning tasks. A typical way is de- biasing, which removes the undesirable feature contribution based on the pre-defined bias for the target task. Geirhos et al. [7] took advantage of data augmentation techniques to generate de-biased examples from training data. Lee et al. [17] and Shetty et al. [18] synthesized de-biased examples by leveraging a generative model for image stylization or object removal. Wang al. [19] quantified the local feature bias by using the neural gray-level co-occurrence matrix. Bahng et al. [1] proposed a framework that leverages a
model to remove pixel-level local undesirable features. This family of methods successfully removes the pre-defined bias from the undesirable features, but not generalizable to other types of bias. Even worse, is hard to identify the types of undesirable features in advance since they are not comprehensible even to human.
In this regard, motivated by the transferability of undesirable features in different domains, the usefulness of OOD examples for de-biasing was started to be discussed. Although the representative softmax calibrator, OAT [8], does not need a pre-defined bias type, it suffers from the two limitations, lack of flexibility and feature entanglement. Many aspects, such as high generalizability and in-depth theoretical analysis, are yet to be explored.
3 Proposed Method: TAUFE
In this section, we first formulate the problem following the setup in the relevant literature [3, 8, 9] and then describe our method TAUFE. Moreover, we provide theoretic analysis with empirical evidence on how the softmax-level and feature-level calibrations work differently at the penultimate layer in the perspective of feature extraction.
3.1 Problem Formulation
Let D D= = {2,, ) be the target data obtained from joint distribution over X' x Y, where X' is the in-distribution example space and y is the target label space. A DNN model consists of a general feature extractor fø X' - Z E Rd and task-specific layer go : Z + V. Then, the feature extractor is considered as a compound of sub-feature extractors fa, such that fo(x) = (fo, (z)...., Sag (1)} where fø, : X + IR. A feature is defined to be function mapping from the example space X' to real number, and set of the features is denoted by F {f E
fo X - R].
We now formalize the desirableness of feature. Let D ( be the out-of-distribution (OOD)
data obtained from distribution over the OOD example space X. Then, undesirable and desirable features are defined by Definitions 3.1 and 3.2, respectively.
Definition 3.1 (UNDESIRABLE FEATURE). For each example ă in the OOD data D, we call a feature undesirable if it is highly correlated with at least one true label in expectation. Thus, the set F Fundesirable (p) of undesirable features is defined by
F Fundesirable (p) = f E F: Eggøl max (of(f 2 p0).
(I)
where Corr is a function to produce the correlation between two given inputs (e.g., R²) and p is a constant threshold. one. Intuitively speaking, an undesirable feature influences the model's decisionnmakinng even if not relevant to the target task (i.c., OOD examples).
is an absolute value function to convert a negative correlation into a positive
3

---

Definition 3.2 (DESIRABLE FEATURE). For each example a and its corresponding label in the in-distribution data D, we call feature desirable if highly correlated with the true label in expectation and does not belong to F uundesirable (p). Thus, the set Fdesirable (t) of desirable features defined by
ddesirable (c) = - F/FFF unddeivabble (zzg)) [ Corr (fx) e).
where is constant threshold; Corr and are the same those for Definition 3.1.
Note that Definitions 3.1 and 3.2 are generally applicable to any supervised Icarning tasks including classification and regression. By these dcfinitions, feature vector obtained by the feature extractor could be mixture desirable and undesirable features. DNNs can totally memorize even undesirable features owing to their high expressive power, leading to the statistical bias in in-distribution training data. Therefore, the main challenge to prevent the problem of biasing toward the undesirable features, which will be discussed in the next section.
3.2 Main Concept: Feature-Level Calibration
We introduce the notion of the feature-level calibration, which directly manipulates the activations of the general feature extractor fs. The key idea is to force the feature activations of all OOD examples to be zero vectors, thereby preventing the undesirable features from being carried over into the last task-specific layer go. Equation (3) shows the difference in the objective function among standard learning, OAT (softmax-level calibration) [8], and TAUFE (feature-level calibration):
STANDARD: min E.
@.0 (R.0)D
OAT: min E 6.0 (x,0)D [((1((c(())))))
(19(Jo(())), Amm)]]
+
E ZED
(3)
TAUFE: min E (10((((()))) + E RED
0,0 (r,9)ED
where lunir = [1/e,. 1/c and is the target loss for each original task (e.g., cross-entropy loss for classification mean squared error (MSE) loss for regression). The first term is the same for all three methods, but there is difference in the second term. Both OAT and TAUFF use the OOD examples (i.e., D) to avoid the memorization of the undesirable features, but only TAUFF is not dependent on the task-specific layer ge in its regularization mechanism. Therefore, this feature-level calibration is easily applicable to any type of tasks for practical use and, at the same time, reduces the impact of the undesirable features on the model's prediction.
More importantly. TAUFE is remarkably simple. We contend that its simplicity should be strong benefit because simple regularization often makes huge impact and gains widespread use, as witnessed by weight decay and batch normalization.
Algorithm describes the overall procedure of TAUFE, which is self-explanatory.
Algorithm TAUFE
INPUT: D: target data, D: OOD data, epochs: total number of epochs, b: batch size
OUTPUT: dr, 05: network parameters
1: 1 dr, 0g <- Initialize the network parameters;
2: for i 1 to N do
3: for for
|D]/b do Draw a minni-bacc B from D: / A target mini-batch. */
4:
5:
Draw a mini-batch B from D; / An OOD mini-batch. */ 1 Update for the feature extractor by the feature-level calibration. */
6:
7:
Φt+1 6: - a l (E( [g9, (fo, (z)),)) ] + Ege fll. (3)
8:
1* Update for the task task-specific model by the standard manner. */
9:
9.41 0₂ a V (( [e( (fo, (z)),v)}=
10:
1 1+
11:
return tr. 0₁

---

Target Examples
OOD Examples
Decision Boundary
Zeto Voctor
Antivation Space
-Chest
Closs
Actiation Space
Antivtion Space
Class
Class
(a) Standard.
(b) Softmax-level calibration (OAT).
(c) Feature-level calibration (TAUFE).
Figure Effect of the softmax-leve and feature-level calibrations on the penultimate layer activations.
3.3 Theoretical and Empirical Analysis
We analyze that the feature-level calibration works better than the softmax-level calibration in terms of feature disentanglement on the penultimate layer activations. The use of OOD examples with the softmax-level calibration has been theoretically proven to remove undesirable feature contributions to the last linear classifier [8]. However, the proof holds under the strong assumption that desirable and undesirable features should be disentangled perfectly before entering the last classifier layer. Because this assumption does not hold in practice, we provide an in-depth analysis on the use of OOD examples on the perspective of feature extraction without any assumption.
Theoretic Analysis of Softmax-Level Calibration. The effect of the softmax-Ievel calibration is tightly related with label smoothing. which is a regularization technique [20] that uses the target label combined with uniform mixture over all possible labels. Let be the penultimate layer activation and Wi be a weight row-vector of the last linear classifier assigned to the k-th class. Then, the logit 2 wi for the k-th class can be thought of the negative Euclidean distance between and a weight template Wk, because || wx || 2 2 w wy 22 2 where 2 2 and w w2 are usually constant across classes. Therefore, when OAT assigns the uniform softmax probability to OOD examples, each logit 2²'ws is forced into being the same value, which means that the penultimate layer activation z is equally distant to the templates (i.e., clusters) of all classes.
As shown in Figure 2(b). forcing all OOD examples into being equally distant to all class templates is mathematically equivalent to locating them on the hyper-plane across the decision boundaries. While the hyper-plane is orthogonal to the space composed of desirable features, it is likely onto a decision boundary. Accordingly, the undesirable features move the activations of the desirable features toward a decision boundary, and the two types of features are entangled. Overall, although the softmax- level calibration helps remove undesirable features, it partially entangles the undesirable features with the desirable features, degrading the prediction performance.
Theoretic Analysis of Feature-Level Calibration. In contrast to the softmax-level calibration, the feature-level calibration explicitly forces the activations of all OOD examples into approaching the zero vector [21], as shown Figure 2(c). This regularization reduces the norm of all target examples without changing the angle between the activations for different classes if they share undesirable features. Since this angle plays decisive role for classification [22], the feature-level calibration removes the undesirable features while effectively maintaining the disentanglement between desirable and undesirable features. See ğB of the supplementary material for in-depth theoretical analysis.
Empirical Analysis. To empirically support our analysis, in Table 1, we quantitatively cal- culate the cosine similarity of activations across all in-distribution classes. Compared with the standard learning method, TAUFE (feature-level calibration) decreases the cosine similarity be- tween classes, whereas OAT (softmax- cal- ibration) rather increases the cosine similarity. That is, OAT is prone to move the activations of in di-sttibbbutinn examples toward the decision boundary, though it reduces the negative effect of the undesirable features on the classification task. In contrast, it is noteworthy that TAUFE renders the activations of different in-distribution classes more distinguishable. Furthermore, we visualize the penultimate activations of the two in-distribution classes in CIFAR-10 together with those of OOD examples in Figure As shown in Figure 3(b), OAT simply locates the activations of OOD examples around the decision boundary. However, TAUFE forces them into the zero vector without
Table 1: Average cosine similarity between all activation pairs across different classes on CIFAR- 10 for Standard, OAT, and TAUFE.
Datasets
Methods
In-dist.
Out-of-dist.
Standard
OAT
TAUFE
CIFAR-10
LSUN
0.116
0.286
0.095
5

---

40
40
Decision Boundary
40
20
20
0
-20
-20
-20
-40
-40
-40
-40 -20
20
40
-40
-20
0
20
40
-40
-20
0
20
40
(a) Standard.
(b) Softmax-lever calibration (OAT). (c) Feature- calibration (TAUFE).
Figure 3: TSNE visualization of the penultimate layer activations. In-distribution examples are in pink for the automobile class and in blue for the bird class, while all OOD examples are in grey.
much change in the angles between different classes Therefore, the empirical evidences confirm that TAUFE successfully reduces the negative effect of undesirable features the classification task.
4 Experiments
We compare TAUFE with the standard learning method (denoted as "Standard") and the state-of- the-art method [8] Standard trains the network without any calibration process for OOD examples. In addition, we include the few-shot learning settings because DNNs are easily biased toward the undesirable features especially when the number of training examples is small. All methods are implemented with PyTorch 1.8.0 and executed using four NVIDIA Tesla V100 GPUs. For reproducibility, we provide the source code at https: /ggttuub. com/kaist wwkkasttiaaaaa/aaeeaaaaaaaaaaaaa www.ddmTaFFTSS In support of reliable evaluations, repeat every test five times and report the average.
To show high flexibility to diverse types of tasks, we rigorously validate the efficacy of TAUFE for three popular visual recognition tasks: (i) image classification, (ii) bounding box regression, and (iii) weakly supervised object localization (WSOL). Please note that OAT does not support the bounding box regression task because of the absence of the softmax layer.
4.1 Task I: Image Classification
Dataset. We choose CIFAR-10 10, CIFAR- [23], and ImageNet [24] for the target in nn-iistribution data. For the CIFAR datasets, two out- -id------itttittiinn datasets are carefully mixed for evaluation LSUN [25], a scene understanding dataset 59M images with 10 classes such as bedroom and living room, and SVHN [26]. real-world house numbers dataset of 70K images with 10 classes. The ImageNet dataset is divided into 12K images 10 randomly selected classes (ImageNet-10) and 1.1M images of the rest 990 classes (ImageNet-990); the former and the latter used in in-distribution data and OOD data, respectively. A large-scale collection of place scene images with 365 classes, Places365 [27]. is also used as another OOD data for ImageNet-10.
Training Configuration. For CIFAR datasets, ResNet-18 18 [28] is trained from scratch for 200 epochs using SGD with a momentum of 0.9, batch size of 64, weight decay of 0.0005. To support the original resolution, we drop the first pooling layer and change the first convolution layer with a kernel size of 3. stride size of 1, and a padding size of 1. An initial learning rate of 0.1 is decayed by a factor of 10 100-th and 150-th epochs, following the same configuration in OAT [8]. For the ImageNet- 10 dataset, ResNet-50 is used without any modification, but the resolution of ImageNet- 10 is resized into 64x x 64 and 224x 124x224 224 order to see the effect of different resolutions. Resized random crops and random horizontal flips applied for data augmentation.
TAUFE requires only one additional hyperparameter, the scaling factor À for the feature-level cal- ibration in Equation (3). The value of À is set to be 0.1 and 0.01 for CIFARs and ImageNet- 10, respectively, where the best values are obtained via grid search. The corresponding hyperparameter in OAT for loftmax-level calibration is set to be 1, following the original paper. In addition, both few-shot and full-shot learning settings are considered for evaluation. Given the number of the examples for use in few-shot learning. N examples are randomly sampled over all classes from both in-distribution and OOD data, and thus 2N examples total are used for training. For full-shot learning, N is set to be the total number of training examples in the target in indiitribbuton data.

---

Table 2: Classification accuracy (%) of TAUFE compared with Standard and OAT on two CI- FARs (32x32), ImageNet- 10 (44x64), and ImageNet-100 (2224x224) under few-shot and full-shot learning settings. The highest values are marked bold.
Datasets
Examples (N)
Methods
In-dist.
Out-of-dist.
500
1,000
2,500
5,000
Full-shot
Standard
38.58
52.63
72.94
82.38
94.22
OAT
40.55
52.80
73.24
82.56
94.38
CIFAR-10 10 (32x32)
SVHN
TAUFE
41.58
56.72
73.61
82.88
94.45
OAT
40.73
53.16
73.51
82.71
94.61
LSUN
TAUFE
42.51
56.79
74.15
83.73
95.02
Standard
11.07
13.99
24.28
41.47
73.84
OAT
10.92
14.56
24.67
42.21
74.82
CIFAR-100
SVHN
TAUFE
11.30
15.13
24.91
43.61
75.38
(32x32)
OAT
11.27
15.24
24.75
43.09
75.15
LSUN
TAUFF
12.26
15.97
25.36
44.50
75.69
Standard
38.82
43.66
56.17
66.80
78.30
ImageNet-10 (64x64)
ImageNet-99 990 990
OAT
38.95
44.09
$7.29
69.41
79.29
TAUFE
42.80
46.04
60.40
70.51
81.09
OAT
41.06
43.81
56.47
67.20
79.30
Places365
TAUFE
43.25
47.61
60.02
68.25
80.89
Standard
44.82
56.29
73.60
82.49
86.97
ImageNct-Il 10 (224x224)
ImageNet-990
OAT
46.41
58.66
75.62
83.6
87.66
TAUFF
48.39
59.06
76.47
85.05
89.24
OAT
48.10
56.88
74.98
83.41
88.78
Places365
TAUFE
50.08
59.27
77.22
84.81
89.06
Performance Comparison. Table 2 shows the classification accuracy of the three methods under few-shot and full-shot learning settings. Overall, TAUFE shows the highest classification accuracy at any few-shot settings for all datasets. Specifically, TAUFE outperforms OAT by 0.07% to 9.88%, though OAT also shows consistent performance improvement. OAT's lower performance attributed to the property that is prone to force the activations of in-distribution examples toward the decision boundary as analyzed in $ 3.3. Adding LSUN as OOD for CIFARs is more effective than adding SVHN, because LSUN is more similar to CIFARs than SVHN, thus sharing more undesirable features. For ImageNet- 10, adding Places365 is more effective than adding ImageNet-990 when the number of training examples is not enough, but adding ImageNet-990 becomes more effective as the size of training data increases. Because ImageNet-990 has more diverse background scenes than Places365, we conjecture that the effect of Places365 saturates faster than that of ImageNet-999 as more OOD examples are exposed to the DNN model. Besides, no significant difference observed depending on the resolution of ImageNet-10.
Performance with Semi-Supervised Learning. We semi-supervised learning framework for baseline in addition to the standard supervised learning framework, because TAUFE can also improve the accuracy semi-supervised classifier by deactivating the undesirable features. MixMatch [29], a popular semi-supervised learner for image classification, is enhanced with TAUFE by to 2.02% and 2. 2.34%, respectively, on two CIFAR datasets. $ 7.1 shows the details.
Effect on Adversarial Robustness. We investigate the effect of TAUFE on adversarial robustness. Overall, TAUFE improves the accuracy adversarial examples by up to 2.76% when adding the LSUN dataset as OOD. & 7.2 shows the details.
Effect on OOD Detection. As TAUFF is not intended to detect OOD examples, its effect on 0OD detection is not noticeable, as shown C of the supplementary material.
4.2 Task II: Bounding Box Regression
Bounding box regression an essential sub-task object localization and object detection. We compare TAUFE with only Standard because OAT does not work for regression.
7

---

Table 3: IoU (%) of TAUFE compared with Standard on CUB200 (224x224) and CAR (224x224)
under sho-shot and full-shot learning settings. The highest values are marked in bold.
L1
LI-IoU
D-IoU
Datasets
Methods
# Examples (N)
# Examples (N)
# Examples (N)
In-dist.
Out-of-dist.
2,000
4,000
Full
2,000
4.000
Full
2,000
4,000
Full
Standard
66.41
73.10
76.42
66.57
73.28
76.67
66.82
73.18
76.57
CUB200 (224x224)
ImageNet
TAUFE
67.16
74.31
77.12
67.22
74.40
77.24
67.03
74.22
77.00
Places365
TAUFE
66.70
73.55
76.86
66.87
73.63
77.01
66.88
73.66
76.88
Standard
83.06
85.50
90.56
83.52
86.54
91.25
83.62
87.93
91.09
CAR
ImageNet
TaUFF
85.23
87.82
91.32
85.82
89.11
91.40
85.30
89.06
91.35
(224x224)
Places365
TAUFE
84.26
87.59
90.86
84.73
88.83
91.28
84.60
88 64 91.20
Table 4: GT-known Loc of TAUFE compared with Standard on CUB200 (224x224) and CAR x24x)22) under few-shot and full-shot learning settings. The highest values are marked in bold.
Datasets
Examples (N)
Methods
In-dist.
Out-of-dist.
2,000
4,000
Full-shot
Standard
54.45
58.37
64.02
OAT
55.24
60.24
64.91
CUB200
ImageNet
TAUFE
59.68
61.88
65.56
(224x224)
OAT
56.97
60.01
64.27
Places365
TAUFE
58.24
60.90
64.84
Standard
62.09
67.12
70.54
OAT
63.77
67.24
71.64
CAR
ImageNet
TAUFE
65.82
69.05
72.14
(32x32)
OAT
63.16
68.58
71.66
Places365
TAUFE
65.70
67.64
71.62
Dataset. Two datasets are used as the target in-distribution data for the bounding box regression lask-Caltech-Ulee Caltech-UCSD Birds-200-200 (CUB200) [30], collection of 6,033 bird images with 200 classes, and Standford Cars (Car) [31], a collection of 8, 144 car images with 196 classes. For each resolution, the two datasets contain class label and bounding box coordinates of
image of the top-left and bottom-right corners. ImageNer" and Places365 are used as OOD data.
Training Configuration. ResNet-50 is trained from scratch using SGD for 100 epochs. Following the prior work [32], the last classification layer ResNet-50 is converted to a box regressor that predicts the bounding box coordinates of the top-left and bottom-right corners. In addition, we use three types of different loss functions: (i) L11, I.1 -smooth loss, (ii) L.- combination of I.1 and IoU, and (iii) D- [33], a combination of L11, IoU, and the normalized distance between the predicted box and the target box. The remaining configurations are the same as those in 4.1.
Evaluation Metric. We adopt the Intersection over Union (IoU), which is the most widely-used metric for bounding box regression and defined by loU(b, 5,) EN [b n b///,, Ub where b; and b, are the ground-trute and predicted bounding boxes of the object in the i-th example.
Performance Comparison. Table 3 shows the IoU accuracy of the two methods. Overall, TAUFE consistently boosts the performance on bounding box regression for all datasets regardless of the loss type. Quantitatively, the box regression performance considerably improves with TAUFF by up to 2.97% when using L.--oUU. This result indicates that the of OOD examples with the feature-level calibration indeed alleviates the undesirable bias problem. Interestingly, adding ImageNet as OOD for both CUB200 and CAR is more effective than adding Places365, possibly because ImageNet contains a higher number of classes which reflect more diverse undesirable features.
2 All bird and vehicle relevant classes excluded from the ImageNet dataset.

---

4.3 Task III: Weakly Supervised Object Localization (WSOL)
WSOL is a problem of localizing a salient foreground object in an image by using only weak supervision (i.e., image-level class labels). It can be considered a mix of classification and regression because it uses class labels but aims at bounding box regression. The seminal WSOL work, class activation mapping (CAM) [34], has shown that the intermediate classifier activations focus on the most discriminative parts of the target object in the image. Thus, by simply averaging all local activations, we can estimate how much the corresponding pixels contribute discriminate the object in the scene. CAM is used as the standard learning method. We refer the reader to the surveys [35] for more details about WSOL.
Dataset. Like the bounding box regression task, CUB200 and CAR are used as in-ditribution data, while Places365 and ImageNet are used as OOD data.
Training Configuration. ResNet-50 is trained from scratch for 100 epochs using SGD with a batch size of 64. An initial learning rate of 0.1 is decayed by a factor of 10 at 50-th and 75-th epochs. The remaining configurations are the same as those in 8 4.1.
Evaluation Metric. We adopt the localization accuracy with known ground-truth class (GT-known Loc), which is the most widely-used metric for WSOL and defined by GT L
b,) 1 DE= 1 (Ib 6,) 2 5) where 6; is the ground-truth box of the object in the i-th example and b; N
is the tightest box around the largest connected component of the activation mask for the i-th example. The IoU threshold 5 is set to be 0.5, following the prior work [34, 35].
Performance Comparison. Table shows the GT-known accuracy of the three methods. Overall, TAUFE shows the best localization accuracy at any few-shot settings for all datasets. Specifically, TAUFR outperforms OAT by 0.71% to 8.03 8.03%, though OAT also shows consistent performance improvement. This result indicates that TAUFE successfully removes the undesirable features such as background to locate an object in an image. Adding ImageNet as OOD is more effective than adding Places365 for the same reason. Besides, the performance gain of TAUFE over Standard is typically larger for few-shot learning than for full-shot learning, as observed the other tasks.
Limitations and Future Work
Although TAUFE has shown consistent performance improvements in three types of real-world machine learning tasks, there are some issues that need to be further discussed. First, the effectiveness of an OOD dataset for given a target dataset and a task needs to be formulated theoretically. Owing to the transferability of undesirable features, any OOD dataset can be effective but its effectiveness varies as shown in $ 4. The difference in the effectiveness may come from the amount of shared undesirable features between the target dataset and each OOD dataset. Therefore, formulating the effectiveness based on such factors is an interesting research direction. Second, the applicability of TAUFF need to be verified for a wide range of learning frameworks including self-supervised learning, semi-supervised learning, and meta-learning because the bias toward undesirable features is likely to be observed regardless of the learning frameworks. Thus, we will clarify the outcome of TAUFE with varying the learning frameworks as future work.
Conclusion
In this paper, we propose TAUFF, a novel lask-agnostic framework to reduce the bias toward undesirable features when training DNNs. Since the existing softmax-level calibration method confines its applicability to only the classification task, we overcome the limitation by introducing the feature-level calibration that directly manipulates the feature output of a general feature extractor (c.g.. a convolutional neural network). To remove the effect of undesirable features on the final task-specific module, TAUFE simply deactivates many undesirable features extracted from the OOD data by regularizing them as zero vectors. Moreover, we provide an insight on how differently feature-level and softmax-level calibrations affect feature extraction by theoretic and empirical analysis on the penultimate layer activation. The consistent performance improvement on three types of tasks clearly demonstrates the aask-agnostic nature of TAUFE. Overall, we believe that our work sheds the light on the usability of the OOD data in diverse tasks.

---

Table 5: Classification accuracy (%) of TAUFE under few-shot semi-supervised learning settings.
Table Accuracy (%) of TAUFE under the PGD adversarial attacker.
In-dis.
CIEAR-10
CIFAR-100
In-dist.
CIEAR-I0
Out dist.
SVII
LSUN
SVIN
LSUN
Out-of-dis
SVHN
LSUN
Methods
MixMarch
TAMFFRii
TAUIFMMI
MisMMsch
TAUFEMG
TAUFEMis
Mechods
Standard
PGD
TAUFFPGO
TAUFEPGI
Accuracy
58, 32
90.02
90.10
51.38
$2.32
32
52.58
Cleam. acc.
94.22
71.22
72.35
72.31
7 Adv. soc.
70.00
144.26
144.37
/45.48
7 Supplementary Experiments
7.1 Performance of TAUFE with Semi-Supervised Learning
Baseline. MixMatch [29] is of the state-of-the-art semi-supervised learning frameworks for image classification. By leveraging unlabeled examples with automatic label guessing and mix-up techniques, MixMatch nearly reaches the accuracy of fully supervised learning using only a small number of labeled examples.
Experiment Setting, CIFAR-I0 and CIFAR- 100 are used for in-distribution datasets, and LSUN and SVHN are used for two OOD datasets. We use the default or best hyperparameter values suggested by the authors [29]. Specifically, the sharpening temperature T is set to be 0.5, the number of augmentations
to be 2. the Beta distribution parameter a to be 0.75, and the loss weight for unlabeled examples A be 100. We fix the number of epochs to be 1, 1,024 and the batch size to be 64, and linearly ramp up do in the first 16,000 optimization steps. We use 25 labeled examples per class initially labeled data, because MixMatch shown nearly reach the full-supervision accuracy on that setting [29].
Result. Table 5 shows the classification accuracy of TAUFE combined with MixMatch on two CIFAR datasets under few-shot settings-i.e. 252 for CIFAR-I0 and N=2,500 for CIFAR-100; TAUFEMs represents the TAUFE combined with MixMatch. TAUFEMix consistently improves the performance of MixMatch on two CIFAR datasets. Similar to the supervised learning in 3 4.1, adding LSUN OOD is more effective than adding SVHN; compared with MixMatch, the performance of TAUFFMix is improved by up to 2.02% on CIFAR-10 and by up to 2.34% on CIFAR-100. This result shows that TAUFE successfully deactivates the negative effect of undesirable features even in the semi-supervised learning setting.
7.2 Effect of TAUFE on Adversarial Robustness
Baseline. We use the projected gradient descent (PGD) [36] attack / learning method, which employs an iterative procedure the fast gradient sign method (FGSM) [37] to find the worst-case examples having the maximum training loss.
Experiment Setting. CIFAR- 10 and CIFAR- are used for in-distribution datasets, and LSUN and SVHN are used for two OOD datasets which are exposed in the training phase. The hyperparameters of PGD are favorably set to be the best values reported in the original paper. The attack learning rate is set to be 2, and PGD,, indicates the PGD attacks with iterative FGSM procedures. For adversarial learning, the adversarial examples generated by PGD, are used as the input of training. To measure the adversarial accuracy, the adversarial examples generated by PGD100 are used for testing. This combination of step numbers was also used in the PGD work [36].
The clean accuracy is the classification accuracy on the original test data, while the adverserial accuracy is that on the PGD100 perturbed adversarial examples of the test data.
Evaluation Metric.
Result. Table 6 shows the adversarial robustness of TAUFE compared with the standard learning method. Overall, TAUFE improves the accuracy adversarial examples by up to 2.76% when adding the LSUN dataset as OOD examples. This result indicates that the undesirable feature deactivation of TAUFE is helpful for the adversarial learning models.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by Institute of Information & Communications Technology Planning & Evaluation (ITTP) grant funded by the Korea government (MSIT) (No. 2020-0-00862, DB4DL: High-Usability and Performance In-Memory Distributed DBMS for Deep Learning).
References
[1] Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In ICML, pages 528-539, 2020.
[2] Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P Xing. High-frequency component helps explain the generalization of convolutional neural networks. In CVPR, pages 8684-86889 2020.
[3] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry, Adversarial examples are not bugs, they are features. arXiv preprint 175,0211111100 2019.
[4] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530, 2016.
[5] Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: survey. arXiv preprint arXiv:2007.08799, 2020.
[6] Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl-Dickstein. On the expressive power of deep neural networks. In ICML, pages 2847-2854, 2017.
[7] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. ICLR, 2019.
[8] Saehyung Lee, Changhwa Park, Hyungyu Lee, Jihun Yi, Jonghyun Lee, and Sungroh Yoon. Removing undesirable feature contributions using out-of-distribution data. In ICLR, 2021.
[9] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In ICLR, 2019.
[10] Kemal Oksuz, Baris Can Cam, Sinan Kalkan, and Emre Akbas. Imbalance problems in object detection: review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
[11] Bharat Singh and Larry S Davis. An analysis of scale invariance in object detection snip. In CVPR, pages 3578-3587, 2018.
[12] Philippe Weinzaepfel and Grégory Rogez. Mimetics: Towards understanding human actions out of context. International Journal of Computer Vision, 129(5):1655-1690 2021.
[13] Yi Li and Nuno Vasconcelos. Repair: Removing representation bias by dataset resampling, In CVPR, pages 9572-9581, 2019.
[14] Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R Bowman, and Noah A Smith. Annotation artifacts in natural language inference data. arXiv preprint 18X00:1803.02324, 2018.
[15) Xiaoyong Yuan, Pan He, Qile 7hu, and Xiaolin Li. Adversarial examples: Attacks and defenses for deep learning. IEEE Transactions on Neural Networks and Learning Systems, 2005):2805- 2824, 2019.
[16] Muzammal Naseer, Salman Khan, Muhammad Haris Khan, Fahad Khan, and Fatih Porikli. Cross-domain transferability of adversarial perturbations. In NeurIPS, pages 12885-12895, 2019.
[17] HyunJae Lee, Hyo-Fun Kim, and Hyeonseob Nam. SRM: style-based recalibration module for convolutional neural networks. In ICCV, pages 1854-1862, 2019.

---

[18] Rakshith Shetty, Bernt Schiele. and Mario Fritz. Not using the car to see the sidewalk- quantifying and controlling the effects of context in classification and segmentation. In CVPR, pages 8218-8226, 2019.
[19] Haohan Wang, Zexue He, Zachary C Lipton, and Eric P Xing. Learning robust representations by projecting superficial statistics out. In ICLR, 2018.
[20] Rafael Müller, Simon Kornblith, and Geoffrey Hinton. When does label smoothing help? In NeurlPS, pages 4696-4705, 2019.
[21] Dongmin Park, Hwanjun Song, Minseok Kim, and Jac-Gil Lee. TRAP: Two-level regularized autoencoder-basse embedding for power-law distributed data. In Proceedings of The Web Conference 2020, pages 1615-1624, 2020.
[22] Hanting Chen, Yunhe Wang, Chunjing Xu, Boxin Shi, Chao Xu, Qi Tian, and Chang Xu. Addernet: Do we really need multiplications in deep learning? In CVPR, pages 1468-1477, 2020.
[23] Alex Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis, University of Toronto, 2009.
[24] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009.
[25] Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. LSUN: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint 1506.00303336366 20155.
[26] Pierre Sermanet, Soumith Chintala, and Yann LeCun. Convolutional neural networks applied to house numbers digit classification. ICPR, pages 3288-3291, 2012.
[27] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligenc, 40(6)::452-146:) 2017.
[28] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.
[29] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. MixMatth: holistic approach semi-supervised learning. In NeurlPS, pages 5050-5060, 2019.
[30] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech- UCSD Birds-200-2011 Dataset. Technical Report 1-201-01001--00 California Institute of Technology, 2011.
[31] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3D object representations for fine-grained categorization. In ICCVW, pages 554-561, 2013.
[32] Yihui He, Chenchen Zhu, Jianren Wang, Marios Savvides, and Xiangyu Zhang. Bounding box regression with uncertainty for accurate object detection. In CVPR, pages 2888-2897, 2019.
[33] Zhaohui Zheng, Ping Wang. Wei Liu, Jinze Li, Rongguang Ye, and Dongwei Ren. Distance-IoU loss: Faster and better learning for bounding box regression. AAAI, pages 12993-13000, 2020.
[34] Bolei Zhou, Aditya Khosla, Agata Lapedriza. Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In CVPR, pages 2921-2929, 2016.
[35] Dingwen Zhang, Junwei Han, Gong Cheng, and Ming-Hsuan Yang. Weakly supervised object localization and detection: A survey. IEEE Transactions Pattern Analysis and Machine Intelligence, 2021.
[36] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
12

---

assets... [37] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. In ICLR, 2015.
[38] Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod K Varshney, and Dawn Song. Anomalous example detection in deep learning: A survey, IEEE Access, 8:132330-132347, 2020.
[39] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples neural networks. JCLR, 2017.
[40] Weitang Liu, Xiaoyun Wang, John D Owens, and Yixuan Li, Energy-based out-of-distribution detection. In NeurIPS, 2020.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes] See Section 1.
(b) Did you describe the limitations of your work? [Yes] See Section 5.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See the supplemental material.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section 3.3.
(b) Did you include complete proofs of all theoretical results? [Yes] See Section 3.3 and the supplemental material.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed reproduce the main perimental results (either in the supplemental material or as a URL)? [Yes] See the supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 4.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] See the supplemental material.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section 4.
4. If you using existing assets (e.g., code, data, models) or curating/releasing
(a) If your work uses existing assets, did you cite the creators? [Yes] We used public benchmark datasets. See Section
(b) Did you mention the license of the assets? [Yes] The datasets are all public. See Section 4. (c) Did you include any new assets either the supplemental material or as URL? [No] (d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you deseribe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "aGKtBRGj7-J", "reviewer": "Reviewer_32rF", "summary": "TAUFE, a novel regularizer has been presented to handle OOD. It rigorously validates its performance on three tasks, classification, regression, and a mix of them, on CIFAR-10, CIFAR-100, ImageNet, CUB200, and CAR datasets with promising results.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Novelty:\nThe paper overall is a novel take on OOD problem. TAUFE, a novel regularizer claims to deactivate undesirable features using OOD examples in the feature extraction layer and thus removes the dependency on the task-specific softmax layer. \n\nStrengths:\n1. Code is given zipped as well as URL for verification.\n2. Apart from classification tasks, where OOD has started showing some grip, this work also takes care of regression and a mix of them.\n3. The experiments and validation on standard datasets creates a good impression wrt results.\n4. Mathematical formulation and comparison with softmax layer is sound.\n\nWeakness:\n1. there was ample scope to add finer details of implementation and ablation studies in the supplementary section.\n2. Ablation on self to understand the pitfalls where method is failing and identifying the corner and failure cases will have made the paper better as well as laid path for future work in need to fill gaps.\n3. \"deactivates all undesirable features \" - is this term not too bold to say?\n\nRelevance to NeurIPS:\nVery relevant, aligned to hot area of research \"out-of-distribution\" handling.\n\nClarity:\nThe paper is well written.\n\nRelated work Comparison:\nThere is substantial comparison with other works and standard datasets.\n\nEffort: In terms of thought process, experiments effort is there. However some effort on analysis and corner case / failure identification would have made this a very good effort.\n\nSuggestions:\n1. Probably, varying the types of data of the datasets (datatype) might help in getting deeper in the problem as a generalization tool.]\n2. Some explanation needed what is meant by undesirable features and desirable features? Can a bias on desirable features help in drawing the boundary line? Is there a possibility of a membership function based boundary line?\n3. How will these work in adverserial examples?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Not applicable here.", "review_text": "", "extra_scores": {"ethical_concerns": "Plagiarism is 10% as seen in Docoloc - https://www.docoloc.de/d88c4d5162dd4b21eeb6add446e2b5bbCEuFHG85rulAsphkx9/en/konto.hhtml?dogetresult=52 - hence OK - cleared - green signal.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "RTh1sT9u1ku", "reviewer": "Reviewer_d4nE", "summary": "The paper proposes a simple regularization that encourages logits to be close to zero on a designated out-distribution during training. The authors show that this improves accuracy across various tasks and datasets.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nThe proposed method is quite simple and the results in the paper indicate that it can be very effective at boosting accuracy. The method is also task-agnostic which means it can be applied beyond just, e.g. image classification. Also, the authors have used several different training out-distributions which gives more information about the dependence on the choice of this distribution than many other papers in the field. The inclusion of few-shot results is also interesting.\n\n\nWeaknesses:\nA major issue with this paper is the weakness of their baseline. Even when only using a ResNet18 on CIFAR10 a plain trained classifier can attain 95.0% using AutoAugment. Similarly, an outlier exposure (OE being, as far as I can tell, identical to the OAT that they compare to) trained model can achieve 95.5% when using the same 80mTiny dataset as out-distribution as the authors. On CIFAR100 (77.4% and 77.3%, respectively). In other words, their improvement beyond their baseline is much smaller than their gap to a simple model trained using AutoAugment. Would their results continue to hold when using stronger baselines?\n\nAnother issue is that they only compare the classification performance in terms of accuracy. Outlier-aware methods are typically designed to improve out-of-distribution detection. Does their method perform better or worse than OE/OAT on this task? Even if it is not the main focus of the paper, these results need to at least be discussed and included in the appendix.\n\nSince the focus of the paper is so heavily on improving the performance on the original task, I also think that they need to include baselines from the semi-supervised literature (for example, [1], but hopefully also more recent works).\n\n\nClarity:\nThe clarity of the paper needs much improvement. Especially experimental details are not fully laid out, e.g. line 236 \"best values are obtained via a grid search.\" What was the grid? How were the validation sets chosen? What was the performance metric that they optimized for?\n\nI also find Table 1 a little unclear. The caption states \"Average cosine similarity between each classe’s activation on CIFAR-10 for Standard, OAT,and TAUFE\". I understand this to mean that the cosine similarity between the features of samples within the same class was computed and that those results were averaged across classes. If I understand this correctly, then I do not understand how this is a bad thing. Higher similarity within classes does not seem like an issue. The accompanying discussion in the main text seems to indicate that the authors are instead talking about the cosine similarities between the different classes. How is this computed exactly? The mean of each class is computed and those are compared? Or the cosine similarities with samples from other classes are computed and then averaged? \n\nThe authors also don't explain what ImageNet-10 is. They write in line 222 that it consists of \"randomly selected classes\". Which classes? This needs to be in the appendix.\n\n\nReproducibility:\nI very much appreciate that the authors provided their code in the submission.\n\n\nAdditional feedback:\nThe definitions of desirable and undesirable features should probably be modified slightly to be called rho-desirable or rho-undesirable. This is especially important in Eq. 2 where F_out is defined in terms of a rho, that is not explicitly given. The authors clearly meant to assume that rho is the same for both definitions but the way it is written now is sloppy. Also, the discussion below it should maybe point out, that \"a feature vector [...] could be a mixture of desirable and undesirable features\" but that it could very well contain neither.\n\nAnother point that I would appreciate some clarification on is in line 240 where the authors state that they only sample N points from the out-distribution - even in the full-shot regime. This appears somewhat non-standard so I would have liked some justification for this choice in the experimental setup.\n\n[1] There are many consistent explanations of unlabeled data: Why you should average; Ben Athiwaratkun, Marc Finzi, Pavel Izmailov, Andrew Gordon Wilson; at ICLR'19", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I am not quite sure why the authors chose to move their limitations section into the appendix. They clearly still had plenty of space on the final page and the page limit was extended to 9 pages in part to allow for this sort of discussion in the main paper. I recommend moving this back into the main paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {"ethical_concerns": "There is the issue of using 80mTiny as an out-distribution since the dataset has been retracted by the authors over ethical concerns. Arguably, it is okay to use it for comparison with old work if the authos also provide results on a different OOD dataset that can be used for future works to compare with. Fortunately, the authors do provide experiments using SVHN as well. However, this issue needs to be discussed at the very least. As it stands right now I cannot agree with the author's sentiment in point 1. d) of the checklist that their paper conforms to the ethics guidelines. "}}, {"review_id": "PzyxE-0BGNK", "reviewer": "Reviewer_UyJi", "summary": "This paper describes a simple yet effective approach to deactivate the undesirable features using out-of-distribution data. By shrinking the undesirable features from feature extraction layers instead of task-specific layers using L2 penalty, this method is task agnostic. Authors demonstrated and compared performance with task-specific approaches on classification, regression and mixed tasks using several benchmark data sets. ", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The key idea of this paper is to leverage L2 penalty to shrink the undesirable features extracted from OOD examples towards zero by assuming OOD and ID examples share such undesirable features. Compared with the recently published task-specific OAT approach, this method is task-agnostic yet simple to implement. Since L2 penalty shrinks the correlated features as groups, can authors provide some analysis to show the TAUFE approach de-activates groups of undesirable features but not much on the groups of desirable features? Authors provide Figure 2 illustration with some descriptions from lines 180-186. It would be great if authors can give in-depth analysis to show L2 penalty indeed shrinks undesirable feature groups without deactivating too much on desirable feature groups.\n\nRelated comment is on empirical analysis of the claimed properties using experiments. Authors successfully show TAUFE renders the activation of ID classes more distinguishable. However, it is necessary to empirically show that the undesirable feature groups have indeed been deactivated whereas not as much for the desirable feature groups. For example, authors might visualize the examples in desirable and undesirable feature spaces, respectively. Even if the method is demonstrated as task-agnostic analytically and empirically as requested, it can be OOD-dependent, i.e., the undesirable feature deactivation would depend on the choice of OOD examples. As such, the generalizability can be limited to the selected OOD examples from training only. \n\nThe reviewer is willing to raise the rating if authors successfully address the major concerns as mentioned above.\n \nMinor comments:\nCan authors also report experimental results for standard method using both ID and OOD examples?  \nTAUFE does not seem to outperform OAT across all settings. It is also not clear whether OAT has been appropriately optimized for its performance. \n\n------------- New comments after reading authors' rebuttal\nAuthors have successfully addressed my concerns by providing an in-depth analysis to show the TAUFE approach deactivates the groups of undesirable features but not much on the groups of desirable features. As such, I raise my review rating to be positive. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N/A", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "EGWngUKwG5a", "reviewer": "Reviewer_MG8Y", "summary": "Problem: DNN predictions may get influenced by the undesirable features that are irrelevant to the task at hand, which causes poor  generalization. \nSolution: Existing work showed calibration on out-of-distribution (OOD) samples can remove the contribution of undesirable features in classification. The caveat is its applicability to feature extraction. The paper proposes TAUFE, a novel regularizer that deactivates all the undesirable features in OOD examples from the feature-extraction layer, thereby removing the dependency on the task specific softmax layer.\nResults: TAUFE outperforms state-of-the-art methods on the standard benchmarks for classification, regression and the mix of the two.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Distinction from state-of-the-art:\nThe proposed approach clearly identifies the gap between the existing techniques to be applied for task agnostic feature extraction and the lack of flexibility and feature entanglement. \n\nPros:\nThe paper is well written, easy to understand and digest the concepts.\n\nThe very idea of task agnostic training of a deep model for feature extraction and then doing the tasks specific predictions is interesting.\n\nThe paper accurately highlights how deep models use easy features in images, video, object detection, etc, more often they can be irrelevant, in which case the proposed method can be more useful for accurate predictions.\n\nCons:\n\nSince data augmentation seems to produce unbiased examples (as explained in [17]), how does TAUFE fare against those methods? Maybe why not compare against MixUp like [8] and show the effectiveness, at least for classification if not for regression?\n\nWill TAUFE withstand the adversarial attacks? Maybe this is too much in a single paper, but it is worth throwing some light into the performance of TAUFE for adversarial attacks.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are unknown performance for adversarial attacks. \nIn fact the authors should do a better job explaining the limitations and the societal impact.", "review_text": "", "extra_scores": {"ethical_concerns": "None that are bothering at this point.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 -- 4 "}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a simple regularisation that encourages features to be close to zero on a pre-specified OOD training set.

Reviewers think the proposed approach is novel, simple, and effectively demonstrated on a variety of experiments. An ethical issue has been raised on the 80mTiny dataset used in experiments, for which the authors promise to remove it. So it comes down to the question whether the remaining result is significant enough to demonstrate the claims, for which the reviewers do not see it as an issue.

Because of this issue with the 80mTiny dataset, the paper is being marked as conditionally accept.

Reviewers raise another concern on missing comparison to semi-supervised learning approaches, and the authors agree to add in experiments in revision. Label/logit smoothing and feature denoising have also been extensively discussed in adversarial robustness literature, it would improve the paper if the authors can evaluate the proposed approach on adversarial examples.

Also I would suggest to openly discuss the limitation of the proposed approach in the main text rather than in appendix.

----

UPDATE: Upon reviewing the revision, the decision has been updated to accept.
Decision: Accept (Poster)