Task-Agnostic Undesirable Feature Deactivation Using Out-of-Distribution Data

1 Introduction . Undesirable features , which are informally defined as those not relevant to a target task , frequently appear in training data ; for example , the background is an undesirable feature for classifying animals in images . The undesirable features are mainly caused by the statistical bias in in-distribution training data . In fact , many undesirable features are statistically correlated with labels , even though they are unnecessary and sometimes even harmful for the target task [ 1 ] ; for example , the “ desert ” background feature is correlated with “ camels ” because the camels frequently appear in a desert . However , such undesirable features ( e.g. , desert background ) rather yield unreliable predictions because they are easily shifted in other unseen data ( e.g. , images of the camels on the road ) . Meanwhile , deep neural networks ( DNNs ) are known to overly capture any high-frequency data components which are even imperceptible to a human [ 2 , 3 ] . This property is attributed to the vulnerability of DNNs that can totally overfit to random labels or adversarial examples owing to their extremely high capacity [ 3 , 4 , 5 , 6 ] . Accordingly , DNNs are easily biased toward the undesirable features as well , thereby often showing unsatisfactory generalization to unseen examples [ 7 ] . Thus , it is very important to prevent overfitting to the undesirable features . In this regard , a few research efforts have been devoted to remove the negative influence of undesirable features by leveraging out-of-distribution ( OOD ) data [ 8 , 9 ] . Under the assumption that in-distribution and OOD data share undesirable features , OOD data is treated as a useful resource to alleviate the aforementioned undesirable bias . Notably , a recent study [ 8 ] proposed a softmax-level calibration , ∗Corresponding author . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . which assigns uniform softmax probabilities to all possible labels for all examples in OOD data . Although this approach shows decent de-biasing performance in the classification task , the softmaxlevel calibration has two limitations : • Lack of Flexibility : The softmax-level calibration is designated only for the classification task . However , the bias toward undesirable features occurs in numerous tasks , such as object localization and bounding box regression . Therefore , a flexible , task-agnostic approach is required for easily supporting other downstream tasks too . • Feature Entanglement : Even desirable features can be entangled with undesirable ones by assigning the uniform softmax probability invariably to all possible labels for OOD examples . Thus , the negative influence of the undesirable features is not perfectly removed because they still remain and affect the activation of desirable features ( See § 3.3 for details ) . In this paper , we propose a novel task-agnostic and feature-level calibration method , called TAUFE ( Task-Agnostic Undesirable Feature dEactivation ) , which explicitly forces a model to produce zero values for many undesirable features in OOD examples . Differently from the softmax-level calibration that regularizes the classification layer ( Figure 1 ( a ) ) , TAUFE exploits the penultimate layer right before the classification layer and deactivates its activation only for OOD examples ( Figure 1 ( b ) ) . Thus , TAUFE is applicable to any task that requires another task-specific layer other than the classification layer , and the undesirable features are removed in the feature level without feature entanglement . The superiority of the proposed feature-level calibration over the softmax-level calibration is proven by theoretical and empirical analysis of the feature activation of the penultimate layer . To validate its general efficacy , we conducted extensive experiments through three tasks : ( i ) image classification for classification ; ( ii ) bounding box regression for regression ; and ( iii ) weakly supervised object localization ( WSOL ) for a mix of them . We tested multiple pairs of in-distribution and OOD data : CIFAR-10 , CIFAR-100 , ImageNet , CUB200 , and CAR for in-distribution ; and SVHN , LSUN , and Places365 for OOD . The experiment results demonstrate that TAUFE consistently outperforms the softmax-level calibrator [ 8 ] by up to 9.88 % for classification and by up to 8.03 % for the mix of classification and regression . Our main contributions are summarized as follows : 1 . We propose a simple yet effective method , TAUFE , to deactivate undesirable features in learning , which is easily applicable to any standard learning task with recent DNNs . 2 . We provide an insight on how feature-level and softmax-level calibration differently affect feature extraction by theoretic and empirical analysis on the penultimate layer activation . 3 . We validate the task-agnostic nature of TAUFE through three tasks and show its performance advantage over the state-of-the-art method . 2 Background and Related Work . Negative Impact of Undesirable Features . DNNs tend to overly capture all available signals from training data even when they are not essential for solving a given task [ 2 , 3 ] . The occurrence of the undesirable features and their negative impact have been recently witnessed in various types of learning tasks . In image classification , a classification model often uses background or texture features as an undesirable shortcut for making a prediction instead of using the intrinsic shape of a target class [ 3 , 7 ] . In object detection , a detector model easily overfits to the background features for localizing target objects in a scene [ 10 , 11 ] . In video action recognition , a recognition model often relies on static cues in a single frame rather than temporal actions over consecutive frames [ 12 , 13 ] . In natural language processing ( NLP ) tasks , a language model often makes its predictions based on frequent but meaningless words instead of using semantically meaningful words [ 14 ] . Connection with Adversarial Examples . DNNs are easily deceived by adversarial perturbations of the inputs , so-called adversarial examples [ 15 ] . Differently from standard learning , the undesirable features are maliciously added and then make the model incur more errors . In addition , it is widely known that such adversarial perturbations are transferable even from different domains [ 16 ] ; that is , an adversarial attack can drastically degrade the generalization capability of the classifier without knowing its internals [ 16 ] . To remedy this problem , the use of OOD examples has gained great attention in that they enhance the robustness against the adversarial examples by preventing the model from overfitting to the undesirable features [ 8 ] . Removing Undesirable Feature Contribution . Numerous studies have attempted to prevent the overfitting to the undesirable features in standard supervised learning tasks . A typical way is debiasing , which removes the undesirable feature contribution based on the pre-defined bias for the target task . Geirhos et al . [ 7 ] took advantage of data augmentation techniques to generate de-biased examples from training data . Lee et al . [ 17 ] and Shetty et al . [ 18 ] synthesized de-biased examples by leveraging a generative model for image stylization or object removal . Wang et al . [ 19 ] quantified the local feature bias by using the neural gray-level co-occurrence matrix . Bahng et al . [ 1 ] proposed a framework that leverages a bias-characterizing model to remove pixel-level local undesirable features . This family of methods successfully removes the pre-defined bias from the undesirable features , but is not generalizable to other types of bias . Even worse , it is hard to identify the types of undesirable features in advance since they are not comprehensible even to a human . In this regard , motivated by the transferability of undesirable features in different domains , the usefulness of OOD examples for de-biasing was started to be discussed . Although the representative softmax calibrator , OAT [ 8 ] , does not need a pre-defined bias type , it suffers from the two limitations , lack of flexibility and feature entanglement . Many aspects , such as high generalizability and in-depth theoretical analysis , are yet to be explored . 3 Proposed Method : TAUFE . In this section , we first formulate the problem following the setup in the relevant literature [ 3 , 8 , 9 ] and then describe our method TAUFE . Moreover , we provide a theoretic analysis with empirical evidence on how the softmax-level and feature-level calibrations work differently at the penultimate layer in the perspective of feature extraction . 3.1 Problem Formulation . Let D = { xi , yi } Ni=1 be the target data obtained from a joint distribution over X ×Y , where X is the in-distribution example space and Y is the target label space . A DNN model consists of a general feature extractor fφ : X → Z ∈ Rd and a task-specific layer gθ : Z → Y . Then , the feature extractor is considered as a compound of d sub-feature extractors fφj such that fφ ( x ) = { fφ1 ( x ) , . . . , fφd ( x ) } where fφj : X → R. A feature is defined to be a function mapping from the example space X to a real number , and a set of the features is denoted by F = { f ∈ fφ : X → R } . We now formalize the desirableness of a feature . Let D̃ = { x̃i } Mi=1 be the out-of-distribution ( OOD ) data obtained from a distribution over the OOD example space X̃ . Then , undesirable and desirable features are defined by Definitions 3.1 and 3.2 , respectively . Definition 3.1 ( UNDESIRABLE FEATURE ) . For each example x̃ in the OOD data D̃ , we call a feature undesirable if it is highly correlated with at least one true label in expectation . Thus , the set Fundesirable ( ρ ) of undesirable features is defined by Fundesirable ( ρ ) = { f ∈ F : Ex̃∈D̃ [ maxy∈Y |Corr ( f ( x̃ ) , y ) | ] ≥ ρ } , ( 1 ) where Corr is a function to produce the correlation between two given inputs ( e.g. , R2 ) and ρ is a constant threshold . | · | is an absolute value function to convert a negative correlation into a positive one . Intuitively speaking , an undesirable feature influences the model ’ s decision-making even if it is not relevant to the target task ( i.e. , OOD examples ) . Definition 3.2 ( DESIRABLE FEATURE ) . For each example x and its corresponding label y in the in-distribution data D , we call a feature desirable if it is highly correlated with the true label in expectation and does not belong to Fundesirable ( ρ ) . Thus , the set Fdesirable ( ) of desirable features is defined by Fdesirable ( ) = { f ∈ F/Fundesirable ( ρ ) : E ( x , y ) ∈D [ |Corr ( f ( x ) , y ) | ] ≥ } , ( 2 ) where is a constant threshold ; Corr and ρ are the same as those for Definition 3.1 . Note that Definitions 3.1 and 3.2 are generally applicable to any supervised learning tasks including classification and regression . By these definitions , a feature vector obtained by the feature extractor could be a mixture of desirable and undesirable features . DNNs can totally memorize even undesirable features owing to their high expressive power , leading to the statistical bias in in-distribution training data . Therefore , the main challenge is to prevent the problem of biasing toward the undesirable features , which will be discussed in the next section .

Summary: Problem: DNN predictions may get influenced by the undesirable features that are irrelevant to the task at hand, which causes poor  generalization.  Solution: Existing work showed calibration on out-of-distribution (OOD) samples can remove the contribution of undesirable features in classification. The caveat is its applicability to feature extraction. The paper proposes TAUFE, a novel regularizer that deactivates all the undesirable features in OOD examples from the feature-extraction layer, thereby removing the dependency on the task specific softmax layer. Results: TAUFE outperforms state-of-the-art methods on the standard benchmarks for classification, regression and the mix of the two.