Learning De-biased Representations with Biased Representations

1 INTRODUCTION . Most machine learning algorithms are trained and evaluated by randomly splitting a single source of data into training and test sets . Although this is a standard protocol , it is blind to a critical problem : the existence of dataset bias ( Torralba & Efros , 2011 ) . For instance , many frog images are taken in swamp scenes , but swamp itself is not a frog . Nonetheless , a neural network will exploit this bias ( i.e. , take “ shortcuts ” ) if it yields correct predictions for the majority of training examples . If bias is sufficient to achieve high accuracy , there is little motivation for models to learn the complexity of the intended task , despite its full capacity to do so . Consequently , a model that relies on bias will achieve high in-distribution accuracy , yet fail to generalise when the bias shifts . We tackle this “ cross-bias generalisation ” problem where a model does not exploit its full capacity due to the “ sufficiency ” of bias cues for prediction of the target label in training data . For example , language models make predictions based on the presence of certain words ( e.g. , “ not ” for “ contradiction ” ) ( Gururangan et al. , 2018 ) without much reasoning on the actual meaning of sentences , even if they are in principle capable of sophisticated reasoning . Similarly , convolutional neural networks ( CNNs ) achieve high accuracies on image classification by using local texture cues as shortcut , as opposed to more reliable global shape cues ( Geirhos et al. , 2019 ; Brendel & Bethge , 2019 ) . Existing methods attempt to remove a model ’ s dependency on bias by de-biasing the training data through data augmentation ( Geirhos et al. , 2019 ) or re-sampling tactics ( Li & Vasconcelos , 2019 ) . Others have introduced a pre-defined set of biases that a model is trained to be independent against ( Wang et al. , 2019 ) . These prior works assume that bias can easily be defined or quantified , but real-world biases often do not ( e.g. , texture bias above ) . To address this limitation , we propose a novel framework to train a de-biased representation by encouraging it to be “ different ” from a set of representations that are biased by design . Our insight is that biased representations can easily be obtained by utilising models of smaller capacity ( e.g. , bag of words for word bias and CNNs of small receptive fields for texture bias ) . Experiments show that our method is effective in reducing a model ’ s dependency on “ shortcuts ” in training data , as evidenced by improved accuracies in test data where the bias is either shifted or removed . ining sampletra ining sampletest p ( Y |S , B ) biasB si g n al S true conditional nal distribution p ( Btr ) = p ( Bte ) p ( Str , Btr , Y tr ) = p ( Ste , Bte , Y te ) cross-domain Btr ⊥⊥ Y tr p ( Str , Btr , Y tr ) = p ( Ste , Bte , Y te ) cross-bias p ( Str , Btr , Y tr ) = p ( Ste , Bte , Y te ) in-distribution biasB si g n al S biasB si gn a l S biasB si gn a l S target Y = colour Figure 1 : Learning scenarios . Different distributional gaps may take place between training and test distributions . Our work is addressing the cross-bias generalisation problem . Background colours on the right three figures indicate the decision boundaries of models trained on given training data . 2 PROBLEM DEFINITION . We provide a rigorous definition of our over-arching goal : overcoming the bias in models trained on biased data . We show that the problem we tackle is novel and realistic . 2.1 CROSS-BIAS GENERALISATION . We first define random variables , signal S and bias B as cues for the recognition of an input X as certain target variable Y . Signals S are the cues essential for the recognition of X as Y ; examples include the shape and skin patterns of frogs for frog image classification . Biases B ’ s , on the other hand , are cues not essential for the recognition but correlated with the target Y ; many frog images are taken in swamp scenes , so swamp scenes can be considered as B . A key property of B is that intervening on B should not change Y ; moving a frog from swamp to a dessert scene does not change the “ frogness ” . We assume that the true predictive distribution p ( Y |X ) factorises as∫ p ( Y |S , B ) p ( S , B|X ) , signifying the sufficiency of p ( S , B|X ) for recognition . Under this framework , three learning scenarios are identified depending on the change of relationship p ( S , B , Y ) across training and test distributions , p ( Str , Btr , Y tr ) and p ( Ste , Bte , Y te ) , respectively : in-distribution , cross-domain , and cross-bias generalisation . See Figure 1 for a summary . In-distribution . p ( Str , Btr , Y tr ) = p ( Ste , Bte , Y te ) . This is the standard learning setup utilised in many benchmarks by splitting data from a single source into training and test data at random . Cross-domain . p ( Str , Btr , Y tr ) 6= p ( Ste , Bte , Y te ) and furthermore p ( Btr ) 6= p ( Bte ) . B in this case is often referred to as “ domain ” . For example , training data consist of images with ( Y tr=frog , Btr=wilderness ) and ( Y tr=bird , Btr=wilderness ) , while test data contain ( Y te=frog , Bte=indoors ) and ( Y te=bird , Bte=indoors ) . This scenario is typically simulated by training and testing on different datasets ( Ben-David et al. , 2007 ) . Cross-bias . p ( Btr ) 6⊥⊥ p ( Y tr ) 1 and the dependency changes across training and test distributions : p ( Btr , Y tr ) 6= p ( Bte , Y te ) . We further assume that p ( Btr ) = p ( Bte ) , to clearly distinguish the scenario from the cross-domain generalisation . For example , training data only contain images of two types ( Y tr=frog , Btr=swamp ) and ( Y tr=bird , Btr=sky ) , but test data contain unusual class-bias combinations ( Y te=frog , Bte=sky ) and ( Y te=bird , Bte=swamp ) . Our work addresses this scenario . 2.2 EXISTING CROSS-BIAS GENERALISATION METHODS AND THEIR ASSUMPTIONS . Under cross-bias generalisation scenarios , the dependency p ( Btr ) 6⊥⊥ p ( Y tr ) makes bias B a viable cue for recognition . The model trained on such data becomes susceptible to interventions on B , limiting its generalisabililty when the bias is changed or removed in the test data . There exist prior approaches to this problem , but with different types and amounts of assumptions on B . We briefly recap the approaches based on the assumptions they require . In the next part §2.3 , we will define our novel problem setting that requires an assumption distinct from the ones in prior approaches . 1⊥⊥ and 6⊥⊥ denote independence and dependence , respectively . When an algorithm to disentangle bias B and signal S exists . Being able to disentangle B and S lets one collapse the feature space corresponding to B in both training and test data . A model trained on such normalised data then becomes free of biases . As ideal as it is , building a model to perfectly disentangle B and S is often unrealistic . When a generative algorithm or data collection procedure for p ( X|B ) exists . When additional examples can be supplied through p ( X|B ) , the training dataset itself can be de-biased , i.e. , B ⊥⊥ Y . For example , one can either collect or synthesise unusual images like frogs in sky and birds in swamp to balance out the bias . Such a data augmentation strategy is indeed a valid solution adopted by many prior studies ( Panda et al. , 2018 ; Geirhos et al. , 2019 ; Shetty et al. , 2019 ) . However , collecting unusual inputs can be expensive ( Peyre et al. , 2017 ) , and building a generative model with pre-defined bias types ( Geirhos et al. , 2019 ) may suffer from bias mis-specification and the lack of realism . When a predictive algorithm or ground truth for p ( B|X ) exists . Conversely , when one can tell the bias B for every input X , two approaches are feasible . ( 1 ) The first is a data re-weighting solution : we give greater weights on frogs in sky than frogs in swamps to even out the correlation in p ( B , Y ) ( Li et al. , 2018 ; Li & Vasconcelos , 2019 ) . ( 2 ) The second approach removes the dependency between the model predictions f ( X ) and the biasB . Many existing approaches for fairness in machine learning have proposed independence-based regularisers to encourage f ( X ) ⊥⊥ B ( Zemel et al. , 2013 ) or the conditional independence f ( X ) ⊥⊥ B |Y ( called the “ separation ” constraint , Hardt et al . ( 2016 ) ) . Other approaches have proposed to remove predictability of p ( B|X ) based on f ( X ) through domain adversarial losses ( Li & Vasconcelos , 2019 ; Wang et al. , 2019 ) or projection ( Wang et al. , 2019 ; Quadrianto et al. , 2019 ) . The knowledge on p ( B|X ) is provided in many realistic scenarios . For example , when the aim is to remove gender biases B in a job application process p ( Y |X ) , applicants ’ genders p ( B|X ) are supplied as ground truths . However , there exist cases when B is difficult to even be defined or quantified but can only be indirectly specified . We tackle such a scenario in the next part . 2.3 OUR SCENARIO : CAPTURING BIAS WITH A PARTICULAR SET OF MODELS . Under the cross-bias generalisation scenario , certain types of biases are not easily addressed by the above methods . Take texture bias as an example ( §1 , Geirhos et al . ( 2019 ) ) : ( 1 ) texture B and shape S can not easily be disentangled , ( 2 ) building a generative model p ( X|B ) or collecting unusual images is expensive , ( 3 ) building the predictive model p ( B|X ) for texture requires enumeration ( classifier ) or embedding ( regression ) of all possible textures , which is not feasible . However , slightly modifying the third assumption results in a problem setting that allows interesting application scenarios . Instead of assuming explicit knowledge on p ( B|X ) , we approximate B by defining a set of models G that are biased towards B by design . For texture biases , for example , we define G to be the set of convolutional neural network ( CNN ) architectures with ≤ 5 × 5 overall receptive fields . Then , any learned model g ∈ G can by design make predictions g ( x ) based on the patterns that can only be captured with small receptive fields ( i.e. , textures ) . More precisely , we define G to be a bias-characterising model class for the bias-signal pair ( B , S ) if for every possible joint distribution p ( B , X ) there exists a g ∈ G such that p ( B|X ) ≈ g ( X ) ( recall condition ) and every g ∈ G satisfies g ( X ) ⊥⊥ S |B ( precision condition ) . In practice , G may not necessarily include all biases and may also capture important signals ( i.e. , imperfect recall and precision ) . With this in mind , our framework is formulated so that f ( X ) does not ignore signals captured by G – we do not require G to be perfect . Further detail is provided in §2.3 . There exist many scenarios when such G can be defined , as we are given several evidence for the type of bias . For instance , action recognition models have been reported to rely heavily on static cues without learning temporal cues ( Li et al. , 2018 ; Li & Vasconcelos , 2019 ) ; though the actual bias may not precisely be any static cue , we can still regularise the 3D convolutional networks towards better generalisation across static cue biases by defining G to be the set of 2D convolutional architectures . It has been argued that visual question answering ( VQA ) models , too , rely overly on language biases rather than the visual cues ( e.g . without looking at the image , one knows the answer to what colour is the banana is yellow ) ( Agrawal et al. , 2018 ) . We can define G as the set of models looking at the language modality only . Entailment models are biased towards the presence of certain words ( e.g . when there are many nots , the sentence is contradictory ) , rather than really understanding the underlying meaning of sentences ( McCoy et al. , 2019 ; Niven & Kao , 2019 ) . We can design G to be the set of bag-of-words classifiers ( He et al. , 2019 ; Clark et al. , 2019 ) . Generally , these scenarios exemplify situations when the added architectural capacity is not fully utilised due to the sufficiency of simpler cues for solving the task in the given training set .

Summary: This manuscript discusses the problem of bias shortcut employed by many machine learning algorithms (due to dataset problems or underlying effects of any algorithmic bias within an application). The authors argue that models tend to underutilize their capacities to extract non-bias signals when bias shortcuts provide enough cues for recognition. This is an interesting and important aspect of machine learning models neglected by many recent developments.