Learning De-biased Representations with Biased Representations

1 INTRODUCTION . Most machine learning algorithms are trained and evaluated by randomly splitting a single source of data into training and test sets . Although this is a standard protocol , it is blind to a critical problem : the existence of dataset bias ( Torralba & Efros , 2011 ) . For instance , many frog images are taken in swamp scenes , but swamp itself is not a frog . Nonetheless , a neural network will exploit this bias ( i.e. , take “ shortcuts ” ) if it yields correct predictions for the majority of training examples . If bias is sufficient to achieve high accuracy , there is little motivation for models to learn the complexity of the intended task , despite its full capacity to do so . Consequently , a model that relies on bias will achieve high in-distribution accuracy , yet fail to generalise when the bias shifts . We tackle this “ cross-bias generalisation ” problem where a model does not exploit its full capacity due to the “ sufficiency ” of bias cues for prediction of the target label in training data . For example , language models make predictions based on the presence of certain words ( e.g. , “ not ” for “ contradiction ” ) ( Gururangan et al. , 2018 ) without much reasoning on the actual meaning of sentences , even if they are in principle capable of sophisticated reasoning . Similarly , convolutional neural networks ( CNNs ) achieve high accuracies on image classification by using local texture cues as shortcut , as opposed to more reliable global shape cues ( Geirhos et al. , 2019 ; Brendel & Bethge , 2019 ) . Existing methods attempt to remove a model ’ s dependency on bias by de-biasing the training data through data augmentation ( Geirhos et al. , 2019 ) or re-sampling tactics ( Li & Vasconcelos , 2019 ) . Others have introduced a pre-defined set of biases that a model is trained to be independent against ( Wang et al. , 2019 ) . These prior works assume that bias can easily be defined or quantified , but real-world biases often do not ( e.g. , texture bias above ) . To address this limitation , we propose a novel framework to train a de-biased representation by encouraging it to be “ different ” from a set of representations that are biased by design . Our insight is that biased representations can easily be obtained by utilising models of smaller capacity ( e.g. , bag of words for word bias and CNNs of small receptive fields for texture bias ) . Experiments show that our method is effective in reducing a model ’ s dependency on “ shortcuts ” in training data , as evidenced by improved accuracies in test data where the bias is either shifted or removed . ining sampletra ining sampletest p ( Y |S , B ) biasB si g n al S true conditional nal distribution p ( Btr ) = p ( Bte ) p ( Str , Btr , Y tr ) = p ( Ste , Bte , Y te ) cross-domain Btr ⊥⊥ Y tr p ( Str , Btr , Y tr ) = p ( Ste , Bte , Y te ) cross-bias p ( Str , Btr , Y tr ) = p ( Ste , Bte , Y te ) in-distribution biasB si g n al S biasB si gn a l S biasB si gn a l S target Y = colour Figure 1 : Learning scenarios . Different distributional gaps may take place between training and test distributions . Our work is addressing the cross-bias generalisation problem . Background colours on the right three figures indicate the decision boundaries of models trained on given training data . 2 PROBLEM DEFINITION . We provide a rigorous definition of our over-arching goal : overcoming the bias in models trained on biased data . We show that the problem we tackle is novel and realistic . 2.1 CROSS-BIAS GENERALISATION . We first define random variables , signal S and bias B as cues for the recognition of an input X as certain target variable Y . Signals S are the cues essential for the recognition of X as Y ; examples include the shape and skin patterns of frogs for frog image classification . Biases B ’ s , on the other hand , are cues not essential for the recognition but correlated with the target Y ; many frog images are taken in swamp scenes , so swamp scenes can be considered as B . A key property of B is that intervening on B should not change Y ; moving a frog from swamp to a dessert scene does not change the “ frogness ” . We assume that the true predictive distribution p ( Y |X ) factorises as∫ p ( Y |S , B ) p ( S , B|X ) , signifying the sufficiency of p ( S , B|X ) for recognition . Under this framework , three learning scenarios are identified depending on the change of relationship p ( S , B , Y ) across training and test distributions , p ( Str , Btr , Y tr ) and p ( Ste , Bte , Y te ) , respectively : in-distribution , cross-domain , and cross-bias generalisation . See Figure 1 for a summary . In-distribution . p ( Str , Btr , Y tr ) = p ( Ste , Bte , Y te ) . This is the standard learning setup utilised in many benchmarks by splitting data from a single source into training and test data at random . Cross-domain . p ( Str , Btr , Y tr ) 6= p ( Ste , Bte , Y te ) and furthermore p ( Btr ) 6= p ( Bte ) . B in this case is often referred to as “ domain ” . For example , training data consist of images with ( Y tr=frog , Btr=wilderness ) and ( Y tr=bird , Btr=wilderness ) , while test data contain ( Y te=frog , Bte=indoors ) and ( Y te=bird , Bte=indoors ) . This scenario is typically simulated by training and testing on different datasets ( Ben-David et al. , 2007 ) . Cross-bias . p ( Btr ) 6⊥⊥ p ( Y tr ) 1 and the dependency changes across training and test distributions : p ( Btr , Y tr ) 6= p ( Bte , Y te ) . We further assume that p ( Btr ) = p ( Bte ) , to clearly distinguish the scenario from the cross-domain generalisation . For example , training data only contain images of two types ( Y tr=frog , Btr=swamp ) and ( Y tr=bird , Btr=sky ) , but test data contain unusual class-bias combinations ( Y te=frog , Bte=sky ) and ( Y te=bird , Bte=swamp ) . Our work addresses this scenario . 2.2 EXISTING CROSS-BIAS GENERALISATION METHODS AND THEIR ASSUMPTIONS . Under cross-bias generalisation scenarios , the dependency p ( Btr ) 6⊥⊥ p ( Y tr ) makes bias B a viable cue for recognition . The model trained on such data becomes susceptible to interventions on B , limiting its generalisabililty when the bias is changed or removed in the test data . There exist prior approaches to this problem , but with different types and amounts of assumptions on B . We briefly recap the approaches based on the assumptions they require . In the next part §2.3 , we will define our novel problem setting that requires an assumption distinct from the ones in prior approaches . 1⊥⊥ and 6⊥⊥ denote independence and dependence , respectively . When an algorithm to disentangle bias B and signal S exists . Being able to disentangle B and S lets one collapse the feature space corresponding to B in both training and test data . A model trained on such normalised data then becomes free of biases . As ideal as it is , building a model to perfectly disentangle B and S is often unrealistic . When a generative algorithm or data collection procedure for p ( X|B ) exists . When additional examples can be supplied through p ( X|B ) , the training dataset itself can be de-biased , i.e. , B ⊥⊥ Y . For example , one can either collect or synthesise unusual images like frogs in sky and birds in swamp to balance out the bias . Such a data augmentation strategy is indeed a valid solution adopted by many prior studies ( Panda et al. , 2018 ; Geirhos et al. , 2019 ; Shetty et al. , 2019 ) . However , collecting unusual inputs can be expensive ( Peyre et al. , 2017 ) , and building a generative model with pre-defined bias types ( Geirhos et al. , 2019 ) may suffer from bias mis-specification and the lack of realism . When a predictive algorithm or ground truth for p ( B|X ) exists . Conversely , when one can tell the bias B for every input X , two approaches are feasible . ( 1 ) The first is a data re-weighting solution : we give greater weights on frogs in sky than frogs in swamps to even out the correlation in p ( B , Y ) ( Li et al. , 2018 ; Li & Vasconcelos , 2019 ) . ( 2 ) The second approach removes the dependency between the model predictions f ( X ) and the biasB . Many existing approaches for fairness in machine learning have proposed independence-based regularisers to encourage f ( X ) ⊥⊥ B ( Zemel et al. , 2013 ) or the conditional independence f ( X ) ⊥⊥ B |Y ( called the “ separation ” constraint , Hardt et al . ( 2016 ) ) . Other approaches have proposed to remove predictability of p ( B|X ) based on f ( X ) through domain adversarial losses ( Li & Vasconcelos , 2019 ; Wang et al. , 2019 ) or projection ( Wang et al. , 2019 ; Quadrianto et al. , 2019 ) . The knowledge on p ( B|X ) is provided in many realistic scenarios . For example , when the aim is to remove gender biases B in a job application process p ( Y |X ) , applicants ’ genders p ( B|X ) are supplied as ground truths . However , there exist cases when B is difficult to even be defined or quantified but can only be indirectly specified . We tackle such a scenario in the next part . 2.3 OUR SCENARIO : CAPTURING BIAS WITH A PARTICULAR SET OF MODELS . Under the cross-bias generalisation scenario , certain types of biases are not easily addressed by the above methods . Take texture bias as an example ( §1 , Geirhos et al . ( 2019 ) ) : ( 1 ) texture B and shape S can not easily be disentangled , ( 2 ) building a generative model p ( X|B ) or collecting unusual images is expensive , ( 3 ) building the predictive model p ( B|X ) for texture requires enumeration ( classifier ) or embedding ( regression ) of all possible textures , which is not feasible . However , slightly modifying the third assumption results in a problem setting that allows interesting application scenarios . Instead of assuming explicit knowledge on p ( B|X ) , we approximate B by defining a set of models G that are biased towards B by design . For texture biases , for example , we define G to be the set of convolutional neural network ( CNN ) architectures with ≤ 5 × 5 overall receptive fields . Then , any learned model g ∈ G can by design make predictions g ( x ) based on the patterns that can only be captured with small receptive fields ( i.e. , textures ) . More precisely , we define G to be a bias-characterising model class for the bias-signal pair ( B , S ) if for every possible joint distribution p ( B , X ) there exists a g ∈ G such that p ( B|X ) ≈ g ( X ) ( recall condition ) and every g ∈ G satisfies g ( X ) ⊥⊥ S |B ( precision condition ) . In practice , G may not necessarily include all biases and may also capture important signals ( i.e. , imperfect recall and precision ) . With this in mind , our framework is formulated so that f ( X ) does not ignore signals captured by G – we do not require G to be perfect . Further detail is provided in §2.3 . There exist many scenarios when such G can be defined , as we are given several evidence for the type of bias . For instance , action recognition models have been reported to rely heavily on static cues without learning temporal cues ( Li et al. , 2018 ; Li & Vasconcelos , 2019 ) ; though the actual bias may not precisely be any static cue , we can still regularise the 3D convolutional networks towards better generalisation across static cue biases by defining G to be the set of 2D convolutional architectures . It has been argued that visual question answering ( VQA ) models , too , rely overly on language biases rather than the visual cues ( e.g . without looking at the image , one knows the answer to what colour is the banana is yellow ) ( Agrawal et al. , 2018 ) . We can define G as the set of models looking at the language modality only . Entailment models are biased towards the presence of certain words ( e.g . when there are many nots , the sentence is contradictory ) , rather than really understanding the underlying meaning of sentences ( McCoy et al. , 2019 ; Niven & Kao , 2019 ) . We can design G to be the set of bag-of-words classifiers ( He et al. , 2019 ; Clark et al. , 2019 ) . Generally , these scenarios exemplify situations when the added architectural capacity is not fully utilised due to the sufficiency of simpler cues for solving the task in the given training set .

Summary: The paper describes a methodology for reducing model dependance on bias by specifying a model family of biases (i.e. conv nets with only 1x1 convs to model color biases), and then forcing independence between feature representations of the bias model and the a full model (i.e. conv nets with 3x3 convs to also model edges).