Debiasing Concept-based Explanations with Causal Analysis

1 INTRODUCTION . Explaining the predictions of neural networks through higher level concepts ( Kim et al. , 2018 ; Ghorbani et al. , 2019 ; Brocki & Chung , 2019 ; Hamidi-Haines et al. , 2018 ) enables model interpretation on data with complex manifold structure such as images . It also allows the use of domain knowledge during the explanation process . The concept-based explanation has been used for medical imaging ( Cai et al. , 2019 ) , breast cancer histopathology ( Graziani et al. , 2018 ) , cardiac MRIs ( Clough et al. , 2019 ) , and meteorology ( Sprague et al. , 2019 ) . When the set of concepts is carefully selected , we can estimate a model in which the discriminative information flow from the feature vectors x through the concept vectors c and reach the labels y . To this end , we train two models for prediction of the concept vectors from the features denoted by ĉ ( x ) and the labels from the predicted concept vector ŷ ( ĉ ) . This estimation process ensures that for each prediction we have the reasons for the prediction stated in terms of the predicted concept vector ĉ ( x ) . However , in reality , noise and confounding information ( due to e.g . non-discriminative context ) can influence both of the feature and concept vectors , resulting in confounded correlations between them . Figure 1 provides an evidence for noise and confounding in the CUB-200-2011 dataset ( Wah et al. , 2011 ) . We train two predictors for the concepts vectors based on features ĉ ( x ) and labels ĉ ( y ) and compare the Spearman correlation coefficients between their predictions and the true ordinal value of the concepts . Having concepts for which ĉ ( x ) is more accurate than ĉ ( y ) could be due to noise , or due to hidden variables independent of the labels that spuriously correlated c and x , leading to undesirable explanations that include confounding or noise . In this work , using the Concept Bottleneck Models ( CBM ) ( Koh et al. , 2020 ; Losch et al. , 2019 ) we demonstrate a method for removing the counfounding and noise ( debiasing ) the explanation with concept vectors and extend the results to Testing with Concept Activation Vectors ( TCAV ) ( Kim et al. , 2018 ) technique . We provide a new causal prior graph to account for the confounding information and concept completeness ( Yeh et al. , 2020 ) . We describe the challenges in estimation of our causal prior graph and propose a new learning procedure . Our estimation technique defines and predicts debiased concepts such that the predictive information of the features maximally flow through them . We show that using a two-stage regression technique from the instrumental variables literature , we can successfully remove the impact of the confounding and noise from the predicted concept vectors . Our proposed procedure has three steps : ( 1 ) debias the concept vectors using the labels , ( 2 ) predict the debiased concept vectors using the features , and ( 3 ) use the predict concept vectors in the second step to predict the labels . Optionally , we can also find the residual predictive information in the features that are not in the concepts . We validate the proposed method using a synthetic dataset and the CUB-200-2011 dataset . On the synthetic data , we have access to the ground truth and show that in the presence of confounding and noise , our debiasing procedure improves the accuracy of recovering the true concepts . On the CUB-200-2011 dataset , we use the RemOve And Retrain ( ROAR ) framework ( Hooker et al. , 2019 ) to show that our debiasing procedure ranks the concepts in the order of their explanation more accurately than the regular concept bottleneck models . We also show that we improve the accuracy of CBNs in the prediction of labels using our debiasing technique . Finally , using several examples , we also qualitatively show when the debasing helps improve the quality of concept-based explanations . 2 METHODOLOGY . Notations . We follow the notation of Goodfellow et al . ( 2016 ) and denote random vectors by bold font letters x and their values by bold symbols x . The notation p ( x ) is a probability measure on x and dp ( x = x ) is the infinitesimal probability mass at x = x . We use ŷ ( x ) to denote the the prediction of y given x . In the graphical models , we show the observed and unobserved variables using filled and hollow circles , respectively . Problem Statement . We assume that during the training phase , we are given triplets ( xi , ci , yi ) for i = 1 , . . . , n data points . In addition to the regular features x and labels y , we are given a human interpretable concepts vector c for each data point . Each element of the concept vector measures the degree of existence of the corresponding concept in the features . Thus , the concept vector typically have binary or ordinal values . Our goal is to learn to predict y as a function of x and use c for explaining the predictions . Performing in two steps , we first learn a function ĉ ( x ) and then learn another function ŷ ( ĉ ( x ) ) . The prediction ĉ ( x ) is the explanation for our prediction ŷ . During the test time , only the features are given and the prediction+explanation algorithm predicts both ŷ and ĉ . In this paper , we aim to remove the bias and noise components from the estimated concept vector ĉ such that it explains the reasons for prediction of the labels more accurately . To this end , we first need to propose a new causal prior graph that includes the potential unobserved confounders . 2.1 A NEW CAUSAL PRIOR GRAPH FOR CBMS . Figure 2a shows the ideal situation in explanation via high-level concepts . The generative model corresponding to Figure 2a states that for generating each feature xi we first randomly draw the label yi . Given the label , we draw the concepts ci . Given the concepts , we generate the features . The Ideal Graphical Model Estimation ! '' = $ ! % & = ' '' ! '' ! `` # ( a ) The ideal concepts Realistic Graphical Model ! ) # * `` ( b ) Our more realistic graph Realistic Graphical Model ! + ) # * , '' ( c ) The graph with d̂ ( y ) Figure 2 : ( a ) The ideal view of the causal relationships between the features x , concepts c , and labels y . ( b ) In a more realistic setting , the unobserved confounding variable u impacts both x and c. The shared information between x and y go through the discriminative part of the concepts d. We also model the completeness of the concepts via a direct edge from the features x to the labels y . ( c ) When we use d̂ ( y ) = E [ c|y ] in place of d and c , we eliminate the confounding link u→ c. hierarchy in this graph is from nodes with less detailed information ( labels ) to more detailed ones ( features , images ) . This model in Figure 2a is an explanation for the phenomenon in Figure 1 , because the noise in generation of the concepts allows the x—c edge to be stronger than the c—y edge . However , another ( non-mutually exclusive ) explanation for this phenomenon is the existence of hidden confounders u shown in Figure 2b . In this graphical model , u represents the confounders and d represents the unconfounded concepts . Note that we assume that the confounders u and labels y are independent when x and c are not observed . Another phenomenon captured in Figure 2b is the lack of concept completeness ( Yeh et al. , 2020 ) . It describes the situation when the features , compared to the concepts , have additional predictive information about the labels . The non-linear structural equations corresponding to the causal prior graph in Figure 2b are as follows d = f1 ( y ) + εd , ( 1 ) c = d + h ( u ) , ( 2 ) x = f2 ( u , d ) + f3 ( y ) + εx , ( 3 ) for some vector functions h , f1 , f2 , and f3 . We have εd ⊥⊥ y and u ⊥⊥ y . Our definition of d in Eq . ( 2 ) does not restrict u , because we simply attribute the difference between c and f1 ( y ) to a function of the latent confounder u and noise . Our causal prior graph in Figure 2b corresponds to a generative process in which to generate an observed triplet ( xi , ci , yi ) we first draw a label yi and a confounder ui vector independently . Then we draw the discriminative concepts di based on the label and generate the features xi jointly based on the concepts , label , and the confounder . Finally , we draw the observed concept vector ci based on the drawn concept and confounder vectors . Both causal graphs reflect our assumption that the direction of causality is from the labels to concepts and then to the features , y→ d→ x , to ensure that u and y are marginally independent in Figure 2b . This direction also correspond to moving from more abstract class labels to concepts to detailed features . During estimation , we fit the functions in the x→ d→ y direction , because finding the statistical strength of an edge does not depend on its direction . Estimation of the model in Figure 2b is challenging because there are two distinct paths for the information from the labels y to reach the features x . Our solution is to prioritize the bottleneck path and estimate the y → d → x , then estimate the residuals of the regression using the y → x direct path . Our two-stage estimation technique ensures that the predictive information of the features maximally flow through the concepts . In the next sections , we focus on the first phase and using a two-stage regression technique borrowed from the instrumental variables literature to eliminate the noise and confounding in estimation of the d→ x link . Algorithm 1 Debiased CBMs Require : Data tuples ( xi , ci , yi ) for i = 1 , . . . , n. 1 : Estimate a model d̂ ( y ) = E [ c|y ] using ( ci , yi ) pairs . 2 : Train a neural network as an estimator for pφ̂ ( d|x ) using ( xi , d̂i ) ) pairs . 3 : Use pairs ( xi , yi ) to estimate function gθ by fitting ∫ gθ ( d ) dpφ̂ ( d = d|xi ) to yi . 4 : Compute the debiased explanations E [ d|xi ] − 1n ∑n i=1E [ d|xi ] for i = 1 , . . . , n. 5 : return The CBM defined by ( pφ̂ , gθ ) and the debiased explanations . 2.2 INSTRUMENTAL VARIABLES . Background on two-stage regression . In causal inference , instrumental variables ( Stock , 2015 ; Pearl , 2009 ) denoted by z are commonly used to find the causal impact of a variable x on y when x and y are jointly influenced by an unobserved confounder u ( i.e. , x← u→ y ) . The key requirement is that z should be correlated with x but independent of the confounding variable u ( i.e . z→ x→ y and z ⊥⊥ u ) . The commonly used 2-stage least squares first regresses x in terms of z to obtain x̂ followed by regression of y in terms of x̂ . Because of independence between z and u , x̂ is also independent of u . Thus , in the second regression the confounding impact of u is eliminated . Our goal is to use the two-stage regression trick again to remove the confounding factors impacting features and concept vectors . The instrumental variable technique can be used for eliminating the biases due to the measurement errors ( Carroll et al. , 2006 ) . Two-Stage Regression for CBMs . In our causal graph in Figure 2b , the label y can be used for the study of the relationship between concepts d and features x . We predict d as a function of y and use it in place of the concepts in the concept bottleneck models . The graphical model corresponding to this procedure is shown in Figure 2c , where the link u → c is eliminated . In particular , given the independence relationship y ⊥⊥ u , we have d̂ ( y ) = E [ c|y ] ⊥⊥ h ( u ) . This is the basis for our debiasing method in the next section .

Summary: Abstract: The paper proposed a way to learn unbiased (debiased) concept-based explainable models in the presence of unobserved confounders by the use of labels as instrumental variables. The proposed algorithm has 3 main steps: (1) regresses concept labels from the final labels (2) replace the original concepts with the learnt concepts and learn a model of debiased concepts as a function of features (3) predict label as a function of debiased concepts. The authors show in the experimental section that their training method captures the most salient concepts using the ROAR (Remove and Retrain) evaluation framework much better than the vanilla (non de-biased) approach.