Debiasing Concept-based Explanations with Causal Analysis

1 INTRODUCTION . Explaining the predictions of neural networks through higher level concepts ( Kim et al. , 2018 ; Ghorbani et al. , 2019 ; Brocki & Chung , 2019 ; Hamidi-Haines et al. , 2018 ) enables model interpretation on data with complex manifold structure such as images . It also allows the use of domain knowledge during the explanation process . The concept-based explanation has been used for medical imaging ( Cai et al. , 2019 ) , breast cancer histopathology ( Graziani et al. , 2018 ) , cardiac MRIs ( Clough et al. , 2019 ) , and meteorology ( Sprague et al. , 2019 ) . When the set of concepts is carefully selected , we can estimate a model in which the discriminative information flow from the feature vectors x through the concept vectors c and reach the labels y . To this end , we train two models for prediction of the concept vectors from the features denoted by ĉ ( x ) and the labels from the predicted concept vector ŷ ( ĉ ) . This estimation process ensures that for each prediction we have the reasons for the prediction stated in terms of the predicted concept vector ĉ ( x ) . However , in reality , noise and confounding information ( due to e.g . non-discriminative context ) can influence both of the feature and concept vectors , resulting in confounded correlations between them . Figure 1 provides an evidence for noise and confounding in the CUB-200-2011 dataset ( Wah et al. , 2011 ) . We train two predictors for the concepts vectors based on features ĉ ( x ) and labels ĉ ( y ) and compare the Spearman correlation coefficients between their predictions and the true ordinal value of the concepts . Having concepts for which ĉ ( x ) is more accurate than ĉ ( y ) could be due to noise , or due to hidden variables independent of the labels that spuriously correlated c and x , leading to undesirable explanations that include confounding or noise . In this work , using the Concept Bottleneck Models ( CBM ) ( Koh et al. , 2020 ; Losch et al. , 2019 ) we demonstrate a method for removing the counfounding and noise ( debiasing ) the explanation with concept vectors and extend the results to Testing with Concept Activation Vectors ( TCAV ) ( Kim et al. , 2018 ) technique . We provide a new causal prior graph to account for the confounding information and concept completeness ( Yeh et al. , 2020 ) . We describe the challenges in estimation of our causal prior graph and propose a new learning procedure . Our estimation technique defines and predicts debiased concepts such that the predictive information of the features maximally flow through them . We show that using a two-stage regression technique from the instrumental variables literature , we can successfully remove the impact of the confounding and noise from the predicted concept vectors . Our proposed procedure has three steps : ( 1 ) debias the concept vectors using the labels , ( 2 ) predict the debiased concept vectors using the features , and ( 3 ) use the predict concept vectors in the second step to predict the labels . Optionally , we can also find the residual predictive information in the features that are not in the concepts . We validate the proposed method using a synthetic dataset and the CUB-200-2011 dataset . On the synthetic data , we have access to the ground truth and show that in the presence of confounding and noise , our debiasing procedure improves the accuracy of recovering the true concepts . On the CUB-200-2011 dataset , we use the RemOve And Retrain ( ROAR ) framework ( Hooker et al. , 2019 ) to show that our debiasing procedure ranks the concepts in the order of their explanation more accurately than the regular concept bottleneck models . We also show that we improve the accuracy of CBNs in the prediction of labels using our debiasing technique . Finally , using several examples , we also qualitatively show when the debasing helps improve the quality of concept-based explanations . 2 METHODOLOGY . Notations . We follow the notation of Goodfellow et al . ( 2016 ) and denote random vectors by bold font letters x and their values by bold symbols x . The notation p ( x ) is a probability measure on x and dp ( x = x ) is the infinitesimal probability mass at x = x . We use ŷ ( x ) to denote the the prediction of y given x . In the graphical models , we show the observed and unobserved variables using filled and hollow circles , respectively . Problem Statement . We assume that during the training phase , we are given triplets ( xi , ci , yi ) for i = 1 , . . . , n data points . In addition to the regular features x and labels y , we are given a human interpretable concepts vector c for each data point . Each element of the concept vector measures the degree of existence of the corresponding concept in the features . Thus , the concept vector typically have binary or ordinal values . Our goal is to learn to predict y as a function of x and use c for explaining the predictions . Performing in two steps , we first learn a function ĉ ( x ) and then learn another function ŷ ( ĉ ( x ) ) . The prediction ĉ ( x ) is the explanation for our prediction ŷ . During the test time , only the features are given and the prediction+explanation algorithm predicts both ŷ and ĉ . In this paper , we aim to remove the bias and noise components from the estimated concept vector ĉ such that it explains the reasons for prediction of the labels more accurately . To this end , we first need to propose a new causal prior graph that includes the potential unobserved confounders . 2.1 A NEW CAUSAL PRIOR GRAPH FOR CBMS . Figure 2a shows the ideal situation in explanation via high-level concepts . The generative model corresponding to Figure 2a states that for generating each feature xi we first randomly draw the label yi . Given the label , we draw the concepts ci . Given the concepts , we generate the features . The Ideal Graphical Model Estimation ! '' = $ ! % & = ' '' ! '' ! `` # ( a ) The ideal concepts Realistic Graphical Model ! ) # * `` ( b ) Our more realistic graph Realistic Graphical Model ! + ) # * , '' ( c ) The graph with d̂ ( y ) Figure 2 : ( a ) The ideal view of the causal relationships between the features x , concepts c , and labels y . ( b ) In a more realistic setting , the unobserved confounding variable u impacts both x and c. The shared information between x and y go through the discriminative part of the concepts d. We also model the completeness of the concepts via a direct edge from the features x to the labels y . ( c ) When we use d̂ ( y ) = E [ c|y ] in place of d and c , we eliminate the confounding link u→ c. hierarchy in this graph is from nodes with less detailed information ( labels ) to more detailed ones ( features , images ) . This model in Figure 2a is an explanation for the phenomenon in Figure 1 , because the noise in generation of the concepts allows the x—c edge to be stronger than the c—y edge . However , another ( non-mutually exclusive ) explanation for this phenomenon is the existence of hidden confounders u shown in Figure 2b . In this graphical model , u represents the confounders and d represents the unconfounded concepts . Note that we assume that the confounders u and labels y are independent when x and c are not observed . Another phenomenon captured in Figure 2b is the lack of concept completeness ( Yeh et al. , 2020 ) . It describes the situation when the features , compared to the concepts , have additional predictive information about the labels . The non-linear structural equations corresponding to the causal prior graph in Figure 2b are as follows d = f1 ( y ) + εd , ( 1 ) c = d + h ( u ) , ( 2 ) x = f2 ( u , d ) + f3 ( y ) + εx , ( 3 ) for some vector functions h , f1 , f2 , and f3 . We have εd ⊥⊥ y and u ⊥⊥ y . Our definition of d in Eq . ( 2 ) does not restrict u , because we simply attribute the difference between c and f1 ( y ) to a function of the latent confounder u and noise . Our causal prior graph in Figure 2b corresponds to a generative process in which to generate an observed triplet ( xi , ci , yi ) we first draw a label yi and a confounder ui vector independently . Then we draw the discriminative concepts di based on the label and generate the features xi jointly based on the concepts , label , and the confounder . Finally , we draw the observed concept vector ci based on the drawn concept and confounder vectors . Both causal graphs reflect our assumption that the direction of causality is from the labels to concepts and then to the features , y→ d→ x , to ensure that u and y are marginally independent in Figure 2b . This direction also correspond to moving from more abstract class labels to concepts to detailed features . During estimation , we fit the functions in the x→ d→ y direction , because finding the statistical strength of an edge does not depend on its direction . Estimation of the model in Figure 2b is challenging because there are two distinct paths for the information from the labels y to reach the features x . Our solution is to prioritize the bottleneck path and estimate the y → d → x , then estimate the residuals of the regression using the y → x direct path . Our two-stage estimation technique ensures that the predictive information of the features maximally flow through the concepts . In the next sections , we focus on the first phase and using a two-stage regression technique borrowed from the instrumental variables literature to eliminate the noise and confounding in estimation of the d→ x link . Algorithm 1 Debiased CBMs Require : Data tuples ( xi , ci , yi ) for i = 1 , . . . , n. 1 : Estimate a model d̂ ( y ) = E [ c|y ] using ( ci , yi ) pairs . 2 : Train a neural network as an estimator for pφ̂ ( d|x ) using ( xi , d̂i ) ) pairs . 3 : Use pairs ( xi , yi ) to estimate function gθ by fitting ∫ gθ ( d ) dpφ̂ ( d = d|xi ) to yi . 4 : Compute the debiased explanations E [ d|xi ] − 1n ∑n i=1E [ d|xi ] for i = 1 , . . . , n. 5 : return The CBM defined by ( pφ̂ , gθ ) and the debiased explanations . 2.2 INSTRUMENTAL VARIABLES . Background on two-stage regression . In causal inference , instrumental variables ( Stock , 2015 ; Pearl , 2009 ) denoted by z are commonly used to find the causal impact of a variable x on y when x and y are jointly influenced by an unobserved confounder u ( i.e. , x← u→ y ) . The key requirement is that z should be correlated with x but independent of the confounding variable u ( i.e . z→ x→ y and z ⊥⊥ u ) . The commonly used 2-stage least squares first regresses x in terms of z to obtain x̂ followed by regression of y in terms of x̂ . Because of independence between z and u , x̂ is also independent of u . Thus , in the second regression the confounding impact of u is eliminated . Our goal is to use the two-stage regression trick again to remove the confounding factors impacting features and concept vectors . The instrumental variable technique can be used for eliminating the biases due to the measurement errors ( Carroll et al. , 2006 ) . Two-Stage Regression for CBMs . In our causal graph in Figure 2b , the label y can be used for the study of the relationship between concepts d and features x . We predict d as a function of y and use it in place of the concepts in the concept bottleneck models . The graphical model corresponding to this procedure is shown in Figure 2c , where the link u → c is eliminated . In particular , given the independence relationship y ⊥⊥ u , we have d̂ ( y ) = E [ c|y ] ⊥⊥ h ( u ) . This is the basis for our debiasing method in the next section .

Summary: The focus of the work is on model interpretability using concept-based explanation. The authors consider the issue of concepts being correlated with confounding information in the features. They propose a causal graph for representing the system and use instrumental variable methods to remove the impact of unobserved confounders. The proposed method is evaluated on synthetic and real data.