DISSECT: Disentangled Simultaneous Explanations via Concept Traversals

1 INTRODUCTION . Explanation of the internal inferences of deep learning models remains a challenging problem that many scholars deem promising for improving safety , evaluating fairness , and beyond [ 13 , 16 , 23 , 49 ] . Many efforts in explainability methods have been working towards providing solutions for this challenging problem . One way to categorize them is by the type of explanations , some post hoc techniques focusing on the importance of individual features , such as saliency maps [ 19 , 47 , 76 , 79 ] , some on importance of individual examples [ 33 , 34 , 39 , 87 ] , some on importance of highlevel concepts [ 35 ] . There has been active research into the shortcomings of explainability methods ( e.g . [ 1 , 30 , 61 , 70 , 84 ] ) . Scholars have also proposed tests to determine when attention can be used as an explanation [ 84 ] . While these methods focus on information that already exists in the data , either by weighting features or concepts in training examples or by selecting important training examples , recent progress in generative models [ 12 , 29 , 36 , 40 , 45 ] has lead to another family of explainability methods that provide explanations by generating new examples or features [ 14 , 32 , 68 , 73 ] . New examples or features can be used to generate counterfactuals [ 82 ] allowing users to ask : what if this sample were to be classified as the opposite class , and how would it differ ? This way of explaining mirrors the way humans reason , justify decisions [ 3 ] , and learn [ 4 , 6 , 83 ] . Additionally , users across visual , auditory , and sensor data domains found examples the most preferred means of explanations [ 31 ] . To illustrate the benefits of counterfactual explanations , consider a dermatology task where an explanation method is used to highlight why a certain sample is classified as benign/malignant ( Fig . 1 ) . Explanations like heatmaps , saliency maps , or segmentation masks only provide partial information . Such methods might hint at what is influential within the sample , potentially focusing on the lesion area . However , they can not show what kind of changes in color , texture , or inflammation could transform the input at hand from benign to malignant . Retrieval-based approaches that provide examples that show a concept are not enough for answering “ what-if ” questions either . A retrieval-based technique might show input samples of malignant skin lesions that have similarities to a benign lesion in patient A , but from a different patient B , potentially from another body part or even a different skin tone . Such examples do not show what this benign lesion in patient A would have to look like if it were classified as malignant instead . On the other hand , counterfactuals depict how to modify the input sample to change its class membership . A counterfactual explanation visualizes what a malignant tumor could look like in terms of potential color or texture , or inflammation changes on the skin . Better yet , multiple counterfactuals could highlight several different ways that changes in a skin lesion could reveal its malignancy . For a classifier that relies on surgical markings [ 86 ] as well as meaningful color/texture/size changes to make its decision , a single explanation might fail to reveal this flaw resulting from dependence on spurious features , but multiple distinct explanations shed light on this phenomenon . Multiple explanations are strongly preferred in several other domains , such as educational settings , knowledge discovery , and algorithmic recourse generation [ 17 , 54 ] . In this work , we propose a generation-based explainability method called DISSECT that generates Concept Traversals ( CTs ) –sequences of generated examples with increasing degrees of concepts ’ influence on a classifier ’ s decision . While current counterfactual generation techniques fail to satisfy the most consistently agreed-upon properties desired for an explainability method simultaneously [ 50 , 60 , 68 , 73 , 74 ] , DISSECT aims to overcome this challenge . CTs are generated by jointly training a generator , a discriminator , and a CT disentangler to produce examples that ( 1 ) express one distinct factor [ 50 ] at a time ; ( 2 ) are influential [ 73 , 74 ] to a classifier ’ s decision and are coupled to the classifier ’ s reasoning , due to joint training ; ( 3 ) are realistic [ 73 ] ; ( 4 ) preserve relevant information [ 50 , 60 , 68 ] ; and ( 5 ) are stable across similar inputs [ 21 , 50 , 60 ] . Compared to other approaches that require a human to identify concepts a priori and find samples representing them , e.g . [ 9 , 35 ] , DISSECT uncovers them automatically . We compare DISSECT with several baselines , some of which have been optimized for disentanglement , some used extensively for explanation , and some that fall in between . DISSECT is the only technique that performs well across all these dimensions . Other baselines either have a hard time with influence , lack fidelity , generate poor quality and unrealistic samples , or do not disentangle properly . We evaluate DISSECT using 3D Shapes [ 7 ] , CelebA [ 44 ] , and a new synthetic dataset inspired by the challenges faced in the dermatology domain . We show that DISSECT outperforms prior work in addressing all of these challenges . We also discuss this work ’ s applications to detect a classifier ’ s potential biases and identify spurious artifacts using simulated experiments . This paper makes five main contributions : 1 ) presents a novel counterfactual explanation approach that manifests several desirable properties outperforming baselines ; 2 ) demonstrates applications through experiments showcasing the effectiveness of this approach for detecting potential biases of a classifier ; 3 ) presents a set of explainability baselines inspired by approaches used for generative disentanglement ; 4 ) translates desired properties commonly referred to in the literature across forms of explanation into measurable quantities for bench-marking and evaluating counterfactual generation approaches ; 5 ) releases a new synthetic dermatology dataset inspired by real-world challenges . 2 RELATED WORK . Our method relates to the active research area of post hoc explainability methods . One way to categorize them is by explanation form . While met with criticisms [ 1 , 75 ] , many feature-based explainability methods are shown to be useful [ 47 , 79 ] , which assign a weight to each input feature to indicate their importance in classification [ 10 , 57 ] . Example-based methods are another popular category [ 33 , 34 , 39 , 87 ] that instead assign importance weights to individual examples . More recently , concept-based methods have emerged that attribute weights to concepts , i.e. , higher-level representations of features [ 22 , 25 , 35 ] such as “ long hair ” . Our work leverages recent progress in generative modeling , where the explanation is presented through several conditional generations [ 52 , 56 ] . Efforts for the “ discovery ” of concepts are also related to learning disentangled representations [ 12 , 29 , 35 , 36 ] , which has been shown to be challenging without additional supervision [ 45 ] . Recent findings suggest that weak supervision in the following forms could make the problem identifiable : how many factors of variation have changed [ 71 ] , using labels for validation [ 46 ] , or using sparse labels during training [ 46 ] . While some techniques like Conditional Subspace VAE ( CSVAE ) [ 38 ] began to look into conditional disentanglement by incorporating labels , their performance has not yet reached their unconditional counterparts . Our work uses a new form of weak supervision to improve upon existing methods : the posterior probability and gradient of the classifier-under-test . Many explainability methods have emerged that use generative models to modify existing examples or generate new examples [ 14 , 15 , 26 , 32 , 48 , 68 , 73 ] . Most of these efforts use pre-trained generators and generate a new example by moving along the generator ’ s embedding . Some aim to generate samples that would flip the classifier ’ s decision [ 15 , 32 ] , while others aim to modify particular attribution of the image and observe the classifier ’ s decision change [ 14 ] . More recent work allows the classifier ’ s predicted probabilities or gradients to flow through the generator during training [ 68 ] , and visualize the continuous progression of a key differential feature [ 73 ] . This progressive visualization has additional benefits , as Cai et al . [ 9 ] showed that pathologists could iterate more quickly using a tool that allowed them to progressively change the value of a concept , compared to purely using examples . Additionally , continuous exaggeration of a concept facilitated thinking more systematically about the specific features important for diagnosis . Most of these approaches assume that there is only one path that crosses the decision boundary , and they generate examples along that path . Our work leverages both counterfactual generation techniques and ideas from the disentanglement literature to generate diverse sets of explanations via multiple distinct paths influential to the classifier ’ s decision , called Concept Traversals ( CT ) . Each CT is made of a sequence of generated examples that express increasing degrees of a concept . 3 METHODS . Our key innovation is developing an approach to provide multiple potential counterfactuals . We show how to successfully incorporate the classifier ’ s signal while disentangling relevant concepts . We review existing generative models for building blocks of our approach , and follow Singla et al . [ 73 ] for DISSECT ’ s backbone . This choice is informed by the good conditional generation capability of [ 73 ] and being a strong interpretability baseline . We then introduce a novel component for unsupervised disentanglement of explanations . Multiple counterfactual explanations could reveal several different paths that flip the classification outcome and shed light on different biases of the classifier . However , a single explanation might fail to reveal such information . Disentangling distinct concepts without relying on predefined user labels reduces the burden on the user side . Additionally , it can surface concepts that the user might have missed otherwise . However , unsupervised disentanglement is particularly challenging [ 45 ] . To address this challenge , we design a disentangler network . The disentangler , along with the generator , indirectly enforces different concepts to be as distinct as possible . This section summarizes our final design choices . For more details about baselines , DISSECT , and ablations studies , see §A.1 , §A.2 , and §A.6 , respectively . Notation . Without loss of generality , we consider a binary classifier f : X → Y = { −1 , 1 } such that f ( x ) = p ( y = 1|x ) where x ∼ PX . We want to find K latent concepts that contribute to the decision-making of f . Given x , α ∈ [ 0 , 1 ] , and k , we want to generate an image , x̄ , by perturbing the latent concept k , such that the posterior probability f ( x̄ ) = α . In addition , when conditioning on x and a concept k , by changing α , we hope for a smooth change in the output image x̄ . This smoothness resembles slightly changing the degree of a concept . Putting it together , we desire a generic generation function that generates x̄ , defined as G ( x , α , k ; f ) : X × [ 0 , 1 ] × { 0 , 1 , . . . , K − 1 } → X , where f ( G ( x , α , k ; f ) ) ≈ α . The generation function G conditions on x and manipulates the concept k such that a monotonic change in the f ( G ( x , 0 , k ) ) , . . . , f ( G ( x , 1 , k ) ) is achieved .

Summary: This paper proposes a generative approach to model explanations, where explanations are composed of automatically generated counterfactual examples that differ by the observed magnitude of a given learned concept. Through a series of extensive experiments, the authors show that the proposed approach, DISSECT, can successfully disentangle useful and realistic concepts.  The main contributions of the paper is the novel approach that generates several counterfactual examples across a concept spectrum, and a new synthetic dermatology dataset that allows testing such methods where they matter most.