Semi-supervised counterfactual explanations

Counterfactual explanations for machine learning models are used to find minimal interventions to the feature values such that the model changes the prediction to a different output or a target output . A valid counterfactual explanation should have likely feature values . Here , we address the challenge of generating counterfactual explanations that lie in the same data distribution as that of the training data and more importantly , they belong to the target class distribution . This requirement has been addressed through the incorporation of auto-encoder reconstruction loss in the counterfactual search process . Connecting the output behavior of the classifier to the latent space of the auto-encoder has further improved the speed of the counterfactual search process and the interpretability of the resulting counterfactual explanations . Continuing this line of research , we show further improvement in the interpretability of counterfactual explanations when the auto-encoder is trained in a semi-supervised fashion with class tagged input data . We empirically evaluate our approach on several datasets and show considerable improvement in-terms of several metrics . 1 INTRODUCTION . Recently counterfactual explanations have gained popularity as tools of explainability for AI-enabled systems . A counterfactual explanation of a prediction describes the smallest change to the feature values that changes the prediction to a predefined output . A counterfactual explanation usually takes the form of a statement like , “ You were denied a loan because your annual income was 30 , 000 . If your income had been 45 , 000 , you would have been offered a loan ” . Counterfactual explanations are important in the context of AI-based decision-making systems because they provide the data subjects with meaningful explanations for a given decision and the necessary actions to receive a more favorable/desired decision in the future . Application of counterfactual explanations in the areas of financial risk mitigation , medical diagnosis , criminal profiling , and other sensitive socio-economic sectors is increasing and is highly desirable for bias reduction . Apart from the challenges of sparsity , feasibility , and actionability , the primary challenge for counterfactual explanations is their interpretability . Higher levels of interpretability lead to higher adoption of AI-enabled decision-making systems . Higher values of interpretability will improve the trust amongst data subjects on AI-enabled decisions . AI models used for decision making are typically black-box models , the reasons can either be the computational and mathematical complexities associated with the model or the proprietary nature of the technology . In this paper , we address the challenge of generating counterfactual explanations that are more likely and interpretable . A counterfactual explanation is interpretable if it lies within or close to the model ’ s training data distribution . This problem has been addressed by constraining the search for counterfactuals to lie in the training data distribution . This has been achieved by incorporating an auto-encoder reconstruction loss in the counterfactual search process . However , adhering to training data distribution is not sufficient for the counterfactual explanation to be likely . The counterfactual explanation should also belong to the feature distribution of its target class . To understand this , let us consider an example of predicting the risk of diabetes in individuals as high or low . A sparse counterfactual explanation to reduce the risk of diabetes might suggest a decrease in the body mass index ( BMI ) level for an individual while leaving other features unchanged . The model might predict a low risk based on this change and the features of this individual might still be in the data distribution of the model . However , they will not lie in the data distribution of individuals with low risk of diabetes because of other relevant features of low-risk individuals like glucose tolerance , serum insulin , diabetes pedigree , etc . To address this issue , authors in Van Looveren & Klaise ( 2019 ) proposed to connect the output behavior of the classifier to the latent space of the auto-encoder using prototypes . These prototypes guide the counterfactual search process in the latent space and improve the interpretability of the resulting counterfactual explanations . However , the auto-encoder latent space is still unaware of the class tag information . This is highly undesirable , especially when using a prototype guided search for counterfactual explanations on the latent space . In this paper , we propose to build a latent space that is aware of the class tag information through joint training of the auto-encoder and the classifier . Thus the counterfactual explanations generated will not only be faithful to the entire training data distribution but also faithful to the data distribution of the target class . We show that there are considerable improvements in interpretability , sparsity and proximity metrics can be achieved simultaneously , if the auto-encoder trained in a semi-supervised fashion with class tagged input data . Our approach does not rely on the availability of train data used for the black box classifier . It can be easily generalized to a post-hoc explanation method using the semi-supervised learning framework , which relies only on the predictions on the black box model . In the next section we present the related work . Then , in section 3 we present preliminary definitions and approaches necessary to introduce our approach . In section 4 , we present our approach and empirically evaluate it in section 5 . 2 RELATED WORK . Counterfactual analysis is a concept derived from from causal intervention analysis . Counterfactuals refer to model outputs corresponding to certain imaginary scenarios that we have not observed or can not observe . Recently Wachter et al . ( 2017 ) proposed the idea of model agnostic ( without opening the black box ) counterfactual explanations , through simultaneous minimization of the error between model prediction and the desired counterfactual and distance between original instance and their corresponding counterfactual . This idea has been extended for multiple scenarios by Mahajan et al . ( 2019 ) , Ustun et al . ( 2019 ) , Poyiadzi et al . ( 2020 ) based on the incorporation of feasibility constraints , actionability and diversity of counterfactuals . Authors in Mothilal et al . ( 2020 ) proposed a framework for generating diverse set of counterfactual explanations based on determinantal point processes . They argue that a wide range of suggested changes along with a proximity to the original input improves the chances those changes being adopted by data subjects . Causal constraints of our society do not allow the data subjects to reduce their age while increasing their educational qualifications . Such feasibility constraints were addressed by Mahajan et al . ( 2019 ) and Joshi et al . ( 2019 ) using a causal framework . Authors in Mahajan et al . ( 2019 ) addresses the feasibility of counterfactual explanations through causal relationship constraints amongst input features . They present a method that uses structural causal models to generate actionable counterfactuals . Authors in Joshi et al . ( 2019 ) propose to characterize data manifold and then provide an optimization framework to search for actionable counterfactual explanation on the data manifold via its latent representation . Authors in Poyiadzi et al . ( 2020 ) address the issues of feasibility and actionability through feasible paths , which are based on the shortest path distances defined via density-weighted metrics . An important aspect of counterfactual explanations is their interpretability . A counterfactual explanation is more interpretable if it lies within or close to the data distribution of the training data of the black box classifier . To address this issue Dhurandhar et al . ( 2018 ) proposed the use of auto-encoders to generate counterfactual explanations which are “ close ” to the data manifold . They proposed incorporation of an auto-encoder reconstruction loss in counterfactual search process to penalize counterfactual which are not true to the data manifold . This line of research was further extended by Van Looveren & Klaise ( 2019 ) , they proposed to connect the output behaviour of the classifier to the latent space of the auto-encoder using prototypes . These prototypes improved speed of counterfactual search process and the interpretability of the resulting counterfactual explanations . While Van Looveren & Klaise ( 2019 ) connects the output behaviour of the classifier to the latent space through prototypes , the latent space is still unaware of the class tag information . We propose to build a latent space which is aware of the class tag information through joint training of the auto-encoder and the classifier . Thus the counterfactual explanations generated will not only be faithful the entire training data distribution but also faithful the data distribution of the target class . In a post-hoc scenario where access to the training data is not guaranteed , we propose to use the input-output pair data of the black box classifier to jointly train the auto-encoder and classifier in the semi-supervised learning framework . Authors in Zhai & Zhang ( 2016 ) , Gogna et al . ( 2016 ) have explored the use semisupervised auto-encoders for sentiment analysis and analysis of biomedical signal analysis . Authors in Haiyan et al . ( 2015 ) propose a joint framework of representation and supervised learning which guarantees not only the semantics of the original data from representation learning but also fit the training data well via supervised learning . However , as far as our knowledge goes , semi-supervised learning has not been used to generate counterfactual explanations and we experimentally show that semi-supervised learning framework generates more interpretable counterfactual explanations . 3 PRELIMINARIES . Let D = { xi , yi } i=1 ... N be the supervised data set where xi 2 X is d-dimensional input feature space for a classifier and yi 2 Y = { 1 , 2 , . . . , ` } is the set of outputs for a classifier . Throughout this paper we assume the existence of a black box classifier h : X ! Y trained on D such that ŷ = h ( x ) = argmaxc2Y p ( y = c | x , D ) where p ( y = c | x , D ) is prediction score/probability for class c with an input x . Based on Wachter et al . ( 2017 ) , counterfactual explanations can be generated by trading off between prediction loss and sparsity . This is achieved by optimizing a linear combination of the prediction loss ( Lpred ) and loss of sparsity ( Lsparsity ) as L = c·Lpred+Lsparsity . Prediction loss typically measures the distance between current prediction and the target class , whereas sparsity loss function measures the perturbation from the initial instance x0 with class tag t0 . This approach generates counterfactual explanations which can reach their target class with a sparse perturbation to the initial instance . However , they need not necessarily respect the input data distribution of the classifier , hence , resulting in unreasonable values for xcfe . Authors in Dhurandhar et al . ( 2018 ) addressed this issue through incorporation of L2 reconstruction error for xcfe evaluated through an autoencoder ( AE ) trained on the input data X as LXrecon ( x ) = kx AEX ( x ) k22 where AEX represents the auto-encoder trained on entire training dataset X . The auto-encoder loss function LXrecon penalizes counterfactual explanations which do not lie within the data-distribution . However , Van Looveren & Klaise ( 2019 ) illustrated that incorporating LXrecon in L may result in counterfactual explanations which lie inside the input data-distribution but they may not be interpretable . To this end , Van Looveren & Klaise ( 2019 ) proposes addition of a prototype loss function LXproto to L to make xcfe more interpretable and improve the counterfactual search process through prototypes in the latent space of auto-encoder . LXproto is the L2 error between the latent encoding of x and cluster centroid of the target class in the latent space of the encoder defined as protot ( short for target prototype ) as LXproto ( x , protot ) = kENCX ( x ) prototk22 , where ENCX represents encoder part of the auto-encoder AEX and ENCX ( x ) represents the projection of x on to the latent space of the auto-encoder . Given a target t , the corresponding protot can be defined as protot = 1 K KX k=1 ENCX ( x t k ) ( 1 ) where xtk represent the input instances corresponding to the class t such that { ENCX ( xtk ) } k=1 , ... , K are the K nearest neighbors of ENCX ( x0 ) . For applications where target class t is not pre-defined , a suitable replacement for protot is evaluated by finding the nearest prototype protoj of class j 6= t0 to the encoding of x0 , given by j = argmini 6=t0 kENCX ( x0 ) protoik2 . Then prototype loss Lproto can be defined as Lproto ( x , protoj ) = kENCX ( x0 ) protojk22 . According to Van Looveren & Klaise ( 2019 ) the loss function Lproto explicitly guides the encoding of the counterfactual explanation to the target prototype ( or the nearest protoytpe protoi 6=t0 ) . Thus we have a loss function L given by L = c · Lpred + Lsparsity + · LXrecon + ✓ · LXproto ( 2 ) where c , and ✓ are hyper-parameters tuned globally for each data set . For detailed descriptions of these parameters and their impact on the counterfactual search , we refer the readers to Ltd . In this paper , we propose an alternate version of this loss function . The constituent loss functions LXrecon and LXproto are based on an auto-encoder trained in an unsupervised fashion . In the next section we motivate the use of an auto-encoder trained using class tagged data in a semi-supervised fashion . 4 SEMI-SUPERVISED COUNTERFACTUAL EXPLANATIONS x y h ( a ) Classification x z x X 1 X ( b ) Autoencoder x z y x D 1 D ⇠ ( c ) Jointly trained model For the supervised classification data set D = hX , Yi machine learning methods learn a classifier model h : X 7 ! Y ( see figure 1a ) . This process of learning involves minimizing a loss function of the form : Eentropy = P ` j PN i I ( yi = j ) ⇤ log ( p ( y = j | xi , D ) ) . Auto-encoder is a neural network framework which learns a latent space representation z 2 Z for input data x 2 X along with an invertible mapping ( 1X ) ( see figure 1b ) in an unsupervised fashion . The subscript X represents the unsupervised training of X and 1X only on the dataset X . The un-supervised learning framework tries to learn data compression using continuous map X , while minimizing the reconstruction loss : Eautoenc = q 1 N PN i |xi x0i| 2 . In this paper we consider only undercomplete autoencoders that produce a lower dimension representation ( Z ) of an high dimensional space ( X ) , while the decoder network ensures the reconstruction guarantee ( x ⇡ 1 ( ( x ) ) . A traditional undercomplete auto-encoder captures the correlation between the input features for the dimension reduction .

Summary: This paper continues an emerging line of research to find interpretable (post-hoc) counterfactual explanations of classifier predictions. While prior work has made advances in ensuring that resulting counterfactuals lie in the same data distribution as the original dataset by using auto-encoders, this paper provides a semi-supervised approach in an attempt to also ensure that they lie in the data distribution corresponding to the counterfactual label. The authors show the benefits of their approach on six datasets:  German Credit data, Adult Census data, MNIST, COMPAS, PIMA, and Breast Cancer Wisconsin data.