On The Quality Assurance Of Concept-Based Representations

1 INTRODUCTION . Addressing the lack of interpretability of deep neural networks ( DNNs ) has given rise to explainability methods , most common of which are feature importance methods ( Ribeiro et al. , 2016 ; Lundberg & Lee , 2017 ) that quantify the contribution of input features to certain predictions ( Bhatt et al. , 2020 ) . However , input features are not necessarily the most intuitive explanations , in particular when using low-level features such as pixels . Concept-based explainability ( Ghorbani et al. , 2019 ; Koh et al. , 2020 ; Yeh et al. , 2020 ; Ciravegna et al. , 2021 ) remedies this issue by constructing an explanation at a concept level , where concepts are considered high-level and semantically meaningful units of information commonly used by humans to explain their decisions . Furthermore , concepts allow users to improve a model ’ s performance via concept interventions , in which mispredicted concepts are corrected using expert knowledge ( Koh et al. , 2020 ) . In practice , what constitutes a concept is data-dependent , ranging from a group of pixels for image data ( Ghorbani et al. , 2019 ; Koh et al. , 2020 ; Yeh et al. , 2020 ) , to a sequence of words and sub-graphs for text and graph-based data , respectively ( Yeh et al. , 2020 ; Magister et al. , 2021 ) . While all of these definitions are specific to the data modality , they commonly refer to an intermediate representation of the input data that has certain properties . Summarising concepts as intermediate representations of the data makes them analogous to factors of variation in disentanglement learning , where the assumption is that there exists a generative process capable of producing a high-dimensional dataset using a finite number of factors ( Bengio et al. , 2013 ) . Such factors constitute a disentangled intermediate representation of the data with interpretability ( Bengio et al. , 2013 ; Higgins et al. , 2017 ) , fairness ( Creager et al. , 2019 ) , and predictive performance ( Locatello et al. , 2019 ; 2020b ) properties . The difference between Concept Learning ( CL ) and Disentanglement Learning ( DGL ) remains in that concepts in CL are often formed based on the supervision directly from concept labels or from a downstream task , whereas generative models ( e.g. , Variational Autoencoders ( VAEs ) ( Kingma & Welling , 2014 ; Higgins et al. , 2017 ) ) that serve as the basis of DGL are un-/semi-supervised and factors of variation are directly informed by the distribution of the input data . Since CL and DGL were developed independently , much of their connection remains unexplored . In particular , the metrics used to evaluate the quality of intermediate representations in each sub- field are not aligned , despite their overlapping goals . Metrics in the concept literature ( Koh et al. , 2020 ; Kazhdan et al. , 2020 ; Yeh et al. , 2020 ) are mainly concerned with the properties of learnt concepts w.r.t . the downstream task . On the other hand , given the lack of a downstream task , metrics in disentanglement literature ( Higgins et al. , 2017 ; Ridgeway & Mozer , 2018 ; Locatello et al. , 2019 ) are mainly concerned with the properties of the learnt representations , referred to as latent codes , w.r.t . the ground truth factors of variation . We argue that concepts/latent codes , as surrogate to inputs , need to have the following key properties : ( i ) They should correspond to semantically meaningful and coherent input sub-spaces ; ( ii ) They should preserve the amount of mutual information observed in ground truth concepts or factors of variation ( when available ) ; and ( iii ) They should capture sufficient statistics to predict the downstream task ( when available ) as well as raw inputs do . In this paper , we consider the properties ( ii ) and ( iii ) and make the following key contributions : - We unify the language and notation across CL and DGL by framing factors of variation and latent codes in DGL as ground truth concepts and concept representations in CL , respectively . - We introduce metrics for evaluating the quality of learnt concepts/codes in presence and absence of access to ground truth concepts/factors of variation and when concepts/codes are correlated . - We conduct a systematic empirical comparison of state-of-the-art methods from four families of methods : supervised CL , unsupervised CL , semi-supervised DGL , and unsupervised DGL . - We make the code used for our metrics , methods , and datasets available in an open-source library.1 2 BACKGROUND AND RELATED WORK . Notation In both CL and DGL the aim is to find a low-dimensional intermediate representation ĉ that explains the downstream task ( s ) in CL , or the data ’ s factors of variation in DGL . In CL , this low-dimensional representation corresponds to a matrix ĉ ∈ Ĉ ⊆ Rd×k in which the i-th column constitutes a d-dimensional representation of the i-th concept . As zero-padding can be used to ensure equal length across different concept representations , for notational simplicity we assume that all concepts use a d-dimensional vector as their representation . Under this view , elements in ĉ ( : ,i ) ∈ Rd are expected to have high values ( under some reasonable aggregation function ) if the i-th concept is considered to be activated for the input that generated this representation . For example , in the case where d = T and each concept can take up to T discrete values , ĉ ( : ,i ) ∈ [ 0 , 1 ] T can represent a probability distribution over all values that the i-th concept can take . As most CL methods assume d = 1 , for succinctness we use ĉi in place of ĉ ( : ,i ) when d = 1 . We adopt the same representation for latent codes in DGL and let ẑ ∈ Ẑ ⊆ Rd×k be a latent code matrix such that each dimension ẑ ( : ,i ) ( or a non-overlapping subset of dimensions ) encodes one , and only one , independent factor of variation zj . Nevertheless , note that in practice , d tends to be 1 for most DGL methods . Finally , for simplicity , we use ĉ to refer to both learnt concept representations and latent codes . Ground truth concepts and factors of variations are referred to as c ∈ C ⊆ Rk . In line with ( Koh et al. , 2020 ; Kazhdan et al. , 2020 ; Yeh et al. , 2020 ) we make use of : ( i ) a concept encoder function g : X ′ 7→ Ĉ that maps a transformation of the inputs x ∈ X ⊆ Rm , as performed by a function φ : X 7→ X ′ , to a concept intermediate representation ; and ( ii ) a label predictor function f : Ĉ 7→ Y that maps the concept representations to a downstream task ’ s set of labels y ∈ Y ⊆ RL . These two functions can be combined to give a set of predictions for sample x ∈ X by computing f ( g ( φ ( x ) ) ) . In DGL autoencoders , one can think of ( g ◦ φ ) ( · ) as the autoencoder ’ s encoder model and of the autoencoder ’ s decoder model as a function that approximates ( g ◦φ ) −1 ( · ) . Supervised concept learning In supervised CL , access to concept labels c ( i ) ∈ Nk , in addition to target labels y ( i ) ∈ RL , is assumed for inputs x ( i ) ∈ Rm . In other words , we have training data { ( x ( i ) , c ( i ) , y ( i ) ) } Ni=1 , where N is the number of training samples . In its most common form , supervised CL divides the prediction into two distinct steps of : ( i ) mapping an input sample to its concept representation via a concept encoder g ; and ( ii ) mapping a sample ’ s concept representation to its task labels via a label predictor f ( · ) . Together , these two functions constitute a Concept Bottleneck Model ( CBM ) ( Koh et al. , 2020 ) , because their final prediction relies on the input going through the bottleneck g ( φ ( x ) ) , which is trained to be component-wise aligned with c. 1Code will be released after review . Concept-based Model Extraction ( CME ) ( Kazhdan et al. , 2020 ) constructs a CBM from a pretrained model by building a non-trivial φ ( · ) mapping function using the model ’ s latent space . Using such a latent representation instead of raw inputs typically makes CME more data efficient than CBM ( Kazhdan et al. , 2021 ) . Similarly , Concept Whitening ( CW ) ( Chen et al. , 2020 ) constructs a CBM by introducing a pluggable batch normalization module whose activations ( g◦φ ) ( · ) are trained to be aligned with representative sets of binary concepts . It achieves this by forcing different feature maps of the normalization module to be decorrelated and orthogonal while incentivizing activations in a given axis to be high when its corresponding pre-defined concept is activated . Unsupervised Concept Learning Unlike supervised CL , in unsupervised CL concept annotations are not available and concepts are discovered in an unsupervised manner . Ghorbani et al . ( 2019 ) extract concepts from a trained classifier for image data . Images belonging to each class are first segmented with multiple resolutions . The segments are then clustered as examples of class concepts and their importance scores are measured using TCAV ( Kim et al. , 2018 ) . Unlike Ghorbani et al . ( 2019 ) , Completeness-aware Concept Discovery ( CCD ) ( Yeh et al. , 2020 ) is data modality agnostic and extracts class-independent concepts . CCD builds on TCAV to first extract a set of concept vectors { h ( i ) } ki=1 , each of which is a unit vector in X ′ . These vectors are then used to construct a concept representation g ( φ ( x ) ) = ĉ ∈ Ĉ ⊆ Rk by setting ĉi to TH ( 〈φ ( x ) , h ( i ) 〉 , β ) , the β-thresholded inner product of φ ( x ) and concept vector h ( i ) . When complete , a concept representation should contain sufficient statistics to obtain a high performance in the original model ’ s prediction task f ( φ ( x ) ) . If this is the case , then there must exist a mapping ψ : Ĉ 7→ X ′ that recovers φ ( x ) from g ( φ ( x ) ) . Thus , CCD constructs a set of concept vectors such that f ( ψ ( g ( φ ( x ) ) ) ) is able to achieve a similar task performance to that of f ( φ ( x ) ) . Notice that although f ( · ) is still a label predictor function here , it is not applied to the raw inputs or to their concept representation . Similarly , Self-Explainable Neural Networks ( SENNs ) ( Alvarez-Melis & Jaakkola , 2018 ) learn to produce concept-based explanations without explicit concept supervision through a robustness regularization term that encourages a differentiable model to locally act linearly . It proceeds by first learning a concept representation g ( x ) = ĉ ∈ Rk from the encoder of an autoencoding model and , second , generating a prediction f ( ĉ ) = G ( θ ( x ) 1ĉ1 , · · · , θ ( x ) k ĉk ) using an aggregation function G to weight the importance of each concept with a linear coefficient learnt through a differentiable model θ ( · ) . Concept weights θ ( x ) can then serve as an explanation for SENN ’ s predicted label . Disentanglement Learning Generative models ( e.g. , VAEs ( Kingma & Welling , 2014 ) ) used in DGL assume that data is generated from a set of independent factors of variation c ∈ C , sampled from factorable distribution p ( c ) = ∏ i p ( ci ) , such that a sample x is generated according to the conditional distribution p ( x|c ) . Thus , the goal of DGL is to find a function g ( · ) that maps inputs to a disentangled latent representation , such that a subset of non-overlapping dimensions of the latent representation corresponds to a unique factor of variation ci . In light of recent work showing the theoretical impossibility of learning disentangled representations in an unsupervised manner ( Locatello et al. , 2019 ) , a promising line of work suggests providing the inductive bias required to learn disentangled representations through weak supervision . In this work , we focus on DGL methods where weak supervision comes via pairs of observations whose corresponding ground truth factors of variation share at least one common element ( Locatello et al. , 2020a ) and contrast them against vanilla unsupervised DGL methods such as VAEs ( Kingma & Welling , 2014 ) and β-VAEs ( Higgins et al. , 2017 ) .

Summary: The authors have put decent effort to bring concept-based representation learning and disentanglement learning together under one umbrella in terms of the quality of generated concepts in presence as well as absence of ground truth concept labels. Some related metrics were proposed for evaluation of the quality of concepts for both these methods. Based on these studies, presented in the paper, some important conclusions were made based on requirements of concept supervision and their effects on final concept quality as well as the predictive performance of the model.