On The Quality Assurance Of Concept-Based Representations

1 INTRODUCTION . Addressing the lack of interpretability of deep neural networks ( DNNs ) has given rise to explainability methods , most common of which are feature importance methods ( Ribeiro et al. , 2016 ; Lundberg & Lee , 2017 ) that quantify the contribution of input features to certain predictions ( Bhatt et al. , 2020 ) . However , input features are not necessarily the most intuitive explanations , in particular when using low-level features such as pixels . Concept-based explainability ( Ghorbani et al. , 2019 ; Koh et al. , 2020 ; Yeh et al. , 2020 ; Ciravegna et al. , 2021 ) remedies this issue by constructing an explanation at a concept level , where concepts are considered high-level and semantically meaningful units of information commonly used by humans to explain their decisions . Furthermore , concepts allow users to improve a model ’ s performance via concept interventions , in which mispredicted concepts are corrected using expert knowledge ( Koh et al. , 2020 ) . In practice , what constitutes a concept is data-dependent , ranging from a group of pixels for image data ( Ghorbani et al. , 2019 ; Koh et al. , 2020 ; Yeh et al. , 2020 ) , to a sequence of words and sub-graphs for text and graph-based data , respectively ( Yeh et al. , 2020 ; Magister et al. , 2021 ) . While all of these definitions are specific to the data modality , they commonly refer to an intermediate representation of the input data that has certain properties . Summarising concepts as intermediate representations of the data makes them analogous to factors of variation in disentanglement learning , where the assumption is that there exists a generative process capable of producing a high-dimensional dataset using a finite number of factors ( Bengio et al. , 2013 ) . Such factors constitute a disentangled intermediate representation of the data with interpretability ( Bengio et al. , 2013 ; Higgins et al. , 2017 ) , fairness ( Creager et al. , 2019 ) , and predictive performance ( Locatello et al. , 2019 ; 2020b ) properties . The difference between Concept Learning ( CL ) and Disentanglement Learning ( DGL ) remains in that concepts in CL are often formed based on the supervision directly from concept labels or from a downstream task , whereas generative models ( e.g. , Variational Autoencoders ( VAEs ) ( Kingma & Welling , 2014 ; Higgins et al. , 2017 ) ) that serve as the basis of DGL are un-/semi-supervised and factors of variation are directly informed by the distribution of the input data . Since CL and DGL were developed independently , much of their connection remains unexplored . In particular , the metrics used to evaluate the quality of intermediate representations in each sub- field are not aligned , despite their overlapping goals . Metrics in the concept literature ( Koh et al. , 2020 ; Kazhdan et al. , 2020 ; Yeh et al. , 2020 ) are mainly concerned with the properties of learnt concepts w.r.t . the downstream task . On the other hand , given the lack of a downstream task , metrics in disentanglement literature ( Higgins et al. , 2017 ; Ridgeway & Mozer , 2018 ; Locatello et al. , 2019 ) are mainly concerned with the properties of the learnt representations , referred to as latent codes , w.r.t . the ground truth factors of variation . We argue that concepts/latent codes , as surrogate to inputs , need to have the following key properties : ( i ) They should correspond to semantically meaningful and coherent input sub-spaces ; ( ii ) They should preserve the amount of mutual information observed in ground truth concepts or factors of variation ( when available ) ; and ( iii ) They should capture sufficient statistics to predict the downstream task ( when available ) as well as raw inputs do . In this paper , we consider the properties ( ii ) and ( iii ) and make the following key contributions : - We unify the language and notation across CL and DGL by framing factors of variation and latent codes in DGL as ground truth concepts and concept representations in CL , respectively . - We introduce metrics for evaluating the quality of learnt concepts/codes in presence and absence of access to ground truth concepts/factors of variation and when concepts/codes are correlated . - We conduct a systematic empirical comparison of state-of-the-art methods from four families of methods : supervised CL , unsupervised CL , semi-supervised DGL , and unsupervised DGL . - We make the code used for our metrics , methods , and datasets available in an open-source library.1 2 BACKGROUND AND RELATED WORK . Notation In both CL and DGL the aim is to find a low-dimensional intermediate representation ĉ that explains the downstream task ( s ) in CL , or the data ’ s factors of variation in DGL . In CL , this low-dimensional representation corresponds to a matrix ĉ ∈ Ĉ ⊆ Rd×k in which the i-th column constitutes a d-dimensional representation of the i-th concept . As zero-padding can be used to ensure equal length across different concept representations , for notational simplicity we assume that all concepts use a d-dimensional vector as their representation . Under this view , elements in ĉ ( : ,i ) ∈ Rd are expected to have high values ( under some reasonable aggregation function ) if the i-th concept is considered to be activated for the input that generated this representation . For example , in the case where d = T and each concept can take up to T discrete values , ĉ ( : ,i ) ∈ [ 0 , 1 ] T can represent a probability distribution over all values that the i-th concept can take . As most CL methods assume d = 1 , for succinctness we use ĉi in place of ĉ ( : ,i ) when d = 1 . We adopt the same representation for latent codes in DGL and let ẑ ∈ Ẑ ⊆ Rd×k be a latent code matrix such that each dimension ẑ ( : ,i ) ( or a non-overlapping subset of dimensions ) encodes one , and only one , independent factor of variation zj . Nevertheless , note that in practice , d tends to be 1 for most DGL methods . Finally , for simplicity , we use ĉ to refer to both learnt concept representations and latent codes . Ground truth concepts and factors of variations are referred to as c ∈ C ⊆ Rk . In line with ( Koh et al. , 2020 ; Kazhdan et al. , 2020 ; Yeh et al. , 2020 ) we make use of : ( i ) a concept encoder function g : X ′ 7→ Ĉ that maps a transformation of the inputs x ∈ X ⊆ Rm , as performed by a function φ : X 7→ X ′ , to a concept intermediate representation ; and ( ii ) a label predictor function f : Ĉ 7→ Y that maps the concept representations to a downstream task ’ s set of labels y ∈ Y ⊆ RL . These two functions can be combined to give a set of predictions for sample x ∈ X by computing f ( g ( φ ( x ) ) ) . In DGL autoencoders , one can think of ( g ◦ φ ) ( · ) as the autoencoder ’ s encoder model and of the autoencoder ’ s decoder model as a function that approximates ( g ◦φ ) −1 ( · ) . Supervised concept learning In supervised CL , access to concept labels c ( i ) ∈ Nk , in addition to target labels y ( i ) ∈ RL , is assumed for inputs x ( i ) ∈ Rm . In other words , we have training data { ( x ( i ) , c ( i ) , y ( i ) ) } Ni=1 , where N is the number of training samples . In its most common form , supervised CL divides the prediction into two distinct steps of : ( i ) mapping an input sample to its concept representation via a concept encoder g ; and ( ii ) mapping a sample ’ s concept representation to its task labels via a label predictor f ( · ) . Together , these two functions constitute a Concept Bottleneck Model ( CBM ) ( Koh et al. , 2020 ) , because their final prediction relies on the input going through the bottleneck g ( φ ( x ) ) , which is trained to be component-wise aligned with c. 1Code will be released after review . Concept-based Model Extraction ( CME ) ( Kazhdan et al. , 2020 ) constructs a CBM from a pretrained model by building a non-trivial φ ( · ) mapping function using the model ’ s latent space . Using such a latent representation instead of raw inputs typically makes CME more data efficient than CBM ( Kazhdan et al. , 2021 ) . Similarly , Concept Whitening ( CW ) ( Chen et al. , 2020 ) constructs a CBM by introducing a pluggable batch normalization module whose activations ( g◦φ ) ( · ) are trained to be aligned with representative sets of binary concepts . It achieves this by forcing different feature maps of the normalization module to be decorrelated and orthogonal while incentivizing activations in a given axis to be high when its corresponding pre-defined concept is activated . Unsupervised Concept Learning Unlike supervised CL , in unsupervised CL concept annotations are not available and concepts are discovered in an unsupervised manner . Ghorbani et al . ( 2019 ) extract concepts from a trained classifier for image data . Images belonging to each class are first segmented with multiple resolutions . The segments are then clustered as examples of class concepts and their importance scores are measured using TCAV ( Kim et al. , 2018 ) . Unlike Ghorbani et al . ( 2019 ) , Completeness-aware Concept Discovery ( CCD ) ( Yeh et al. , 2020 ) is data modality agnostic and extracts class-independent concepts . CCD builds on TCAV to first extract a set of concept vectors { h ( i ) } ki=1 , each of which is a unit vector in X ′ . These vectors are then used to construct a concept representation g ( φ ( x ) ) = ĉ ∈ Ĉ ⊆ Rk by setting ĉi to TH ( 〈φ ( x ) , h ( i ) 〉 , β ) , the β-thresholded inner product of φ ( x ) and concept vector h ( i ) . When complete , a concept representation should contain sufficient statistics to obtain a high performance in the original model ’ s prediction task f ( φ ( x ) ) . If this is the case , then there must exist a mapping ψ : Ĉ 7→ X ′ that recovers φ ( x ) from g ( φ ( x ) ) . Thus , CCD constructs a set of concept vectors such that f ( ψ ( g ( φ ( x ) ) ) ) is able to achieve a similar task performance to that of f ( φ ( x ) ) . Notice that although f ( · ) is still a label predictor function here , it is not applied to the raw inputs or to their concept representation . Similarly , Self-Explainable Neural Networks ( SENNs ) ( Alvarez-Melis & Jaakkola , 2018 ) learn to produce concept-based explanations without explicit concept supervision through a robustness regularization term that encourages a differentiable model to locally act linearly . It proceeds by first learning a concept representation g ( x ) = ĉ ∈ Rk from the encoder of an autoencoding model and , second , generating a prediction f ( ĉ ) = G ( θ ( x ) 1ĉ1 , · · · , θ ( x ) k ĉk ) using an aggregation function G to weight the importance of each concept with a linear coefficient learnt through a differentiable model θ ( · ) . Concept weights θ ( x ) can then serve as an explanation for SENN ’ s predicted label . Disentanglement Learning Generative models ( e.g. , VAEs ( Kingma & Welling , 2014 ) ) used in DGL assume that data is generated from a set of independent factors of variation c ∈ C , sampled from factorable distribution p ( c ) = ∏ i p ( ci ) , such that a sample x is generated according to the conditional distribution p ( x|c ) . Thus , the goal of DGL is to find a function g ( · ) that maps inputs to a disentangled latent representation , such that a subset of non-overlapping dimensions of the latent representation corresponds to a unique factor of variation ci . In light of recent work showing the theoretical impossibility of learning disentangled representations in an unsupervised manner ( Locatello et al. , 2019 ) , a promising line of work suggests providing the inductive bias required to learn disentangled representations through weak supervision . In this work , we focus on DGL methods where weak supervision comes via pairs of observations whose corresponding ground truth factors of variation share at least one common element ( Locatello et al. , 2020a ) and contrast them against vanilla unsupervised DGL methods such as VAEs ( Kingma & Welling , 2014 ) and β-VAEs ( Higgins et al. , 2017 ) .

Summary: The authors consider the question of whether recent concept-based learning algorithms, as well disentangled representation learning algorithms, result in high-quality representations. In particular, they consider what high-quality should mean in terms of the relationship with ground truth concepts and the ability to make accurate predictions for a downstream task. To this end, they propose two main metrics for representations that are explicitly or implicitly encouraged to encode concepts: 1) a score that captures how well the learned representation preserves the relationships between concepts (which may be correlated), and 2) a score that captures how well concepts can be split into groups that are useful/useless for predicting particular label dimensions.