Year: 2021

Abstract:
Recent work has focused on concept-based explanations, where deep learning models are explained in terms of high-level units of information, referred to as concepts. In parallel, the field of disentanglement learning has explored the related notion of finding underlying factors of variation in the data that have interpretability properties. Despite their overlapping purpose, the metrics to evaluate the quality of concepts and factors of variation in the two fields are not aligned, hindering a systematic comparison. In this paper, we consider factors of variation as concepts and thus unify the notations in concept and disentanglement learning. Next, we propose metrics for evaluating the quality of concept representations in both approaches, in the presence and in the absence of ground truth concept labels. Via our proposed metrics, we benchmark state-of-the-art methods from both families, and find that, contrary to common assumption, supervision alone may not be sufficient for quality assurance of concept representations. In light of this, we propose a set of guidelines to determine the impact that different degrees of supervision may have on the quality of learnt concept representations.

heading: 1 INTRODUCTION
text: Addressing the lack of interpretability of deep neural networks (DNNs) has given rise to explainability methods, most common of which are feature importance methods (Ribeiro et al., 2016; Lundberg & Lee, 2017) that quantify the contribution of input features to certain predictions (Bhatt et al., 2020). However, input features are not necessarily the most intuitive explanations, in particular when using low-level features such as pixels. Concept-based explainability (Ghorbani et al., 2019; Koh et al., 2020; Yeh et al., 2020; Ciravegna et al., 2021) remedies this issue by constructing an explanation at a concept level, where concepts are considered high-level and semantically meaningful units of information commonly used by humans to explain their decisions. Furthermore, concepts allow users to improve a model’s performance via concept interventions, in which mispredicted concepts are corrected using expert knowledge (Koh et al., 2020).
In practice, what constitutes a concept is data-dependent, ranging from a group of pixels for image data (Ghorbani et al., 2019; Koh et al., 2020; Yeh et al., 2020), to a sequence of words and sub-graphs for text and graph-based data, respectively (Yeh et al., 2020; Magister et al., 2021). While all of these definitions are specific to the data modality, they commonly refer to an intermediate representation of the input data that has certain properties. Summarising concepts as intermediate representations of the data makes them analogous to factors of variation in disentanglement learning, where the assumption is that there exists a generative process capable of producing a high-dimensional dataset using a finite number of factors (Bengio et al., 2013). Such factors constitute a disentangled intermediate representation of the data with interpretability (Bengio et al., 2013; Higgins et al., 2017), fairness (Creager et al., 2019), and predictive performance (Locatello et al., 2019; 2020b) properties. The difference between Concept Learning (CL) and Disentanglement Learning (DGL) remains in that concepts in CL are often formed based on the supervision directly from concept labels or from a downstream task, whereas generative models (e.g., Variational Autoencoders (VAEs) (Kingma & Welling, 2014; Higgins et al., 2017)) that serve as the basis of DGL are un-/semi-supervised and factors of variation are directly informed by the distribution of the input data.
Since CL and DGL were developed independently, much of their connection remains unexplored. In particular, the metrics used to evaluate the quality of intermediate representations in each sub-
field are not aligned, despite their overlapping goals. Metrics in the concept literature (Koh et al., 2020; Kazhdan et al., 2020; Yeh et al., 2020) are mainly concerned with the properties of learnt concepts w.r.t. the downstream task. On the other hand, given the lack of a downstream task, metrics in disentanglement literature (Higgins et al., 2017; Ridgeway & Mozer, 2018; Locatello et al., 2019) are mainly concerned with the properties of the learnt representations, referred to as latent codes, w.r.t. the ground truth factors of variation. We argue that concepts/latent codes, as surrogate to inputs, need to have the following key properties: (i) They should correspond to semantically meaningful and coherent input sub-spaces; (ii) They should preserve the amount of mutual information observed in ground truth concepts or factors of variation (when available); and (iii) They should capture sufficient statistics to predict the downstream task (when available) as well as raw inputs do.
In this paper, we consider the properties (ii) and (iii) and make the following key contributions:
- We unify the language and notation across CL and DGL by framing factors of variation and latent codes in DGL as ground truth concepts and concept representations in CL, respectively. - We introduce metrics for evaluating the quality of learnt concepts/codes in presence and absence of access to ground truth concepts/factors of variation and when concepts/codes are correlated. - We conduct a systematic empirical comparison of state-of-the-art methods from four families of methods: supervised CL, unsupervised CL, semi-supervised DGL, and unsupervised DGL. - We make the code used for our metrics, methods, and datasets available in an open-source library.1

heading: 2 BACKGROUND AND RELATED WORK
text: Notation In both CL and DGL the aim is to find a low-dimensional intermediate representation ĉ that explains the downstream task(s) in CL, or the data’s factors of variation in DGL. In CL, this low-dimensional representation corresponds to a matrix ĉ ∈ Ĉ ⊆ Rd×k in which the i-th column constitutes a d-dimensional representation of the i-th concept. As zero-padding can be used to ensure equal length across different concept representations, for notational simplicity we assume that all concepts use a d-dimensional vector as their representation. Under this view, elements in ĉ(:,i) ∈ Rd are expected to have high values (under some reasonable aggregation function) if the i-th concept is considered to be activated for the input that generated this representation. For example, in the case where d = T and each concept can take up to T discrete values, ĉ(:,i) ∈ [0, 1]T can represent a probability distribution over all values that the i-th concept can take. As most CL methods assume d = 1, for succinctness we use ĉi in place of ĉ(:,i) when d = 1.
We adopt the same representation for latent codes in DGL and let ẑ ∈ Ẑ ⊆ Rd×k be a latent code matrix such that each dimension ẑ(:,i) (or a non-overlapping subset of dimensions) encodes one, and only one, independent factor of variation zj . Nevertheless, note that in practice, d tends to be 1 for most DGL methods. Finally, for simplicity, we use ĉ to refer to both learnt concept representations and latent codes. Ground truth concepts and factors of variations are referred to as c ∈ C ⊆ Rk. In line with (Koh et al., 2020; Kazhdan et al., 2020; Yeh et al., 2020) we make use of: (i) a concept encoder function g : X ′ 7→ Ĉ that maps a transformation of the inputs x ∈ X ⊆ Rm, as performed by a function φ : X 7→ X ′, to a concept intermediate representation; and (ii) a label predictor function f : Ĉ 7→ Y that maps the concept representations to a downstream task’s set of labels y ∈ Y ⊆ RL. These two functions can be combined to give a set of predictions for sample x ∈ X by computing f(g(φ(x))). In DGL autoencoders, one can think of (g ◦ φ)(·) as the autoencoder’s encoder model and of the autoencoder’s decoder model as a function that approximates (g ◦φ)−1(·).
Supervised concept learning In supervised CL, access to concept labels c(i) ∈ Nk, in addition to target labels y(i) ∈ RL, is assumed for inputs x(i) ∈ Rm. In other words, we have training data {(x(i), c(i),y(i))}Ni=1, where N is the number of training samples. In its most common form, supervised CL divides the prediction into two distinct steps of: (i) mapping an input sample to its concept representation via a concept encoder g; and (ii) mapping a sample’s concept representation to its task labels via a label predictor f(·). Together, these two functions constitute a Concept Bottleneck Model (CBM) (Koh et al., 2020), because their final prediction relies on the input going through the bottleneck g(φ(x)), which is trained to be component-wise aligned with c.
1Code will be released after review.
Concept-based Model Extraction (CME) (Kazhdan et al., 2020) constructs a CBM from a pretrained model by building a non-trivial φ(·) mapping function using the model’s latent space. Using such a latent representation instead of raw inputs typically makes CME more data efficient than CBM (Kazhdan et al., 2021). Similarly, Concept Whitening (CW) (Chen et al., 2020) constructs a CBM by introducing a pluggable batch normalization module whose activations (g◦φ)(·) are trained to be aligned with representative sets of binary concepts. It achieves this by forcing different feature maps of the normalization module to be decorrelated and orthogonal while incentivizing activations in a given axis to be high when its corresponding pre-defined concept is activated.
Unsupervised Concept Learning Unlike supervised CL, in unsupervised CL concept annotations are not available and concepts are discovered in an unsupervised manner. Ghorbani et al. (2019) extract concepts from a trained classifier for image data. Images belonging to each class are first segmented with multiple resolutions. The segments are then clustered as examples of class concepts and their importance scores are measured using TCAV (Kim et al., 2018).
Unlike Ghorbani et al. (2019), Completeness-aware Concept Discovery (CCD) (Yeh et al., 2020) is data modality agnostic and extracts class-independent concepts. CCD builds on TCAV to first extract a set of concept vectors {h(i)}ki=1, each of which is a unit vector in X ′. These vectors are then used to construct a concept representation g(φ(x)) = ĉ ∈ Ĉ ⊆ Rk by setting ĉi to TH (〈φ(x),h(i)〉, β), the β-thresholded inner product of φ(x) and concept vector h(i). When complete, a concept representation should contain sufficient statistics to obtain a high performance in the original model’s prediction task f(φ(x)). If this is the case, then there must exist a mapping ψ : Ĉ 7→ X ′ that recovers φ(x) from g(φ(x)). Thus, CCD constructs a set of concept vectors such that f(ψ(g(φ(x)))) is able to achieve a similar task performance to that of f(φ(x)). Notice that although f(·) is still a label predictor function here, it is not applied to the raw inputs or to their concept representation.
Similarly, Self-Explainable Neural Networks (SENNs) (Alvarez-Melis & Jaakkola, 2018) learn to produce concept-based explanations without explicit concept supervision through a robustness regularization term that encourages a differentiable model to locally act linearly. It proceeds by first learning a concept representation g(x) = ĉ ∈ Rk from the encoder of an autoencoding model and, second, generating a prediction f(ĉ) = G(θ(x)1ĉ1, · · · , θ(x)k ĉk) using an aggregation function G to weight the importance of each concept with a linear coefficient learnt through a differentiable model θ(·). Concept weights θ(x) can then serve as an explanation for SENN’s predicted label. Disentanglement Learning Generative models (e.g., VAEs (Kingma & Welling, 2014)) used in DGL assume that data is generated from a set of independent factors of variation c ∈ C, sampled from factorable distribution p(c) = ∏ i p(ci), such that a sample x is generated according to the conditional distribution p(x|c). Thus, the goal of DGL is to find a function g(·) that maps inputs to a disentangled latent representation, such that a subset of non-overlapping dimensions of the latent representation corresponds to a unique factor of variation ci.
In light of recent work showing the theoretical impossibility of learning disentangled representations in an unsupervised manner (Locatello et al., 2019), a promising line of work suggests providing the inductive bias required to learn disentangled representations through weak supervision. In this work, we focus on DGL methods where weak supervision comes via pairs of observations whose corresponding ground truth factors of variation share at least one common element (Locatello et al., 2020a) and contrast them against vanilla unsupervised DGL methods such as VAEs (Kingma & Welling, 2014) and β-VAEs (Higgins et al., 2017).

heading: 3 QUALITY ASSURANCE OF CONCEPTS
text: Given the varying degree of supervision in the approaches above, the quality of their concept representations is expected to vary. The quality of learnt concepts w.r.t. downstream tasks has commonly been studied using predictive performance metrics such as accuracy and AUC. Similarly, there exist several metrics for measuring the properties of learnt concepts w.r.t. the ground truth ones, either focusing on the overall set of learnt concepts or focusing on individual concepts (see Appendix 6.1 for a summary). Nevertheless, these metrics have two main shortcomings.
The first shortcoming is assuming that concepts are independent, implying that if a concept representation encodes information beyond that of the ground truth concept it is aligned with, it counts as
leakage (Mahinpei et al., 2021). Impurities caused by leakage can have far-reaching consequences in terms of human interventions, e.g., perturbations of an impure concept may impact other concepts unintentionally. While we agree with the danger of impurities, we argue that concepts in the real-world often have dependencies, and full disentanglement is not realistic. As such, leakage as described above is not necessarily undesirable when it reflects the correlations found in ground-truth concepts. To support this, we demonstrate the lack of inter-concept independence in a real-world dataset, the Caltech-UCSD Birds-200-2011 (CUB) (Wah et al., 2011a), in Appendix 6.2.
The second shortcoming is the assumption that one has access to ground truth concepts, which are used to verify the properties of the extracted ones. Previous work has shown that access to ground truth concepts is limited given the difficulty of annotating samples with concept labels (Raghavan et al., 2006). The rarity of real-world datasets with such annotations is evidence for this claim.
To address these shortcomings, we first propose the oracle impurity score, a metric that measures the quality of concept representations when one has access to discrete ground truth concepts labels which may be correlated. We then propose the niching purity score and niching impurity score, two efficient metrics which, in conjunction, measure the quality of concept representations when one has only access to a classification task’s labels.

heading: 3.1 ORACLE IMPURITY
text: To circumvent the need for inter-concept independence, we take inspiration from (Mahinpei et al., 2021), where they informally measure concept impurity as how predictive a CBM-generated concept probability is for the ground truth value of other independent concepts. If the pre-defined concepts are independent, then the inter-concept predictive performance should be no better than random. To generalise this assumption beyond independent concepts, we first measure the predictability of ground truth concepts w.r.t. one another. Then we measure the predictability of learnt concepts w.r.t. the ground truth ones. The divergence between the former and the latter acts as an impurity metric, measuring the amount of undesired information that is encoded by the learnt concepts. In order to formally introduce our metric, we begin by formalising a purity matrix.
Definition 1 (Purity Matrix). Given concept representations Γ̂ = {ĉ(i) ∈ Rd×k}ni=1, and corresponding discrete ground truth concept annotations Γ = {c(i) ∈ Nk}ni=1, assume that Γ̂ and Γ are aligned element-wise: for all l ∈ {1, · · · , k}, the l-th concept representation of ĉ(i) encodes for the same concept as the l-th concept label in c(i). The Purity Matrix of Γ̂ given ground truth labels Γ is defined as a matrix π ( Γ̂,Γ ) ∈ [0, 1]k×k whose entries are given by:
π ( Γ̂,Γ ) (i,j) := AUC ({( ψj(ĉ (1) (:,i) ) , c (1) j ) , · · · , ( ψj(ĉ (n) (:,i)), c (n) j )}) where ψj(·) is a classifier (e.g., an MLP) trained to map the i-th concept’s d-dimensional vector representation ĉ(:,i) to a probability distribution over all the values that concept j may take.
The (i, j)-th entry of π ( Γ̂,Γ ) contains the AUC when predicting the ground truth value of concept j given the i-th concept representation. The diagonal entries of this matrix show how good a concept representation is at predicting its corresponding ground truth value, while the off-diagonal entries show how good such a representation is at predicting the ground truth labels of other concepts. Intuitively, one can think of the (i, j)-th entry of this matrix as a proxy of the mutual information between the i-th concept representation and the j-th ground truth concept. While in principle several other methods could have been used to estimate this mutual information, we choose AUC primarily for two reasons: (1) it is computationally tractable and (2) it allows the construction of a metric that can be easily bounded (as discussed below). Furthermore, while in this work, as in the majority of the existing CL literature (Koh et al., 2020; Chen et al., 2020; Yeh et al., 2020), we focus on concepts that are binary in nature, notice that one can trivially extend our definition to be applicable to multivariate concepts by using the mean one-vs-all AUC across all possible labels a ground truth concept may take. Details on how this matrix is computed in practice can be found in Appendix 6.3. This definition of a purity matrix allows us to construct a metric for quantifying the amount of unnecessary information, or impurity, that a concept encoder encodes.
Definition 2 (Oracle Impurity Score (OIS)). Let g : X 7→ Ĉ ⊆ Rd×k be a concept encoder and let ΓX := {x(i) ∈ X}ni=0 and Γ := {c(i) ∈ Nk}ni=0 be ordered sets of testing samples and
their corresponding concept annotations, respectively. If, for any ordered set A we define g(A) as g(A) := {g(a) | a ∈ A}, then the OIS of g(·) given data (ΓX ,Γ) is defined as:
OIS(g,ΓX ,Γ) := 2 ∣∣∣∣∣∣π(g(ΓX),Γ)− π(Γ,Γ)∣∣∣∣∣∣ F
k where we use ||A||F to represent the Frobenius norm of matrix A.
Intuitively, the OIS measures the total deviation of an encoder’s purity matrix with the purity matrix obtained from using the ground truth concept labels only (i.e., the “oracle matrix”). We opt to measure this divergence using the Frobenius norm of their difference, instead of other similarity metrics, in order to obtain a bounded output which can be easily interpreted. More specifically, since each entry in the difference ( π ( g(ΓX),Γ ) − π ( Γ,Γ )) can be at most 1/2, the upper bound
of the Frobenius norm of this difference is k/2. Therefore, our metric includes a normalisation factor 2/k which guarantees that it is in [0, 1]. This allows interpreting an OIS of 1 as a complete misalignment between π ( Γ,Γ ) and π ( g(ΓX),Γ ) (i.e., the i-th concept representation can predict all other concept labels except its corresponding one even when concepts are independent). An impurity score of 0, on the other hand, represents perfect alignment between the two purity matrices (i.e., the i-th concept representation does not encode any unnecessary information for predicting concept i).

heading: 3.2 NICHING-BASED METRICS
text: In nature, a niche is defined as a resource-constrained subspace of the environment that can support different types of life (Darwin, 1859). By analogy, in the context of neural networks each layer can be seen as a computational “environment” with a finite amount of computational resources (i.e., the layer parameters). A “niche of neurons” can be defined as a subspace of the layer (i.e., the “environment”) that is critical for the accurate prediction (i.e., “the survival”) of a specific downstream task (i.e., the “species living in the niche”).
Inspired by the theory of niching, concept bottleneck sensitivity, and sparsity (Dimanov & Jamnik, 2018; Barbiero et al., 2021), we propose two metrics that measure the quality of concept representations in the absence of access to ground truth concept labels and in the presence of task-specific labels. Our metrics quantify how task-label-separable a concept representation is. For this, they first identify a set of concepts that are important for a task label, and then see if such a set, referred to as the label’s concept niche, is complete enough to predict the label. The predictive performance of a set of concepts outside the concept niche of a label identifies impurity and unnecessary leakage in the concept representation. The existence of niches is not an assumption for the metrics to be viable in practise, but rather an observation in many real-world data (see Appendix 6.6 for an example). We start by describing a concept nicher, a function that defines the dependency between a single concept and a single task label. Definition 3 (Concept nicher). Given a label predic-
tor f : Ĉ 7→ Y , where Ĉ ⊆ Rd×k and Y ⊆ RL, we define a concept nicher as a Boolean-valued function νf : {1, · · · , k} × {1, · · · , L} 7→ {0, 1} that returns νf (i, j) = 1 if the j-th output dimension of f(·) has a “strong” dependency, as determined by a reasonable procedure, on the i-th concept representation ĉ(:,i) and 0 otherwise.
The above definition is affected by how a dependency is defined. One efficient way of measuring the dependency, and the way that we proceed, is to use the absolute Pearson correlation, denoted as ρ. We call such an instantiation a concept-correlation nicher (CCorrN) and define it as:
CCorrNf (i, j) := ∣∣ρ({ĉ(1)(:,i), · · · , ĉ(N)(:,i)}, {f(ĉ(1))j , · · · , f(ĉ(N))j})∣∣ > β
where β ∈ [0, 1] is a user-defined threshold. If ĉ(:,i) is not a scalar representation (i.e., d > 1), we use the maximum absolute correlation coefficient between all entries in ĉ(:,i) and the target label f(c)j as a representative correlation coefficient for the entire representation ĉ(:,i). For all experiments we use β = 0.2. A study of the impact of β on CCorrN is in Appendix 6.5.
For each task label, the concept niche defines the set of concepts on which the label depends (i.e., where the concept nicher outputs 1 for the label). One can intuitively see this in Figure 1(b), where the red blocks show the concepts that the concept nicher has identified for y1 and y2 and Figure 1(c) shows the niches. Formally we define a concept niche as: Definition 4 (Concept niche). The concept niche Nj(νf ) for label j, determined by label predictor f(·) and concept nicher νf (·), is defined as Nj(νf ) := { i | i ∈ {1, · · · , k} and νf (i, j) = 1 } .
Abusing notation, we let ¬Nj(νf ) := {1, · · · , k} \ Nj(νf ) be the complement of the set Nj(νf ) (see Figure 1(d)). Although the label predictor f can be any classifier, in our experiments we use a ReLU MLP with hidden layer sizes {20, 20}. In this setup, we let f |Nj(νf ) be the MLP resulting from pruning all connections in f except those between concept representations in the niche Nj(νf ) and the j-th output of f .
We define the Niche Purity Score (NPS) and Niche Impurity Score (NIS) as measures of the predictive capacity of a niche and its complement w.r.t. a specific classification label for a given label predictor function, respectively. A good concept representation has a simultaneous high NPS and low NIS for each label, as high values in the latter score are a symptom of unnecessary leakage in concept representation.
Definition 5 (Niche Purity Score). Given label predictor f : Ĉ 7→ Y , concept nicher νf , concept representations Γ̂ = {ĉ(j)}nj=1, and corresponding ground truth classification labels ΓY = {y(j)}nj=1, the NPS of the i-th output of f(·) is defined as NPSi(f, νf ) := AUC ( {(f |Ni(νf ) ( ĉ (j) (:,Ni(νf )) ) , y (j) i )}nj=1 ) . The NPS of f , defined as NPS(f, νf ) :=∑L
i=1 NPSi(f, νf )/L, is the mean NPS across all labels. Definition 6 (Niche Impurity Score). The NIS of concept niche Nj(νf ) is the NPS of ¬Nj(νf ).
Essentially, the NPS metric measures the quality of niche Ni(νf ) via the AUC of predicting the ground truth label of the i-th label using only the concept representations corresponding to niche Ni(νf ). A NPS score of 1 (its maximum value), thus indicates a perfect niche purity for the i-th output of function f(·) while a worst-case-scenario NPS of 1/2 indicates that a niche is unable to do better than random when predicting its corresponding label. Similarly, the NIS metric measures the impurity of niche Ni(νf ) by measuring the AUC of predicting the ground truth label of the i-th label using concept representations outside the niche corresponding to the i-th label. Therefore, a NIS score of 1/2 indicates the best-case scenario impurity as concepts outside of niche Ni(νf ) hold no useful information to predict the i-th label. In contrast, a worst-case NIS score 1 corresponds to complete impurity as the i-th label can be perfectly predicted from concepts that are irrelevant to it.

heading: 4 EXPERIMENTS
text: Methods, Model Selection, and Training We compare the quality of concept representations in various methods using our metrics. CBM (Koh et al., 2020) is selected from the supervised CL family due to its fundamental role in CL. We focus on jointly trained CBMs, where the task-specific loss and the concept prediction loss are minimised jointly. We also add CW (Chen et al., 2020) from this family as it directly treats decorrelated concept representations. From the unsupervised CL family, CCD (Yeh et al., 2020) is selected due to its data agnostic nature and SENN (AlvarezMelis & Jaakkola, 2018) due to its particular mix of ideas from both DGL and CL literature. From the DGL family, we consider two weakly supervised methods, Adaptive Group Variational Autoencoder (Ada-GVAE) and Adaptive Multilevel Variational Autoencoder (Ada-MLVAE) (Locatello et al., 2020a), as well as two unsupervised methods, namely vanilla Variational AutoEncoders (VAE) (Kingma & Welling, 2014) and β-VAE (with β = 10) (Higgins et al., 2017). For each method and metric, we report the average metric values and 95% confidence intervals obtained from 5 different random seeds. Given that the number of concepts in unsupervised CL and DGL approaches is not known upfront, we allow an extra variation, where the number of learnt concepts is twice the number
of ground truth ones. We observe that model capacity, in particular the encoder’s capacity, impacts the quality of learnt concepts (see Appendix 6.7). Therefore, to minimise the effects of inductive biases arising from the model architecture and training hyperparameters, we use the same architecture and training setups for all methods within the same dataset whenever possible. We include details on training and architecture hyperparameters in Appendix 6.8.2.
Datasets In order to have datasets that are compatible with both CL and DGL, we construct datasets whose samples are fully described by a vector of ground truth generative factors. Moreover, to simulate real-world scenarios, we work with tasks with a varying degree of dependencies in their concept annotations. To achieve this, we first design a parametric binary-class dataset TabularToy(δ), a variation of the tabular dataset proposed by (Mahinpei et al., 2021). We also construct two multiclass image-based parametric datasets: dSprites(λ) and 3dshapes(λ), based on dSprites (Matthey et al., 2017) and 3dshapes (Burgess & Kim, 2018) datasets, respectively. They consist of 3D samples generated from a vector consisting of k = 5 and k = 6 ground truth factors of variation, respectively. Both datasets have one binary concept annotation per factor of variation. Parameters δ ∈ [0, 1] and λ ∈ {0, · · · , k − 1} control the degree of concept inter-dependencies during generation (a value of 0 represents inter-concept independence while higher values represent stronger inter-concept dependencies). Details on how dependencies are introduced, as well as the nature of the individual concepts and task labels for all our datasets, can be found in the Appendix 6.8.1.

heading: 4.1 RESULTS
text: Overall set of learnt concepts is equally predictive of the downstream task across all surveyed methods. The quality of the overall set of learnt concepts w.r.t. downstream tasks, measured by its mean one-vs-all task AUC, does not vary considerably across methods and stays close to 100%. This result replicates the predictive power of the raw inputs and thus shows that the grouped concept representations faithfully capture task-related information in the input space. Plots for predictive AUC of raw inputs and the overall set of learnt concepts can be found in Appendix 6.9.
Supervision may not lead to overall set of learnt concepts being more predictive of individual ground truth concepts compared with no supervision (Figure 2). Since CBM and CW benefit from explicit concept supervision, we expect their learnt representations to predict the ground truth concepts well. While this observation holds in CBM and CW when one uses CW’s entire feature maps as concept representations (shown as “CW Feature Map” in plots), it does not hold for “CW MaxPool-Mean,” where one reduces the CW module’s feature maps to scalar concept scores using a max pool followed by a mean reduction (as seen in Yeh et al. (2020)). This is in alignment with results reported in Mahinpei et al. (2021), and suggests that there may be substantial information loss when reducing CW’s feature maps into scalar representations. Note that this is not observed in ToyTabular(δ) as its concept representations are already scalars. Interestingly, the implicit supervision (i.e., supervision from downstream task as opposed to explicit concept supervision) in unsupervised CL can be just as effective as explicit concept supervision in other methods in terms of the predictive power of the overall set of learnt concepts for individual ground truth ones. This is observed in CCD and to a lesser degree in SENN. We attribute the gap between the two to CCD being designed to maximize the completeness scores of its overall set of learnt concepts w.r.t. the task. Surprisingly, complete lack of supervision in DGL’s VAE and β-VAE appears to be more effective than the weak supervision in Ada-MLVAE and Ada-GVAE. This is unexpected given that such weak supervision was introduced to address the difficulty of learning representations that capture the data’s underlying factors of variations in an unsupervised manner (Locatello et al., 2019).
Metrics that assume concept independence may be misleading (Figure 3). The comparisons above focused on the quality of overall set of learnt concepts, without an indication of the quality of individual learnt concepts w.r.t. the ground truth ones. The alignment between the learnt and ground truth concepts is clear in CBM and CW. In CCD and DGL, where there is no alignment, we compute a greedy alignment between the learnt and ground truth concepts based on the predictive AUC of using a learnt concept to predict each ground truth concept (i.e., a ground truth concept is assumed to be represented by the learnt one that can predict it best). Details on this are in Appendix 6.4.
After the alignment, we investigate the quality of individual learnt concepts w.r.t. the ground truth ones. First, we show the inadequacy of assuming concept independence compared to our oracle impurity metric proposed in Definition 2. We achieve this by comparing our metric against a variant
of itself, labeled as “non-oracle impurity”, that assumes the oracle matrix of any dataset is always a matrix with 1 in its diagonals and 1/2 in its off-diagonals. We observe that the non-oracle impurity misleadingly shows more impurity as the degree of inter-concept dependencies increases. This is due to the fact that our metric, in contrast to the non-oracle impurity, takes into account the dataset’s inherent ground truth inter-concept dependencies as a baseline for what a method’s purity matrix should look like. We observe the same results across other methods (see Appendix 6.10).
Oracle impurity demonstrates that explicit or implicit supervision may not result in purer individual concepts than no supervision (Figure 4). The oracle impurity metric results demonstrate that CBM’s individual concepts consistently experience the least amount of impurity due to receiving explicit supervision, which is to be expected. Unexpectedly, though, we observe that the same explicit concept supervision can lead to highly impure representations: CW’s full feature maps concepts encode the highest amount of impurity on average, a phenomenon observed by Mahinpei et al. (2021) too. This is due to the fact that the learnt concepts are highly predictive of every ground truth one irrespective of their alignment. Thus, while explicit supervision translates to better concept predictive performance, it does not necessarily translate to purer concepts. Looking into implicit supervision, while the overall set of learnt concepts in CCD performs as well as those learnt via explicit concept supervision in CBM and CW, individual concepts do not correspond well to the ground truth ones. This indicates that the information about each ground truth concept is distributed across the overall representation rather than localized to individual concepts. As a result, surprisingly, in absence of extra capacity, CCD and SENN exhibit an OIS in a similar or better range than methods benefiting from explicit supervision such as CW. We attribute CCD’s lower impurity, compared with SENN, to the use of a regularization term that encourages coherence between concept representations in similar samples and misalignment between concept representations in dissimilar samples. More interestingly, however, SENN encodes higher impurities in its concept representations than all DGL approaches despite benefiting from explicit supervision. Within DGL approaches, not only does the lack of supervision result in better predictive performance for the overall set of learnt concepts as discussed before, but it astonishingly also results in lower impurity representations than those of weakly-supervised DGL methods.
Niching-based metrics demonstrate that supervision may not result in task-label-separable concept representations compared to no supervision (Figure 4). Niching-based metrics do not differentiate between explicit and implicit supervision as much as oracle impurity, as they merely focus on task labels that are available in both approaches. Thus, simultaneous low niche impurity and high niche purity is observed in CBM and CCD, despite their differing degree of supervision. There are, however, surprising exceptions to this: in contrast to CBM and CCD, high impurity and
high purity are observed in CW’s full feature maps (i.e., when no scalar reduction is applied), as well as in SENN’s representations. This is particularly true when concept dependencies increase, which shows the vulnerability of this approach when concepts have inter-dependencies. Performing a MaxPool-Mean reduction in CW’s maps reduces the amount of impurities, however, it also reduces the amount of purity (i.e., the learnt concepts are unable to predict their aligned ones with high confidence). Similar to oracle impurity, this is further evidence that some crucial information may be lost in CW’s scalar reduction, confirming our OIS result. Within DGL approaches, not only does the lack of supervision results in better predictive performance for the overall set of learnt concepts, but it surprisingly also results in lower impurity and higher purity that those of weakly-supervised DGL, a phenomenon worth investigating in the future.
Oracle impurity and niching-based metrics are robust to concept inter-dependencies. The highly preserved ranking of the methods using our metrics in both settings with and without dependencies, indicates the robustness of these metrics to concept inter-dependencies as intended.

heading: 5 DISCUSSION AND CONCLUSION
text: The lack of metrics to evaluate and contrast the quality of concept representations within and between CL and DGL methods hinders their systematic comparison. In this paper, we address this limitation by introducing a set of metrics that facilitate comparison from concept representation quality perspective. Our systematic evaluation of several methods using our metrics suggests that: (i) Model design is just as important as degree of supervision: models with the same degree of supervision (e.g., CBM vs. CW and CCD vs. SENN) can encode varying amounts of impurity in their concept representations, in particular when there are strong cross-concept dependencies; (ii) In settings where ensuring the quality of the overall set of learnt concepts is sufficient, implicit supervision from task labels can be just as effective as explicit supervision from concept labels. Thus, given the cost, providing explicit supervision is not recommended; (iii) In settings where ensuring the quality of individual concept representation as well as their overall set is required, explicit supervision as used in CBM is the most recommended option, followed by CCD if the degree of cross-concept dependencies in the data is unknown; and (iv) In either of the above settings the weak supervision provided by DGL methods is not recommended as a substitute for implicit or explicit supervision. In fact, overall DGL methods without any supervisions tend to outperform the weakly supervised ones. Related to metrics, in setting (ii), regardless of access to concept labels, niching-based metrics should be favoured to oracle impurity. Both metrics are able to capture that the overall set of learnt concept benefits from supervision, with the former metric being very efficient and tractable when the number of concepts is high (as it does not require a classifier to be trained for every pair of concepts). On the contrary, in setting (iii), the high computational cost of oracle impurity is justified if concept labels are available. This is because, unlike niching-based metrics, oracle impurity distinguishes the impact of explicit vs. implicit supervision on the quality of individual learnt concepts.
One direction that was not explored in this paper is the evaluation of the quality of concept representations w.r.t. the input space. In fact, Margeloiu et al. (2021) showed that concepts from CBM often do not correspond to semantically meaningful input space. We therefore believe that future work on defining metrics that capture concept-input quality along with the metrics we proposed here can provide a more comprehensive assurance about the quality of learnt concept representations.

heading: 6 ADDITIONAL RESOURCES

heading: 6.1 METRICS RELATED TO PROPERTIES OF CONCEPT REPRESENTATIONS W.R.T. GROUND TRUTH CONCEPTS
text: Refer to Table 1 for a summary of some metrics used for measuring the quality of disentangled representations which are applicable to disentangled concept representations.

heading: 6.2 MUTUAL INFORMATION IN CUB DATASET CONCEPTS
text: We take the Caltech-UCSD Birds-200-2011 (CUB) dataset (Wah et al., 2011a), formed by images of 200 types of birds, and use each sample’s 112 binary attributes as concept annotations (Koh et al., 2020). We find the top-10 concepts with highest mutual information with the label and compute their Pearson correlation coefficients. Our results, shown in Figure 5, highlight that many concepts have non-zero correlations. This shows the lack of inter-concept independence in real-world datasets.

heading: 6.3 PURITY MATRIX IMPLEMENTATION DETAILS
text: We compute the (i, j)-th entry of the purity matrix as follows: we split the original testing data (Xtest, Ytest, Ctest) into two disjoint sets, a new training set (X ′train, Y ′ train, C ′ train) and a new testing set (X ′test, Y ′ test, C ′ test), using a traditional 80%-20% split. We then use the concept representations learnt
for the i-th concept for samples in X ′test to train a ReLU MLP ψj(·) with a single hidden layer with 32 activations to predict to truth value of the j-th ground-truth concept. In other words, we train ψj(·) using labelled samples {( g ( φ(x(l)) ) (:,i) , c (l) j ) | x(l) ∈ X ′train ∧ c(l) ∈ C ′train } . Finally, we set the (i, j)-th entry of the purity matrix as the AUC achieved when evaluating ψj(·) on the new testing set ( g ( φ(X ′test) ) , C ′test ) .

heading: 6.4 CONCEPT ALIGNMENT IN UNSUPERVISED CONCEPT REPRESENTATIONS
text: In the presence of ground truth concept annotations, one can still compute a purity matrix, and therefore the OIS, even when the concept representations being evaluated were learnt without direct concept supervision. We achieve this by finding an injective alignment A : {1, 2, · · · , k} 7→ {1, 2, · · · , k′} between ground truth concepts c ∈ Rk and learnt concept representations ĉ ∈ Rd×k′ . In this setting, we let A(i) = j represent the fact that the i-th ground truth concept ci is best represented by the j-th learnt concept representation c(:,j). In our work, we greedily compute this alignment starting from a set of unmatched ground truth concepts I(0)ground = {1, · · · , k} and a set of unmatched learnt concept representations I(0)learnt = {1, · · · , k′} and iteratively constructing A by adding one match (i, j) at a time. Specifically, at time-step t+1 we match ground truth concept iwith learnt concept j if one can predict concept i from c(:,j) better than every other concept representation c(:,j′) can predict every other ground truth concept ci′ . We evaluate predictability of ground truth concept ci from learnt concept representation ĉ(:,j) by training a ReLU MLP with a single hidden layer with 32 activations and evaluating its AUC on a test set. Once a match between ground truth concept i and learnt concept representation j has been established, we set I(t+1)ground := I (t) ground\{i} and I(t+1)learnt := I (t) learnt\{j}. We repeat this process until we have found a match for every ground truth concept (i.e., until I(t)ground becomes the empty set). Notice that in practice, one needs to compute the predictability of ground truth concept i from concept representation j only once when building alignment A.
6.5 IMPACT OF CCORRN’S β ON NICHE PURITY, IMPURITY, AND SIZE
β is not a parameter to tune, it is rather a knob to be used to assess the stability and robustness of the concept representations. To this aim, we compute niching scores for different values of β and generate a plot as reported in Figure 6. The figure shows the impact of threshold β used in concept-correlation nicher (CCorrN) on niche purity, impurity, and size in dSprites(λ = 0). Across all approaches, increasing β decreases niche purity and niche size, while increasing niche impurity. When β is high, only very few concepts can pass the threshold required to be identified as concept nicher for a label. As a result, niche sizes get smaller, which translates to bigger niche complement and, subsequently, more impurity.
Figure 6 allows a qualitative comparison of concept representations. CBM provides the most robust representation (niching scores are less sensitive to β), while VAEs provide the least stable set of concepts. CCD representations are almost as stable as CBM ones, while CW is a bit closer to VAEs.
To compare different models across different data sets, we select a fixed value for β for all models (β = 0.2) in a range where the dependency between β and concept scores was approximately linear for most models.

heading: 6.6 CONCEPT NICHES IN REAL-WORLD DATA
text: The niching-based metrics reveal information about the task separability and minimality of concepts. To motivate why task separability and minimality are important for evaluating concept quality, we show that these properties are often empirically observed in real world data.
The intuition behind concept niching is relatively simple: for the sake of example, say we have a multilabel task with labels ”bird” and ”car”. Using niching, we can isolate neurons encoding only the information about a car (e.g., a wheel), only the information about a bird (e.g., a feather), and only the information about both of them (e.g., the colour blue). The ideal concept representation has a small niche for each task (one for “car”=“wheel”, “blue” and one for “bird”=“feather”, “blue”), simplifying concept-based explanations and downstream procedures.
Figure 7 shows the absolute values of concepts-to-tasks linear correlation coefficients in the CaltechUCSD Birds-200-2011 dataset (CUB, (Wah et al., 2011b)), as a representative of real-world datasets. The coefficients are computed on ground-truth concept and task labels. The sparsity of the matrix empirically proves that tasks indeed rely on an often small and non-overlapping set of concepts. Thus, the concept niches for each task do not tend to intersect. If this is preserved by the learnt concept representations, then the Niche Impurity Score (NIS), capturing the amount of undesirable mutual information amongst concepts, should be low, while Niche Purity Score (NPS), capturing the amount of inevitable mutual information amongst concepts, should be high. On the other hand,
simultaneously high NIS and NPS suggests that there may be unnecessary mutual information between concepts that counts as leakage.

heading: 6.7 EFFECT OF NETWORK CAPACITY ON ORACLE IMPURITY
text: We motivate the need of fixing the neural network architecture when contrasting different methods by looking at how the capacity of the selected architecture can affect a method’s learnt concept representations. For this, we train a Joint-CBM in the TabularToy(δ = 0) dataset (with α = 0.1) whose encoder and decoder models are simple ReLU MLPs with two hidden layers. In this experiment, we vary the number of activations in either the encoder and decoder models by setting the number of units in their hidden layers to {capacity, capacity/2} units, while keeping the number of hidden units in their corresponding counterpart fixed to {128, 64}. In this setting, we monitor the predictive accuracy of concept representations w.r.t. their aligned ground truth concepts as well as the representations’ oracle impurities. As shown in Figure 8, we observe that as the encoder and decoder capacities decrease, the CBM begins to exhibit significantly higher impurity and lower concept accuracy. Notice that our results also show that the encoder’s capacity has a significantly greater effect on the quality of the learnt representations over the decoder’s capacity. This motivates the need for a constant architecture with enough capacity to achieve a high concept predictability and serves as a basis for our decision to fix the encoder and decoder architectures across all methods in our evaluation.

heading: 6.8 EXPERIMENT DETAILS
text: In this section we provide further details on the datasets used for our experiments as well as the architectures and training regimes used to obtain our presented results.

heading: 6.8.1 DATASETS
text: For benchmarking, we use three datasets: a simple synthetic toy tabular dataset extending that defined in (Mahinpei et al., 2021), dSprites (Matthey et al., 2017), and 3dshapes (Burgess & Kim, 2018). In order to investigate the impact of dependency between concepts on the quality of concept representation in each method, we allow a varying degree of dependencies in concept annotations and propose the following parameterized tasks:
• TabularToy(δ): this dataset consists of inputs {x(i) ∈ R7}Ni=1 such that the j-th coordinate of x(i) is generated by applying a non-invertible nonlinear function fj(z (i) 1 , z (i) 2 , z (i) 3 ) to 3
latent factors {z(i)1 , z (i) 2 , z (i) 3 }. These latent factors are sampled from a multivariate normal distribution with zero mean and a covariance matrix with δ in its off-diagonal elements. The concept annotations for each sample correspond to the binary vector c(i) := [1(z(i)1 > 0),1(z
(i) 2 > 0),1(z (i) 3 > 0)] and the task we use is to determine whether at least two of the latent variables are positive. In other words, we set y(i) to 1 ( (c
(i) 1 + c (i) 2 + c (i) 3 ) ≥ 2). The
individual functions used to generate each coordinate of x(i) are the same as those defined in (Mahinpei et al., 2021). As in (Mahinpei et al., 2021), we use a total of 2, 000 generated samples during training and a total of 1, 000 generated samples during testing.
• dSprites(λ): we define a task based on the dSprites dataset where each sample x(i) ∈ {0, 1}64×64×1 is a grayscale image containing a square, an ellipse, or a heart with varying positions, scales, and rotations. Each sample x(i) is procedurally generated from a vector of ground truth factors of variation z(i) = [shape ∈ {0, 1, 2}, scale ∈ {0, · · · , 5}, θ ∈ {0, · · · , 39}, x ∈ {0, · · · , 31}, y ∈ {0, · · · , 31}] (θ indicating an angle of rotation) and is assigned a binary concept annotation vector c(i) ∈ {0, 1}5 with elements c(i) := [1 ( z (i) 1 <
2 ) ,1 ( z (i) 2 < 3 ) ,1 ( z (i) 3 < 20 ) ,1 ( z (i) 4 < 16 ) ,1 ( z (i) 5 < 16 ) ]. For this task, we construct a
set of 8 labels from the concept annotations by setting y(i) = [ c (i) 2 c (i) 4 ] 10
if c(i)1 = 1 (where we use [b1b2]10 to indicate the base-10 representation of a binary number with digits b1 and b2) and y(i) = 4 + [ c (i) 3 c (i) 5 ] 10
otherwise. Finally, we parameterize this dataset on the dependency number λ ∈ {0, 1, · · · , 4} that indicates the number of random dependencies we introduce across the sample-generating factors of variation (with λ = 0 implying all factors of variation are independent). For example, if λ = 1, we introduce a conditional dependency between factor of variations z1 (“shape”) and z2 (“scale”) by assigning each value of z1 a random subset of values that z2 may take given z1. This subset is sampled by selecting, at random for each possible value of z1, half of all the values that z2 can take. More specifically, if z1 and z2 can take a total of T1 and T2 different values, respectively, then for each a ∈ {0, 1, · · · , T1} we constraint z2 to be able to take only bT2/2c values from the set z2 ∈ Z2(a) defined as:
Z2(a) = {
SWR({0, 1, · · · , b 3T24 c}, bT2/2c) c1 = 1 SWR({bT24 c, · · · , T2 − 1}, bT2/2c) otherwise
where SWR(A,n) stands for Sample Without Replacement and is a function that takes in a setA and a number n and returns a set of n elements sampled without replacement fromA. This process is recursively extended for higher values of λ by letting the dataset generated with λ = i be the same as the dataset generated by λ = i − 1 with the addition of a new conditional dependency between factor of variations zi and z(i+1). Finally, in order to maintain a constant dataset cardinality as λ varies, we subsample all allowed factor of variations in {zλ+2, zλ+3, · · · , z5} by selecting only every other allowed value for each of them. This guarantees that once a conditional dependency is added, the cardinality of the resulting dataset is the same as the previous one. Because of this, all parametric variants of this dataset consist of around ∼ 32, 000 samples.
• 3dshapes(λ): we define a task based on the 3dshapes dataset where each sample x(i) ∈ {0, 1, · · · , 255}64×64×3 is a color image containing a sphere, a cube, a capsule, or a cylinder with varying component hues, orientation, and scale. Each sample x(i) is procedurally generated from a vector of ground truth factors of variation z(i) = [floor hue ∈ {0, 1, · · · , 9},wall hue ∈ {0, 1, · · · , 9}, object hue ∈ {0, 1, · · · , 9}, scale ∈ {0, 1, · · · , 7}, shape ∈ {0, 1, 2, 3}, orientation ∈ {0, 1, · · · , 14}] and is assigned a binary concept annotation vector c(i) ∈ {0, 1}6 with elements c(i) := [1 ( z (i) 1 < 5 ) ,1 ( z (i) 2 <
5 ) ,1 ( z (i) 3 < 5 ) ,1 ( z (i) 4 < 4 ) ,1 ( z (i) 5 < 2 ) ,1 ( z (i) 6 < 7 ) ]. For this task, we construct a
set of 12 labels from the concept annotations by setting y(i) = [ c (i) 1 c (i) 2 c (i) 3 ] 10 if c(i)5 = 1
and y(i) = 8 + [ c (i) 4 c (i) 6 ] 10
otherwise. As in the dSprites task defined above, we further parameterise this dataset with parameter λ ∈ {0, 1, · · · , 5} to control the number of random conditional dependencies we introduce at construction time. The procedure used to introduce such dependencies is the same as in dSprites(λ) but we use c5 rather than c1 for determining the set of values that we sample from. Similarly, we use the same subsampling as in dSprites(λ) to maintain a constant-sized dataset, resulting in all parametric variants of this dataset having around ∼ 16, 000 samples.

heading: 6.8.2 MODEL ARCHITECTURES AND TRAINING
text: In all of the results we report on the TabularToy(δ) dataset, for CBM and CCD we use a 4-layer ReLU MLP with activations [7, 128, 64, 3] as the concept encoder g(x) and a 4-layer ReLU MLP with activations [64, 128, 64, 1] as label predictor f(ĉ). For CW, we use the same architecture with the exception that a CW module is applied to the output of the concept encoder model. For both CBM and CCD, we train each model for 300 epochs with a batch size of 32. For CW, we train each model for 300 epochs, with CW updates occurring every 20 batches, using a batch size of 128. Finally, for the mixture hyperparameter of the joint loss in CBMs, unless specified otherwise we use a value of α = 0.1.
For our CBM and CCD experiments in dSprites(λ) and 3dshapes(λ), we use a Convolutional Neural Network (CNN) with four (Conv + BatchNorm + ReLU + MaxPool) blocks with feature maps {8, 16, 32, 64} followed by a three fully-connected layers with activations {64, 64, k} for the concept encoder model g(x) (with k being the number of ground truth concepts in the dataset). Furthermore, for the label predictor model f(ĉ) we use a simple 4-layer ReLU MLP with activations {k, 64, 64, L} (with L being the number of output labels in the task). For CW’s concept encoder g(x) we use a CNN with three (Conv + BatchNorm + ReLU + MaxPool) blocks with feature maps {8, 16, 32} followed by a (Conv + CW) block with 64 feature maps. For the label predictor, we use a model composed of a MaxPool layer followed by five ReLU fully connected layers with activations {64, 64, 64, 64, L}. All models evaluated for CBM and CCD are trained for 100 epochs using a batch size of 32. In contrast, CW models are trained for 100 epochs using a batch size of 256 and an CW module update step every 20 batches. Finally a value of α = 10 is used during joint CBM training.
For evaluating VAE, β-VAE (β = 10), Ada-GVAE and Ada-MLVAE, we use the same architecture as in CBM’s and CCD’s concept predictor for the encoder and the same architecture as in (Locatello et al., 2020b) for the decoder. The decoder consists of two ReLU fully connected layers with activations {256, 512} followed by four ReLU deconvolutional layers with feature maps {64, 32, 32, input feature maps}. All DGL models are trained for 100 epochs using a batch size of 32. Weakly supervised models are trained with a dataset consisting of 2N3 pairs of images that share at least one factor of variation (with N being the number of samples in the original dataset) while unsupervised models are trained with the same dataset used for CL methods. As in other methods, we train all DGL models using a default Adam optimizer (Kingma & Ba, 2014), with learning rate 10−3.
When evaluating CCD, we use a threshold of β = 0.0 for computing concepts scores and the same regulariser parameters λ1 = 0.1, λ2 = 0.1, = 10−5 as in Yeh et al. (2020)’s released code for their work in (Yeh et al., 2020). Finally, all CCD models, across all tasks, are trained for 100 epochs and a batch size of 32 using a default Adam optimizer, with learning rate 10−3.
When benchmarking SENN, we use the same architecture as in DGL methods for the concept encoder g(x) and for its corresponding decoder. Note that the decoder in this case is only used as part of the regularization term during training. For the weight model θ(ĉ) (i.e., the “parameterizer”), we use a simple ReLU MLP with unit sizes {input shape, 64, 64, k} (k being the number of concepts SENN will learn). Finally, we use an additive aggregation function and use λ = 0.1 as a robustness regularization strength and ζ = 2× 10−5 as the sparsity regularization strength, as done in AlvarezMelis & Jaakkola (2018). We train our SENN models for 100 epochs using a batch size of 32 and a default Adam optimizer with learning rate 10−3.

heading: 6.9 DOWNSTREAM TASK PREDICTIVE AUC
text: Figure 9 shows downstream task predictive AUC for all datasets using raw inputs, in absence of any dependencies as well as the maximum dependencies between concepts. If concept representations from various methods are good surrogates to the inputs, they need to recover the same predictive performance. Figure 10 confirms that this holds to a good degree by looking at the task AUC of a simple ReLU MLP with hidden layers {64, 64} trained to predict the corresponding task labels using the concept representations generated by each method.

heading: 6.10 IMPURITY SCORES (ORACLE AND NON-ORACLE)
text: Figure 11 confirms that, for 3dShapes(λ), across all models the non-oracle impurity misleadingly shows more impurity that often increases with the dependence of concepts.

Review summaries:
The authors have put decent effort to bring concept-based representation learning and disentanglement learning together under one umbrella in terms of the quality of generated concepts in presence as well as absence of ground truth concept labels. Some related metrics were proposed for evaluation of the quality of concepts for both these methods. Based on these studies, presented in the paper, some important conclusions were made based on requirements of concept supervision and their effects on final concept quality as well as the predictive performance of the model.

The authors consider the question of whether recent concept-based learning algorithms, as well disentangled representation learning algorithms, result in high-quality representations. In particular, they consider what high-quality should mean in terms of the relationship with ground truth concepts and the ability to make accurate predictions for a downstream task. To this end, they propose two main metrics for representations that are explicitly or implicitly encouraged to encode concepts: 1) a score that captures how well the learned representation preserves the relationships between concepts (which may be correlated), and 2) a score that captures how well concepts can be split into groups that are useful/useless for predicting particular label dimensions.