TabCBM: Concept-based Interpretable Neural Networks for Tabular Data
Authors: Mateo Espinosa Zarlenga
Zohreh Shams
Michael Edward Nelson
Been Kim
Mateja Jamnik
Venue: Accepted by TMLR ()

Abstract: Concept-based interpretability addresses the opacity of deep neural networks by constructing an explanation for a model's prediction using high-level units of information referred to as concepts. Research in this area, however, has been mainly focused on image and graph-structured data, leaving high-stakes tasks whose data is tabular out of reach of existing methods. In this paper, we address this gap by introducing the first definition of what a high-level concept may entail in tabular data. We use this definition to propose Tabular Concept Bottleneck Models (TabCBMs), a family of interpretable self-explaining neural architectures capable of learning high-level concept explanations for tabular tasks. As our method produces concept-based explanations both when partial concept supervision or no concept supervision is available at training time, it is adaptable to settings where concept annotations are missing. We evaluate our method in both synthetic and real-world tabular tasks and show that TabCBM outperforms or performs competitively compared to state-of-the-art methods, while providing a high level of interpretability as measured by its ability to discover known high-level concepts. Finally, we show that TabCBM can discover important high-level concepts in synthetic datasets inspired by critical tabular tasks (e.g., single-cell RNAseq) and allows for human-in-the-loop concept interventions in which an expert can identify and correct mispredicted concepts to boost the model's performance.

Published in Transactions on Machine Learning Research (07/2023)
TabCBM: Concept-based Interpretable Neural Networks for Tabular Data
Mateo Espinosa Zarlenga
me4660cam.ae.
Deportment of Computer Science and Technology University of Cambridge
Zohrch Shams
1s3150cam.acc.
Department Computer Science and Technology University of Cambridge
Michael Edward Nelson
mukecdheebonn2220malll.c
Kegrock
European Bioinformatics Institute
University of Cambridge
Been Kim
beenkim @ggoogle com
Google DeepMind
Mateja Jamnik*
mateea..fammsskklll.comm uk
Department of Computer Science and Technology University of Cambridge
Reviewed OpenReview: /ttps://openrestt net/ str/fo id= TIsrnkp
Abstract
Concept-based interpretability addresses the opacity of deep neural networks by constructing explanation for model's prediction using high-level units of information referred to as concepts. Research in this area, however, has been mainly focused on image and graph- structured data, lcaving high-stakes tasks whose data is tabular out of reach of existing hodh In this paper, we address this gap by introducing the first definition of what a high-level concept may entail in tabular data. We use this definition to propose Tabular Concept Bottlenock Models (TabCBMs), family of interpretable sclf-cxplaining neural architectures capable of learning high-level concept explanations for tabular tasks. As our method produces concept-based explanations both when partial concept supervision concept supervision is available at training time, it is adaptable to settings where concept annotations are missing. We evaluate our method in both synthetic and real-world tabular tasks and show that TabCBM outperforms or performs competitively compared to state-of- tho-art methods, while providing a high level of interpretability as mcasured by its ability to discover known high-level concepts. Finally, we show that TabCBM can discover important high-level concepts synthetic datasets inspired by critical tabular tasks (e.g., single-cell RNAseq) and allows for human-in-the-loop concept interventions in which expert identify and correct mispredicted concepts to boost the model's performance.
1 Introduction
Artificial Intelligence, spearheaded by advances Deep Neural Networks (DNNs), has recently mastered tasks once believed to be solely achievable by virtue of innate human ingenuity (Krizhevsky al. 2012
Jumper et al. 2021 Brown et. al. 2020 Silver et al. 2017 Although impressive, these achievements have

---

Published in Transactions on Machine Learning Research (07/2023)
cavent: even though DNNs offer powerful learning framework, their complexity obscures their reasoning, which hampers their applicability in tasks that require human-understandaabl explanations for predictions.
Concerns on the legal (Erdélyi & Goldsmith 2018) and ethical (Durán & Jongsma, 2021 Lo Piano 2020
ramifications of deploying such "black- "black-box"" models in real-world tasks have given rise to explainable artificial intelligence (XAT), methods that are paving the way for safe DNN deployment. A Amongst these, concept-based
interpretability (Kim et al. 2018 Koh et al. 2020a Espinosa Zarlenga et al. 2022), where a DNN is explained via high-level damann-unerstanddee that use low- -indimenioonal human- -aaaansadddett representations rather than imput features, these methods reduce the mental load required to decode an explanation, thus enabling casy model inspection.
concepts, has recently gained attention. By constructing explanations
Given the novelty of concept-based explanations, their development has been limited in breadth, with only a few data modalities explored. As such, most concept-learning methods have focused on image (Ghor- bani et al., 2019b; Chen et al. 2020b), sequential (Ych et al.] 2020) Kazhdan et al.] 2020a). and graph- structured (Magister al.) 2021 2022 data, leaving other modalities under-explored. Hence, crucial lab- ular tasks (e.g., genomics, clinical, and financial tasks), where DNNs have recently been successfully de- ployed (Kadra et al., 2021 Borisov et al. 2021), ; have been overlooked by the concept-based XAI literature. More importantly, while the definition of a concept in highly-struecured modalities such as images (e.g., image segments (Ghorbani et al.] 2019b)) or graphs (e.g. motifs (Magister et al. (021)) is well-understood, tabular data does not have the spatial or geometric structure observed in these domains. This renders existing concept-based methods ill-posed for tabular tasks.
In this paper, we address this gap by defining concept in tabular domains and proposing TabCBM, an interpretable neural architecture capable of lcarning tabular concept-based explanations both in the presence and absence of concept annolations. More broadly, our contributions are: (1) We propose, to the best of our knowledge, the first formalisation of a concept in tabular data; (2) we introduce TabCBM, an end-to- end concopt-interpretabbe neural architecture that learns concept-based explanations for tabular tasks, both when presented with examples of such concepts (concepl-superoisee) or when no concept annotations are available sonpeepllusuuperriieed) (3) we show that TabCBM learns meaningful and interpretable concepts in complex datasets, such as single-cell RNAseq tasks, without sacrificing performance; and (4) we validate our method by demonstrating that it can be used in a human-in-the-loop setup where expert may casily identify and intervene on its learnt concepts to improve model performance.
2 Background & Related Work
Concept Learning Initial Concept: Learning methods (Bau et al. 2017 Fong Vedaldi 2018) explored whether DNNs encode known high-level concepts within their learnt latent space. Aiming to circumvent known failure models (Kindermans et al.) 2017 Ghorbani et al.] 2019a) in popular saliency methods (Sel- varaju et al. 2017 Sundararajan et al. 2017), , as well circunvent the high mental load required to decode featuro-level explanations, XAI algorithms such as TCAV (Kim et al. 2018), CaCE (Goyal et al. 2019 and CME (Kazhdan et al. 2020b) explored explaining a DNN using high-level concepts extracted from its latent space. Follow-up work (Koh et al. 2020a] Chen et al., 2020b), however, argued that extracting concept explanations after training may fail to capture conceptual relations with high fidelity. This has given rise to models that are "concept-aware" and instead construct concept-level explanations at inference.
Concept Bottleneck Models (CBMs) (Koh et al. 2020a) provide framework to study most concept- based methods. Formally, CBM is a composition of two models Jz and fe- The first model ga X' - C, a "concept encoder" with parameters o maps input features x E X (e.g., a bird's image) to k-dimensional concept representation è E E R* (k is a property of the task). Each learnt concept's 's value 2 is incentivised to be aligned with a known ground-truth concept o; (e.g., "has white wings"). The second model fe C + Y' S [o, 1/,, a "label predictor" with parameters θ maps é E C to a distribution over downstream task labels y E y (e.g., bird's type). A CBM can be trained, either jointly, sequentially, or independently, by learning fe and go in a concept-superoised setting. In this setup, we are given training set D == {xx, c, y) x E X, (0, 13 y E (1...... ..] where each sample x is annotated with a binary concept vector e and a task label y. Aligning è to using such dataset encourages CBM's "bottleneck" è = (x) to serve as an
2

---

Published in Transactions on Machine Learning Research (07/2023)
explanation for its prediction 9 fe(g)(x)) (x)) and makes the CBM receptive to concept interventions Koh et al. 2020a), where experts can improve a CBM's performance by correcting mispredicted concepts.
To circumvent the need for concept annotations during training, concept-unssperviieed methods discover concepts using feedback from a downstream task. Post-hoc coneept-ussupevised methods such as Con- cept Completeness-awarc Discovery (CCD) (Yeh et al., 2020) learn a concept basis that contains enough information to reconstruct DNN's latent space. Others, like ACE (Ghorbani al, 2019b) and GCEx- plainer [agisterr et al., 2021), , instead discover concepts by clustering DNN's latent space. In parallel, concept-tnsuperrvised interpretable models, such as Self-explaining Neural Networks (SENNs) (Alvarez- Melis & Jaakkola 2018). have also been proposed. Nevertheless, all of these methods have historically been applied to vision, graph, and language datasets, leaving their behaviour in tabular datasets as an open question.
3 What is a concept in a tabular task?
While it is clear what a concept is in images (e.g., image segments (Ghorbani et al. 2019b) ) sequences (e.g.- sentiment (Kazhdan et al.) 2020a]) and graphs (e.g., subgraph motifs (Magister et al. 2021 ) tabular data do not have the well-defined structure obscrved in these other domains. Nevertheless, it has been observed that, in tabular datasets, one commonly finds high inter-correlations or multicollinearity amongst features (Belsley et al. 2005| Wang et al., 2015; Lee et al. 2021 For example, in genomics, where sample is formed by thousands of gene expressions, major diseases can be mostly attributed to the interaction of few subsets of related genes (Jackson et al.)
2018 Lee et al. 2021b. Similarly, in single-cell RNAseq (sc-RN Assgg) tasks (aacauaay & Voet, 2014) a cell's state can be decomposed into groups of highly related genes, known as Gene Exprcssion Programs (GEPs) (Segal et al. 2003 Kotliar et al. 2019), and groups of transcription factors whose expression or lack thereof represent simple biological functions (e.g., undergoing cell division).
Definition Inspired by the feature multicollinearity observed in tabular tasks, we propose that fixed subsets of highly correlated features can be thought of as tabular concepts. Specifically, we argue that subset of correlated features defines a concept if its features are the inputs of a possibly nonlinear scoring function ( whose output represents high-level "meta-feature's activation (e.g., a GEP's activation). We formalise this by defining a tabular concept on an n-dimensional task as a tuple (n (()) where x. E 10 is a
mask indicating the imput features that are related to this concept and s(9) R E,
T5 - =(0,1) 1) is function indicating the concept's activation given the features selected by x(() (i.c., features j with ;, = 1).
To see how this definition can be applied to a real-world example, consider the task of predicting a user's credit score from their past financial history. Previous work (Chen et al. [2018a] has shown that grouping a potential client's past "criminal" loan history (c.g., "number of defaulted loans", "longest delayed payment", "last delayed payment", etc.) into single "delinquency" score can help predict the likelihood of default. In this context, scoring models, such as that proposed by Chen et al. (2018a), can use a simple interpretable model h that predicts a "delinquency" score from known delinquency-related features to train a simple downstream model that predicts the likelihood of loan default. Hence, the subset of features needed for h can be thought of as a high-level concept when predicting a loan's risk. We can then formally describe this as a tabular concept by letting (1), the mask selecting all the features needed to compute this concept, be binary vector indicating each delinguency-relatee feature and s(), the concept's scoring function, be h.
4 Tabular Concept Bottleneck Model
Motivation A key component of our tabular concept definition is that of understanding which subsets of features form concepts. Here, we explore discovering these subsets by building upon related field: feature selection (Liu et al. 1996 Kohavi & John 1997). Methods in this field reduce the noise in high-dimensional data by selecting a small subset of relevant features Lo learn from (Chandrashekar & Sahin 2014).. Recent
advancements in deep learning (Yamada et al. 2020 Huang et al. 2020b), and in particular in self-supervised learning (Cear et al. 2021] Yoon et al.J [2020] Lee ct al.) (022).. ) introduced differentiable neural architectures for feature selection. In our work, we build on Self-supervision Enhanced Feature Selection (SEFS) (Lee) et al. 2021) to design our concept-based architecture. As discuss below, the crux of our approach lies in
3

---

Published in Transactions on Machine Learning Research (07 (20)))
Latent Code Eneader (learnable & shared)
000
Multiveriate Bernoulli Sampler
f(2)
p (x((
Concept Vector Generator (learnable)
2
Concept Seores
c [0,1]"
Predictor
Label $ [0,1)*
Mask Marginal Probabilities (learnable)
Feature Correlation (fxee and compused from training set)
Figure TabCBM: for each learnt concept (here shown for k' 4) we learn both a per-concept feature importance mask #6) and concept score & E [0, 1] that indicates the concept's activation. All concatenated scores are passed to differentiable label predictor to output downstream prediction.
framing tabular concept discovery as learning multiple feature selection masks simultancously from which multiple concept activation scores can be derived and used for downstream tasks.
Model Architecture A Tabular Concept Bottleneck Model (TabCBMM) shown in Figure I takes as imput an n-dimensional tabular sample x E R and discovers k' tabular concepts, where k' is user-selccted hyperparameter. Inspired by our definition of a labular concept, TabCBM aims to discover a set of k' tabular concepts ((*(), {00) from which a downstream task label 9 can be predicted, and explained, via concept score vector è = (*(1) xx...... s (*(*))(() C xx]'². We proceed by relaxing our discovery task so that each concept's 's mask a(() E [0, 1j" and scoring function s(() : IR" + [o, 1] operate within a continuous space [0, 1]. Such relaxation allows concept discovery in a differentiable manner.
Given a training set D == {(x,, 3)) with N samples and labels, we train TabCBM as follows: for each concept E (1,... k), we learn "soft" feature importance mask (1)) 1]" that indicates how "impor- tant" each input feature is for concept i. Each mask can be thought of as feature-selection subproblem and is learnt by adapting the feature selection process used in SEFS. Specifically, the vector 200 is used to indicate the marginal probabilities of a differentiable multivariate Bernoulli sampler which, by looking at the empirical feature correlation matrix C, samples a per-sample masking vector g(3) E {0,1]" that preserves empirical inter-feature dependencie+] This means that, for example, in genomics task where two genes are highly correlated, the sampler will be more likely to mask these two features together than masking one but not the other. This prevents information from masked features from leaking via unmasked but highly- related features. In practice, we the Gaussian copula (Nelsen 2007) described by Lee et al. (2021) and the Bernoulli relaxation and reparameterisation trick described by Wang & Yin (2020) for the differentiable multivariate Bernoulli sampler (see Appendix for details). This means that (() will be continuous.
Once g()) is generated, we mask the input features to compute a new sample whose unselected inputs are near zero (()) = x0) When constructing our TabCBM model, we associate each concept with a learnable model (( R + RM that takes ( and generates a concept embedding representation p (x()) for concept i. Intuitively, (0) (x(0) can be thought of per-sample embedding representation for the i-th concept and can be used to generate concept score indicating whether this concept is "activated" in x. Using learnable model & Th - Rm that generates an m-dimensional latent representation from x(0) (shared across all concepts and learnt end-to-end with TabCBM), we consider the inner product between a concept embedding and o)x() to be a measure of concept "activation". We generate a scalar concept activation score é E [0, 1] through the sigmoidal inner product:
8 ( = é = ((()))
(1)
Although future work may explore different scoring functions, here we opt lo use the sigmoidal inner product between ((((1) and (x10) rather than, say, the cosine similarity between these two vectors. Our choice of
This is only done training time incentivise concept discovery. During testing, we deterministically set g00) 400
4

---

Published in Transactions on Machine Learning Research (07 2223))
activation allows our model to utilise the magnitude of these vectors to help it more easily express concept's activation or deactivation when passed to a sigmoidal function. This avoids the need to threshold the inner product to generate a concept score between 0 and 1, as done by CCD, and leads to better downstream performance (sec Appendix J.. Furthermore, notice that TabCBM generates its latent representation and concept scores using the masked sample x therefore avoiding information from irrelevant features leaking into a concept's 's score, Once all concept scores have been computed, they are concatenated into a concept bottleneck è [0, 1]k which is passed to label predictor model [0, 11 - [o, 114 to predict the downstream task. Once all concept scores have been computed, they can be concatenated to form concept bottleneck representation è E [0, 1 which is passed to a. label predictor model [0, 1 - [0, 1]4 to predict the downstream task.
Desiderata We argue that meaningful concept generators }/ and masks { M must satisfy:
Concept Completeness: one should be able to predict the labels y of a specific downstream task of interest. from the concept scores é.
2. Diversity: concept embeddings (x())) should each capture semantically unique concepts.
3. Coherence: If two samples are very similar, under some similarity metric (e.g., Euclidean distance), their concept scores è should be equally as similar. This should hold even when determining similarity in a latent representation space (e.g., representations learnt by feature extractor such as PCA). Specificity: learnt concepts should be a function of only a small subset of input features (i.e., feature importance masks {
M=z should be sparse).
We incorporate these desiderata into our model's objective function using a weighted loss of four components:
Lconcept-unsup (x. y) == Clask(((e)) y) + XeoCco(x) + AdivCan(()) + Aapec (111) ((()))
(2)
where Aca: Adiv, and Aspec control how much we weigh each property.
The form taken by the task loss Lissii task-specific (e.g., cross-entropy) and incentivises concept complete- ness. As in previous works (Alvarez-Melis & Jaakkola 2018 Ych et al., 2020). this loss function aims to discover concepts that jointly encode all the information necessary for predicting the downstream task. In light of the impossibility of discovering disentangled latent representations for a given distribution without any supervision (Locatello al.] 2019), this loss, therefore, provides the necessary feedback to discover important concepts for this task in the absence of full concept supervision.
For both the coherence and diversity loss terms, we extend the regularisers proposed by Yeh et al. (200 by letting them operate on per-sample concept vectors. Specifically, our coherence loss incentivises the model to generate similar concept scores for samples whose latent, representations are similar (according to their l2 distance). This is achieved by minimising:
Loo(x), XN)
1 NI
Te(x;) xx xx1 ((x)) [[((x))] [[[(x;]]
Σ
Σ
(3)
where e(x;) is the concept score vector for x; and (xx)(x)) represents the set of t-nearest-neighhouus of o(x:), selected using their l2 distances, in the training set {d(x:)][ (I is a hyperparameter selected suggested by Yeh et al. (2020)). ). Similarly, we define the concept diversity loss as:
p3 (((R)) Ps (2 les (x0()())] Ip: (XO)
Ldiv XN) = NW - xE(X)." xx)
Σ
(4)
That is, we learn diverse concepts by minimising the cosine similarity between distinct concept vectors for a sample. This loss term is similar to that used in several state-of-the-ar contrastive learning pipelines (Chen let al. 2020a Zbontar et al. 2021). Finally, we incentivise specificity by minimising the & norm of all feature importance masks:
k'
Espec (0..,,(Kk))
((K') = k'n
6
(5)
5

---

Published in Transactions on Machine Learning Research (07/2023)
This loss, as in several Lasso-bases sparse classifiers Tibshirani, 1996 Liu et al 2017; Margeloiu et al, 2022 increases concept interpretability by forcing the selection of only handful of features per concept.
Providing partial concept supervision TabCBM can benefit from available training concept supervi- sion, even if given only for some training samples. Assume that a subset X conneet--nnntttted of our training data has k binary ground-truth concept annotations C' ;= {c, E {0, (x,, y1) E X Xconcept-annotsedd] This gives us a dataset of triplets Deoncept-sup == {(x, C. (.)..).. During training, assuming k' k, we can then encourage specific concept scores to be aligned with ground-truth concepts in C by minimising
Stotal E(x.g)>D [Cooncept-unsup (x. v)] Mooncept-sup Peoncops-sup [cconcept-sup L (x, c)] (6)
where Aconcept-sup controls how much we value concept aligument with the training concepts vs concept discovery, and Lconcent-sup (x,c) is the mean cross-entropy loss between è and ex:
conceet-eup (x = Σ (olog(()) log(1 - -,))
This loss encourages the first concept scores to be aligned with known ground truth concepts while incentivising the rest of concept scores to discover semantically distinct concepts to those given during training. This enables TabCBM to maintain high performance without many concept annotations while reducing concept impurities [Espinosa Zarlenga el al. 2023) and leakage (Mahinpei et al. 2021 in its bottleneck.
5 Experiments
In this section, we evaluate TabCBM by focusing on four sets of research questions:
Performance (Q1) - Is TabCBM's performance competitive against that of "black-box" models and other concept- based models when trained with different degrees of concept supervision? Concept Score Alignment (Q2a) - Do concept scores predicted by TabCBM align with ground truth tabular concepts in the input task?
Concept Mask Alignment (Q2b) Even when the concept scores are aligned with known ground-truth concepts, this does not guarantee that the learnt feature importance masks match their corresponding ground-truth concept feature masks. Is it the case that TabCBM's concept masks caplure subsets of features known to have some high-level interpretation? so, can we use these masks to provide global feature importance?
Human-in-the-loog Interventions (Q3) Can we improve the downstream task performance of TabCBM if an expert can identify the semantics of a specific concept and intervene on it?
To answer these questions, we first evaluate TabCBM's task and concept accuracy in variety of concept- supervised and concept-unsupervies setups. Next, quantitatively and qualitatively study the concepts discovered by TabCBM when no concept supervision is given. Finally, we explore how TabCBM's perfor- mance can be boosted in a human-in-the-loop setting with the introduction of test-time concept interventions.
Datasets We evaluate our method on both synthetic and real-world tabular datasets. We construct four synthetic tabular datasets of increasing complexity: Synth-Linear, Synth-Nonlin, Synth-Nonlin- Large, and Synth-scRNA. In these datasets, each sample x is annotated with a downstream categorical task label y and set of k ground truth tabular concepts {(+(), 800) (k = 5 for Synth-Nonlit- k 11 for Synth-scRNA, and & otherwise). By construction, s(0) depends only on small number of input features for all i and the label is uniquely described by the sample's concept activation vector c c= (x)))()xx) ... ,(k) (x) Synth-scRNA., in particular, is a synthetic single-cell RNA high-dinennsionll task in which each concept represents one of 11 known GEPs, 10 of which are identity GEPs (i.e. they are aligned with the sample's cell type) and one which is an activity GEPs (i.c., a group of co-regulated genes biological functions). This makes this task representative of impactful application of TabCBM due to the importance of discovering GEPs from
((()))
and transcription factors which represent aerironmental/cpigene

---

Published in Transactions on Machine Learning Research (07/2023)
Table 1: Downstream accuracy (%) and standard errors show that TabCBM (trained without concept supervision) outperforms or is on par with all methods across tasks. Values are rounded up two decimals and errors are not shown if they are less than 10 10-5, Because CCD is a post-hoe method, its reported accuracy corresponds to that obtained when reconstructing its DNN's latent space with its concept scores.
Datases
TabCBM (ours)
SENN
CCD (recon)
MLP
TabNet
TabTransSorner
XGBooot
LightGBM
Synth-Linear
99.54 + 0.06
98.15
0.2
96.47
+ 1.3
97.57
= 0.37
97.57 = 0.37
82.91 + 0.55
96.43
96,8
Synth-Nonlin
98.36
0.15
89.14
0.71
85.90
2.28
87.85
0.98
91.57
81.07 +: 0.80
88.43
89.33
Syoth-Nonlln-Largg
62.78 + 1.13
49.78 + 2,08
51.64
+ 1.71
40.73
+ 6.42
51.01
2.67
54.63 + 1.17
22.48 + 0.48
23.38 + 0.78
SyalleeeRNA
93.66 + 1.41
78.32 = 3.03
08.83
1.73
13.87
= 1.43
90.06
1.10
87:29 + 0.68
90:44 =: 1.06
89.96
1.57
Higes (without high-level)
80.42 0.3
70.61 + 0.12
71.81 1: 0.08
79.90
0.15
79.41
0.16
74.94 1 6:21
08.85
0.02
68.87
0.06
Higgs (wich high-level)
78.62 1: 0.12
73.53 +: 0.71
77.92
4: 0.09
78.44
0.02
78.12
= 0.05
74.22 1: 0.42
75.33
0.04
75.33
1: 0.03
PBMC
93.35 0.16
92.24 0.23
93.14
0.30
91.06
1,95
92.74
91.01
0.33
90.08
0.29
93.01 0.24
FICO
72.08 + 0.42
66.78
+ 0.69
63.46 +: 4.91
67. 98
1.36
71 20 0.87
65.66
+ 0.85
72.33 :: 0.44
72.63
0.12
2015 All of these synthetic datasets allow TabCBM to be evaluated in a controlled environment where we know the underlying set of complete tabular concepts and their relationship to the downstream lask.
sc- sc-RNAseq data in genomics [Koliir et al.
2019 Kharchenko et al. 2014 Satija et al.
Finally, we use three real-world datasets with unknown ground-truth concepts: (1) PBMC (10x Genomics 2016a b) as a high-dimensional single-cell transcriptomic dataset, (2) Higgs (Aad et al. 2012) as a large real-world physics labular dataset (we also include a variant of this dataset without the hand-crafted high- level features defined by Baldi et al. (2014)), and (3) FICO (Fair Isaac Corporation 2019) as a high-stakes financial task whose features are both categorical and continuous. For further dctails of each datasot used
in our evaluation, refer Appendix D
Baselines and model selection We compare TabCBM against other concept-based methods such as CBMs and Concept Embedding Models (CEMs) (Espinosa Zarlenga et al. 2022 when concept supervision is provided, and against SENN and CCD when concept annotations are not provided. Similarly, we evaluate TabNet (Arik & Pfister 2021p, TabTransformer (Huang et al. XGBoost (Chen & Guestrin 2016) and LightGBM (Ke et al.
2020a). Multi-layer Perceptrons (MLPs), 2017] as represenlative methods of high- performing tabular models, where TabNet, TabTransformer and MLP are neural models and the others are gradient-boosted models (GBMs). When studying TabCBM's concept representations, we include Principle Component Analysis (PCA) (Pearson 1901) as a representation-leaniin in high-dimensional tabular datasets. For the sake of fairness, when possible we train all models using the and training configuration. Similarly,
baseline due to its common use
same architecture/capaccii grid search over set of possible values and report test scores for the hyperparameters that maximise the validation loss. In particular, unless specified otherwise, the number of concepts in concept-learning hods (i.e., k²) is set to the number of ground truth concepts k. If k is unavailable for the task (e.g., as in FICO), we estimate k by selecting the value that maximises the validation loss over a fixed set of candidate values. Similarly, for both CCD and TabCBM the number of top-t neighbours for their coherence loss is selected as suggested by Yeh et al. (2020). Furthermore, when training neural models map categorical features to continuous scalars using 1D embedding and, to speed up training,
select a model's 's hyperparameters via
pre-train TabCBM's latent encoder and mask generators using the process described by Tee et al. (2021) (see Appendix C for details). Finally, when presenting our results, we average each metric over five models trained with different initialisations. For further details on our training setup, see Appendix
5.1 Model Performance (Q1)
In this section, we evaluate TabCBM's first desideratum, concept completeness, by contrasting its downstream task accuracy against that of competing methods. The objective of these experiments is to explore whether our method can maintain a competitive performance against state-of-the-art baselines across several tasks. For this, we study two training setups: (1) a concept unsupervised setup (i.e., no concept annotations), and (2) a partially and fully concept supervised setup (i.e., a fraction of samples are concept-amnottee)..
Concept unsupervised Our experiments' results, shown in Table 1 suggest that TabCBM's task ac- curacy is competitive against that of black box and concept-interpretablle models. Particularly, TabCBM
experiment
7

---

Published in Transactions on Machine Learning Research (07/2023)
Linear
Nonlin
Nonlin-Large
SCHNA
Took Nos
100
100
Taek Baok Aco
100
85
Linear
Nonlin
Nonlin-Large
ScRNA
Chuir Conc ABC
ME3
(b)
300
CBM
Hybrid-CRM CRM
CEM
TahCBM
Figure 2: (a) Task accuracy (%) and (b) mean concept AUC (%) in synthetic tasks. We vary the number of supervised concepts & (x-axis) and the fraction r of mancept-annotatee samples (25% and 100% of samples for the top and bottom rows, respectively). We include only concept-supervied methods, disqualifying CCD and SENN. Similarly, only TabCBM and Hybrid-CBM have results shown when the number of training concepts is 0 (i.e., x = 0) as CBM and CEM do not support training without available concept labels.
significantly outperforms existing concept-based architectures (i.c., CCD and SENN) and neural models (i.c., MLP,, TabNet,, Tab/Transformrr) across all lasks. Similarly, we observe that TabCBM outperforms XGBoost and LightGBM in most tasks with the exception of the FICO dataset, in which TabCBM is marginally outperformed by gradient-boosted methods potentially due to the discrete nature of this task's features and the small training set (something which has been previously observed for other neural models in financial tasks (Schmitt ). Nevertheless, our aggregate results strongly indicate that TabCBM does not sacrifice the performance seen in state-of-the-ar black-box models when it is not provided with supervision.
Concept supervised Next, we explore TabCBM's task and concept accuracy when it is provided with concept supervision. For this, we compare TabCBMs against equivalent-capacity CEMs, CBMs, and Hybrid- CBMs (Mahinpei et al. 2021] Espinosa Zarlenga et al. 2022). the latter being a CBM with extra unsupervised bottleneck activations (we add as many activations as TabCBM's bottleneck). We train all models on our synthetic datasets, given that they have ground-truth concept annotations, while we vary the number of supervised concepts. This is done to study TabCBM's sensitivity to concept. incompleteness and is imple- mented by selecting fixed random subset of concepts provide annotations for. Finally, evaluate each method's data efficiency w.r.t. the nunber of concept-annotated samples by providing concept supervision for 100% and 25% of the training samples (see Appendix F for details).
Our task results, shown in Figure 2 suggest that TabCBM attains better task predictive performance than CEMs, CBMs, and Hybrid-CBMs. We observe this regardless of (i) the number of annotated concepts (x-axis) and (ii) the number of both rows of Figure 2) Furthermore, observe this improvement, which is significant in complex datasets, comes without sacrificing TabCBM's mean concept predictive ROC-AUC" for the concepts that receive supervision, as seen by TabCBM's competitive concept accuracy in Figure 2 The same cannot be said of CBMs which struggle to find a balance between concept interpretability and task accuracy when the training set of concepts is incomplete (sco Figure 2h when a small number of training concepts is provided).
annotated samples (TabCBM's task accuracy is nearly constant in
Notice that although TabCBM's task accuracy generally unaffected by the introduction of concept super- vision (as opposed to Hybrid- Hybrid-CBM'ss, we observe an exception in Synth-scRNA. We see that TabCBM's performance in Synth-scRNA drops when concept supervisions are introduced, suggesting that an accuracy- interpretability trade-off may exist in complex tasks. Nevertheless, discussed in Appendix K when provided with enough concept supervision this trade-off causes TabCBM to be only marginally worse than
2This is defined the mean area under the ROC when predicting each training concept e; with concept score 44.
8

---

Published in Transactions on Machine Learning Research (07/2023)
the next best-performing model. In practice, this may be alleviated by fine-tuning Aconcept-sup and it may be a negligible sacrifice considering that it leads to high-fidelity concept explanations for TabCBM's predictions while allowing performance-boostiin test-time concept interventions as we will see in Section 5.4
5.2 Concept Score Alignment (Q2a)
In this section, we explore whether TabCBM learns concepts that aligu with known ground-truth concepts when receives no concept supervision. We show that, under a series of reasonable quantitative metrics, TabCBM's concepts are better aligned with known ground-truth concepts than those learnt by other concept- based methods. This implies that our method, even without any concept supervision, is able to discover concepts that are semantically aligned to high-level units of information that experts may use in the same task. We follow this quantitative analysis by qualitatively studying TabCBM's concepts in Synth-scRNA and showing a clear alignment between TabCBM's concepts and known GEPs.
Quantitative results We investigate the aligmment of TabCBM's concept explanations/scores by consid- ering different metries that capture TabCBM's desiderata. For this, we compare TabCBM's concept-based explanations against those learnt by CCD, SENN, and PCA by looking at (a) their predictive power w.r.t. ground-truth concepts, (b) their alignment to known ground-truth concepts, and (c) their diversity. We first capture the predictive power of learnt concept representations w.r.t. ground truth concepts using the DCT Informativeness score (Info) (Fastwood & Williams) 2018b), which measures the information content of discovered concepts w.r.t. ground truth concepts. Then, capture explanation's alignment with known ground-truth concepts by checking whether learnt concept scores lead to (i) coherent clusters w.r.t. known ground-truth concepts, as captured by a high Concept Aligmment Score (CAS) (Espinosa Zarlenga el al. 2022), (ii) representations that have at least one aligned dimension per ground truth concept, as captured by a high DCI-Compeeteness ( (Compl) (Eastwood & Williams 2018b), and (iii) representations that have one and only one dimension aligned to a ground-truth concept, as captured by a high R score (Ross & Doshi- p. Finally, we quantify the diversity of the set of learnt concepts by computing their Mu- tual Information Gap (MIG) (Chen et al. 2018b) and Disentanglement Score (Dis) (Eastwood & Williams, 2018b), both measurements of whether each ground truth concept is aligned with one and only one learnt concept, complementing the H with metrics that do not assume invertible correspondences. We provide a summary of these metrics in Appondix
Mean concept representation quality metrics (%) measured across all synthetic datasets for concept- based methods (higher values are better). We show the desideratum closely captured by each metric in italics and parenthesis. To disambiguate variances across tasks, we also show each method's relative mcan rank F.
Table 2:
CAS (oobernoz)
MIG (diorrsit)
R2 (ostermaon & diverity)
Dix (diserslty)
Gompl (complatenne)
Tefo (comspladeeae)
TabCBM (ours)
87.55 à: 14.07 (r = 1.5)
57.71 e 28.27 (r = 1.5)
78:36
17.65 (r 1.5)
69.83 d: 23.80 (r= 1.5)
70.44 :: 22.81 (r= 1.5)
80,03 a 11.22 (r 1.25)
SENN
00.11 - 6.26 9.75)
942 + 5.8 (F- 55)
30.53
17:38 (F- 55)
21.49 0.5)
3.5)
29.56 7:30 (F = 5.5)
32.50 25.55 (P 3:05)
OCD
32.85
20.32
20,57 5,95 (f
65.79
(f
30.00 5,80 (1 2)
41.04 630 ir 2.25
06.77 9.30 (
PCA
57.54
12,80 (r 2.75)
3)
1920
24.15
169 (r 3)
(r
187 32.6 ir
We compute these metrics across all of our synthetic tasks, as they have concept annotations, and show their averages across all of these tasks in Table 2 Our results suggest that TabCBM significantly outperforms all other concept-learning methods across all metries. Specifically, high CAS and Rd scores for TabCBM suggest that it is learning concepts that are highly aligned to known ground truth concepts. Similarly, high Dis and MIG scores suggest that TabCBM's concepts are disentangled and diverse. Finally, as indicated by TabCBM's high Info and Compl scores, TabCBM effectively captures significant information needed for predicting the set of ground truth concopts, therefore learning complete explanations of its task and explaining the observed higher performance of TabCBM over other concept-based methods Section 5.
Qualitative Results To qualitatively corroborate our previous results, we visualise TabCBM's discovered concepts in Synth-seRNA using UMAP (Meinnes et al.] 2018), a commonly-used dimensionality reduction method for scRNA data. For each ground truth concept G,, we visualise the Synth-scRNA dataset (both train and test data) via its UMAP projection while colouring each cell's representation according to c.. Then, we select the discovered TabCBM concept whose score 2j has the highest absolute Pearson correlation
9

---

Published in Transactions on Machine Learning Research (07/2023)
Cell Type
Type
Gell Type
Type
GFF
1.0
MMMPP
0.8
0.7
Eg (p -0.67)
E to 0.74)
4 (p 0.68)
Eg (p 0.71)
E3 (p 0.90)
044
UMAP1
UMAPI
UMAPI
1/MAP1
UIMAP
Figure 3: UMAP of Syuth-scRNA showing five randomly selected ground truth GEPs (top row with the right-most plot showing the only activity GEP) and the score of TabCBM's learnt concept. with the highest mutual information (captured by the absolute Pearson correlation p) with each GEP (bottom row). Note that because TabCBM may learn a ground-truth concept's 's complement labelling, we use the absolute correlation to capture strong strong negative (e.g., &g) correlations as well as strong positive correlations (e.g., êιι). We highlight that the TabCBM model whose scores are shown in this plot did not receive any concept supervision.
coefficient with G; and recreate the plot by colouring each cell using ég. This allows us to visually inspect q's alignment with a (linearly) strongly correlated learnt concept score in TabCBM. Notice that although other proxics for mutual information could be used (c.g., Gini impurity), opt for Pearson correlation due to ils efficiency. Similarly, because only study cell's cluster membership, our analysis avoids known issues with hypothesising from low-dimensional data visualisations (Wattenberg et al. 2016). Our results in Figure 3 show that TabCBM's concepts are closely aligned with ground truth concepts. In particular, we see that TabCBM discovers cell sypes/identity GEPs (e.g., & and ég) and the task's only activity GEP (i.e., ên). As seen Appendix G the same cannot. be said of concepts discovered by CCD and SENN.
5.3 Concept Mask Alignment (Q2b)
An advantage of TabCBM over existing concept-based methods is that its concepts allow for easy identifi- cation of the input features that compose them. In this section, we study TabCBM's learnt concopt masks to explore whether they can (1) identify features that are known to be important for ground-truth concepts, and (2) provide sense of global feature importance through mask aggregation. Our experiments show that both properties hold and, therefore, experts analysing data with TabCBM may use its learnt concept masks to discover novel interactions between groups of input features that are important for their task of interest.
Ground-truth concept mask alignment First, we explore whether TabCBM's feature importance masks align with sets of features known to compose each ground truth concept. For this, we study TabCBM in our concept-annotater synthetic datasets, as they have ground truth binary concept annotations and known ground truth concopt feature masks { (1) - We evaluate the aligmment between TabCBM's learnt concept masks (0 and those in (0 by first computing an alignment a (1,2. .,], - 21,2. ... k} between discovered concepts and ground truth concepts. For the sake of efficiency, we compute this align- ment by matching each learnt concept & with the ground truth concept Cj whose training labels have the highest absolute Pearson linear correlation |P(a,e;)) with the training concept scores for &, We then capture how well (()) is aligned with the feature mask of concept G(a(i) using the Mask AUC (M-AUC) defined as M-AUC = = Σ (OC-AUC(7(((()) (0)) Here, ROC-AUC(T(at))) (()) is the area under the ROC curve when predicting ground truth (((()) with the feature importance mask (1), This score is 1 if A00) x (a(()) for all i and if (6) is the complement of ((((()) for all i.
Our results, summarised in Table 3 show that there is significant alignment between TabCBM's con- cept feature importance masks and ground truth concept feature masks. Nevertheless, we observe that in Synth-seRNA, TabCBM's concept masks are not as highly aligned as other tasks. We believe that this is due the high amount of noise found in this dataset, leading to TabCBM focusing on capturing only the most salient differentiating genes of each GEP.
10

---

Published in Transactions on Machine Learning Research (07/2023)
Table 3: M-AUC and G-AUC scores and standard errors across syuthetic datasets. For M-AUC we include only TabCBM as other concept-interpretable methods do not provide features for learnt concepts. All values are shown as percentages rounded up to two decimals (if an error is less than 10 30,, then it is not shown).
Score
Method
Synth-Linear
Synth-Nonlin
Syuth-Nonlin-e Noniin--nn--ee
Syuth-scRNA
M-AUC
TabCBM
100
100
85.71 + 5.60
65.69 1.66
TabCBM
100
100
100
69.01 2.01
TabNet
85.33 + 8.89
75.33 + 4.42
50.37 + 0.52
50.39 0.15
G-AUC
XGBoust
100
56.10
LightGBM
100
100
100
56.11 0.52
for a model is an important problem (Chandrashekar & Sahin 2014 Lee et al., 2021 Imrie et al 2022). In this section, we explore whether TabCBM's concepts can provide a sense of global feature importance for the downstream task via their mean feature important mask Σ, ((1) any method that constructs a vector A E R" indicating the relative importance of each input feature w.r.f. a downstream task, captures how well this vector predicts ground-truth important features for the task. With this aim, assume we have a binary ground truth vector μ E 10. indicating which features are globally important (i.e., selected) w.r.t. a downstream task. We can compute how well identifies these features using the Global feature selection AUC (G-AUC) defined as:
Global feature selection Understanding which features are deemed
/k'. We investigate this by designing a metric that, for
G-AUC max ROC-AUC (s, threshold max (n)) dES
(8)
where threshold(x, () is vector function whose i-th output is given by 12220 Similarly, we let S C [0, 1] be a finite set of reals indicating the thresholds used for binarising P (as a function of its maximum element) and we let ROC-AUC(y, 9) be the mean area under the ROC curve when predicting y; from 8: for all i. Intuitively, if there is a fraction E of fts maximum value that produces μ when thresholding A. then the G-AUC is 100%. In contrast, G-AUC of 0% indicates that á attains its maximum value in all features which are not selected in p.. For elficiency purposes, in practice we use S {0, 0.025, 0.05,...,.1
For every synthetic dataset, we evaluate the G-AUC by computing μt: V =. as an aggregated logical OR over all ground-truth masks. For fairness, we only compute the G-AUC for methods that provide feature importance by construction, therefore not requiring post-hoe methods (o.g., LIME (Ribeiro et al., 2016)) to obtain this mask. Hence, we focus on contrasting feature importance masks by XGBBoost, LightGBM, TabNet, and TabCBM. For XGBoost and LightGBM we calculate fa the total gains of splits using each feature across all decision trees. Similarly, for TabNet, in (Anık & Pfister 2021), we compute á by aggregating all attention masks. Our results, shown in Table 3 indicate that TabCBM perfectly identifies all important features for our simpler synthetic (asks (as GBMs do too) while achieving significantly better G-AUC scores than other methods in Synth-scR.NA, our more complex synthetic task. These results suggest that TabCBM can globally identify feature selection masks, allowing for global interpretability beyond that in current concept-learning methods. For further results showing how the value selected for k' affects both the M-AUC and G-AUC metrics (as well as the task accuracy), see Appendix [.1
5.4 Concept fest-time Interventions (Q4)
A significant advantage of the existence of a concept bottleneck in a CBM is to allow for concept-level interventions at test time. Through interactions with a CBM in a human-in-the-loop setting, an expert can improve the model's end performance by correcting mispredicted concepts and allowing the CBM to update its previously predicted downstream label to take the correction into account. In this section, we explore whether TabCBM allows for such mechanism even when no concept supervision has been provided during training. For this, we begin with the reasonable assumption that a domain expert can identify strong linear correlations between ground-truth concepts (which are known by the expert) and learnt concept scores. Under this assumption, we propose an intervention procodure TabCBM's bottleneck where, upon identifying that a learnt concept score aligned with a known ground-truth concept, an expert can modify the predicted concept's 's score at test time to correct what they believe may be misprediction, in turn possibly
11

---

Published in Transactions on Machine Learning Research (07/2023)
Synth -ineh-
Nyath-Noulin
Synth-NoclIn Nockn Lange
Syntb-scRNN
Ui UN077 Telk MalAosus
303
100
100
103
1.00
50-
10
cmncppt Interwoted
concepte Interweted
conopoo Intacvennd
ContTeps Interwooed
(sugervised concappe
TabCRM (xupervised concapta 50%)
TabCBM (xupervised concagtis 100%)
Figure 4: Task accuracy of TabCBMs when randomly intervening on a varying number of concepts (x-axis) across multiple datasets (columns). We show the effect of interventions on task accuracies when intervening only on learnt concepts that (i) were not provided with training supervision (top row), (ii) were provided with training concept supervision (middle row). The bottom row shows the results of intervening on discovered concepts regardless of whether they received training supervision not.
affecting the TabCBM's downstream task prediction. Our results, discussed below, suggest that TabCBM can boost its performance through test-time interventions even when it receives no concept supervision.
To evaluate interventions in TabCBMs, proceed as follows: we measure the Pearson linear correlation coefficient P(i.)) between every learnt concept score & and every known ground-truth concept ej in the domain dataset. If for a ground-truth concept C₁ we have that é has an absolute correlation cocfficient greater than or equal to a pre-defined "alignment" threshold 3 E [0, 1] (e.g., we use 3 = 0.7), then we consider éÿ to be aligned with Gi- In practice, an expert could identify such an alignment by inspecting concept scores in a subset of the training data and focusing on instances in which a learnt concept score reaches extreme values.
Let A, : {i B |P be the of ground truth concepts that are strongly aligned with learnt concept à. Notice that this set may be empty if concept score not strongly aligned with any known concepts. When A is non-emply, let 7, be the index correlation with learnt concept & (i.e., == argmaxjeą, (IPt)l))- Given set I C 21, ... ,K} of learnt concepts we wish to intervene on, we perform test-time interventions by updating the bottleneck è as follows:
A, of the ground-truth concept with the highest absolute
(A; = 0) or (i4 Z)
190%)
2 =
Cyp
(A, * 0) and (i I) and (P(s.n) 2 0)
(9)
5%)
Cqs))
(A: + 0) and (i I) and (P(s.)) <
where a/155% [95%)] and o the 95-th and 5-th percentiles of the empirical distribution of è in the training set, respectively. These values are used to represent a concept's 's activation or inactivation by making sure that we perform an intervention while maintaining the scores within their empirical distribution. Once this update has been performed, TabCBM updates its prediction by computing f(č) using the new concept score bottleneck. Notice that this update takes into account the fact that a concept score è may be positively or negatively correlated with ground truth concept ej (i.c., TabCBM may learn the complement labelling of ground-truth concept çj by using & to indicate ey activation rather than = 1).
We evaluate this procedure varying size. These results are summarised in Figure 4 For the sake of completeness, we evaluate TabCBM's intervention elfectiveness on concepl-unsupevvised and fully concepl-supervised setups (partially concept- supervised setups look similar to sullvvsup and are included in Appendix H.I.. h. Our results suggest that TabCBM highly receptive concept interventions both when receives concept supervision as well as when it lacks any form of concept supervision (red solid bars /lines vs dashed coloured bars/lines). We can see this by observing the top of Figure where we show how TabCBM's performance improves significantly
synthetic datasets by intervening
randomly selected set of concepts of
12

---

Published in Transactions on Machine Learning Research (07/2023)
when intervening only unsupervised concepts. Only in the more complex dataset, namely our scRNA task, do we observe that interventions in unsupervised TabCBMs, although significantly beneficial to the model's end performance, are limited in number (i.e., on average we can intervene only on 1-2 unsupervised concopts in Syuth-scRNA). This is because only a handful of discovered concepts were significantly lincarly correlated with known ground truth concepts, and suggests that future work can explore how to exploit weaker non-zero correlations, as those seen in Figure g to better utilise partially aligned concepts in real- world tasks. Nevertheless, and perhaps more importantly, introducing unsupervised concepts in TabCBM can significantly improve its performance even when some of its learnt concepts supervised. This can be seen by noticing that across all models, the performance of concept-supervised TabCBMs improves when we enable interventions in unsupervised concepts (see the improvement in the bottom row of Figure compared to the middle row). These results strongly indicate that. TabCBMs can benefit from being deployed in a human--n--te--le thoop setup, especially when a handful of high-level concepts are known (even if this set is not complete w.r.t. the downstream task). For further dctails on interventions, including a discussion on the effect of and an evaluation of TabCBM's interventions against those in CBMs and CEMs showing improvements against those baselines, see Appendix H
6 Discussion
Relation with Group Feature Selection TabCBM's architectural principles are highly related to those in group feature selection, where methods learn multiple fcatures subsets from which an ensemble can be built (Yuan & Lin 2006 Zhou & Zhu, 2010 Imrie et al. 2022). Nevertheless, there are crucial differences between TabCBM and existing group feature selection methods. First, although as in group feature selection TabCBM uses subsets of highly correlated features to construct a highly prodictive model for a downstream (ask, our method is capable of discovering such feature subsets without priori knowledge required about the nature of these subgroups, as it is usually needed in traditional group feature selection methods. Second, TabCBM does not require the masks used for each discovered concept to be distinct from the masks of other discovered concepts, as multiple high-level concepts can be function of overlapping subsets of concepts. This is in contrast to recent group feature selection hods such as Composite Feature Selection (Imrie) et al. 2022). Third, our method allows for concept supervision to be provided at training time to encourage alignment between certain learnt concepts and known ground-truth concepts, something we have not found in previous literature on feature selection. Finally, TabCBM allows for test time interventions through modifications in its predicted concept scores, enabling better performance in human-in-the-loop setting.
Limitations and Future Work Although we have shown that TabCBM achieves high accuracy in a variety of tasks while learning useful concept-based explanations, this comes with a set of limitations. First, as in topic models (Mcaulifee & Blei 2007] and concept discovery methods (Ych et al., 2020). TabCBM requires K,, the number of discovered concepts, to be a priori defined by a user. While selecting a reasonable value for k can be aided by domain knowledge or through linear search over multiple values of k, one is still required to fine-tune k to deploy TabCBM successfully. Luckily, as seen in our k'-ablation discussed in Appendix [. empirically observe that TabCBM is relatively robust to changes in k' as long as TabCBM is not under-parameter parameterised. Nevertheless, future work can explore ways of automatically learning optimal k'' Second, TabCBM can suffer from parameter and computational growth if either the number of concepts or the size of each concept embedding is large. Future work can explore alleviating this practical limitation through the use of parameter sharing between concept embedding generators and/or better transfer learning protocols which enable learning these generators independently (i.e., separately from the rest of the model). Third, given the lack of previous work on concept-based learning for tabular data, our interpretability evaluation was constrained to mostly synthetic datasets. This is because we found it difficult to find public tabular datasets with both concept and label annotations. Nevertheless, we hope that this work will encourage the application of TabCBM to tabular tasks where, when used with domain-specifie experts, one may be able to discover novel properties which one may later use as annotations for training future concept-based models. Finally, our method requires several hyperparameters to be selected before training as it has several regularisers and architectural choices that need to be made a priori. This can lead to intensive hyperparameter scarches and the requirement of access to certain hardware capabilities. To better understand TabCBM's sensitivity to its hyperparameters, we perform an ablation search over all our hyperparameters in Appendix 1.2 and see
13

---

Published in Transactions on Machine Learning Research (07/2023)
that our method can perform well within large range of hyperparameters. To further help future research, we include set of recommendations for each hyperparameter in that appendix.
7 Conclusion
In this paper, we address the lack of concept-based interpretable methods that apply to tabular tasks. We approach this by first arguing that non-linear functions of highly correlated subsets of input features can be thought of as sensible definition of what concept may entail in a tabular domain. Spanning from this definition, and combining multiple ideas from the fields of feature selection and concept learning, we propose TabCBM as the first concept-based interpretable neural architecture designed for tabular tasks, Our extensive evaluation of TabCBM, both in synthetic and real-world datasets, suggests that our model achieves competitive state-of-tho-ar predictive performance in a variety of tabular tasks whilst discovering concepts that align with known ground-truth concepts. We show that the concept representations learnt by TabCBM are disentangled, highly informative, and complete w.r.t. the downstream task. Furthermore, we show, both quantitatively and qualitatively, that TabCBM capable of discovering meaningful concepts both when it is provided with partial concept supervision and when it is provided with concept supervision at all. This enables our model to be deployed in human-in-the-loog setup in which experts can easily identify and correct semantically meaningful concepts, leading to significant improvements in downstream task performance. This work, therefore, serves as an important stepping stone for designing high-performing interpretable labular architectures, enabling the use of deep learning in high-stakes tabular lasks such as medical diagnosis where understanding model's prediction of utmost importance.
Acknowledgements
The authors would like to thank Adrian Weller, Carl Henrik Ek, and Andrei Margeloiu for insightful discus- sions regarding this work and earlier versions of our manuscript. MEZ acknowledges support from the Gates Cambridge Trust via a Gates Cambridge Scholarship. MJ is supported by the EPSRC grant EP EP/ TT019663//1
References
10x Genomics. Pbems from cd44:/oH55r+///2223 naive cells. https://www. wtt.://annnnnccccm//rrrrrrccll/ com/resources/ http:///cc.....aoooooo//////aoar 2022-09-22.
2016a. Accessed:
10x Genomics. Pbems from cd4+/ regulatory t colls. https::/www.1xxxeennnttt http://www....gaaoooocccc//eeeee com/resourcess
peseetttsdd--plu---ae cdd-pppp---p-pplllll----- htw.://eeeeeee.o.........o.eeee. 2016b. Accessed: 2022-
09-22.
Georges Aad, Tatevik Abajyan, Abbott, J Abdallah, Abdel Khalek, Ahmed Ali Abdelalim, R Aben, B Abi, M Abolins, OS AbouZeid, et al. Observation of new particle the scarch for the standard model higgs boson with the atlas detector at the lhe. Physics Letters B, 716(1):1 29, 2012.
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: A system for large-scale machine learning. In 12th (USENIX} symposium on operating systems design and implementation (1 (SDD))) 16), pp. 265 283, 2016.
David Alvarex-Melis and Tommi S Jaakkola. Towards robust interpretability with self-xxlainiig neural networks. In Neural Information Processing Systems (NeurIPS), 2018
Sercan Arık and Tomas Pfister. Tabnet: Attentive interpretable tabular learning. In AAAI, volume 35, pp. 6679-6687, 2021.
Pierre Baldi, Peter Sadowski, and Daniel Whiteson. Searching for exotic particles in high-energy physics with deep learning. Nature communications, 5(1):1 2014.
14

---

Published in Transactions on Machine Learning Research (07 (07/2023)
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 6541 6549, 2017.
David A Belsley, Edwin Kuh, and Roy E Welsch. Regression diagnostics: Identifying inftuential data and sources of collinearity. John Wiley & Sons, 2005.
Vadim Borisov, Tobias Leemann, Kathrin Seßler, Johannes Haug, Martin Pawelezyk, and Gjergji Kasneci. Deep neural networks and tabular data: A survey. arXiv preprint arXio:2100.01
2021.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprinl ar Xre:22005. 14165, 2020.
Girish Chandrashekar and Ferat Sahin. A survey on feature selection methods. Computers Electrical Engineering, 40(1):16 28, 2014.
Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang, and Tong Wang. An in- terpretable model with globally consistent explanations for credit risk. Xiv preprint arXis:1811.12615 2018a.
Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang, and Tong Wang. A holis- tic approach to interpretability in financial lending: Models, visualizations, and summary-explanations. Decision Support Systems, 152:113647, 2022.
Ricky TQ Chen, Xuechen Li, Roger Grosse, and David Duvenaud. Isolating sources of disentanglement variational autoencoders. arXiv preprint 044411802.04442 2018b.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference knowledge discovery and data mining, pp. 785 794, 2016.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 2020a.
PMLR,
Zhi Chen, Yijie Bei, and Cynthia Rudin. Concept whitening for interpretable image recognition. Nature Machine Intelligence, 2(12):772-782 2020b.
Juan Manuel Durán and Karin Rolanda Jongsma. Who is afraid of black box algorithms? On the epistemo- logical and ethical basis of trust in medical AL. Journal of Medical Ethics, 47(5):329 335, 2021.
Cian Eastwood and Christopher K. T. Williams. A framework for the quantitative evaluation of disentan- gled representations. In International Conference on Learning Representations (ICLR). OpenReview.net, 2018a.
Cian Eastwood and Christopher KI Williams. framework for the quantilative evaluation of disentangled representations. In International Conference on Learning Representations, 2018b.
Olivia J Erdélyi and Judy Goldsmith. Regulating artificial intelligence: Proposal for global solution. In Proceedings of the 2018 AAAI/ACM Conference AI, Ethics, and Society, pp. 95 101, 2018.
Mateo Espinosa Zarlenga, Pictro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, Pietro Lio, and Mateja Jamnik. Concept embodding models. In Neural Information Processing Systems (NewrIPS), 2022.
Mateo Espinosa Zarlenga, Pietro Barbiero, Zohreh Shams, Dmitry Kazhdan, Umang Bhatt, Adrian Weller, and Mateja Jamnik. Towards robust. metrics for concept representation evaluation. AAAT, 2023.
15

---

Published in Transactions on Machine Learning Research (07/2023)
Fair Isaac Corporation Explainable machine learning challenge. https://ommuuty.rrii..oooo
2019. Accessed: 2022-09-22.
Ruth Fong and Andrea Vedaldi. Net2vec: Quantifying and explaining how concepts are encoded by filters in doep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition,
pp. 8730-8738, 2018.
Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. In Pro- ceedings of the AAAI conference on artificial intelligence, volume 33, pp. 3681-3688, 2019a.
Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards automatic concept- based expla- nations. In Advances in Neural Information Processing Systems, pp. 9277-9286, 2019b.
Yash Goyal, Amir Feder, Uri Shalit, and Been Kim. Explaining classifiers with causal concept effect (cace). arXiv preprint ar orXic:1907.07000 2019.
Xin Huang, Ashish Khetan, Milan Cvitkovic, and Zohar Karnin. Tabtransformer: Tabular data modcling using contextual embeddings. arXiv preprint Xrv: 066778, 2020a.
Yingkun Huang, Weidong Jin, Zhibin Yu, and Bing Li. Supervised feature selection through deep neural networks with pairwise connected structure. Knowleedge-Base Systems, 204:106202, 2020b.
Fergus Imrie, Alexander Norcliffe, Pietro Lio, and Mihaela van der Schaar. Composite feature selection using deep ensembles, 2022. URL htttt:://aariiii /ttp:///wwwww./////////////// 00631
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelorating deep network training by reducing internal covariate shift. In International conference machine learning, pp. 448-456. PMLR, 2015.
Maria Jackson, Leah Marks, Gerhard HW May, and Joanna B Wilson. The genetic basis of disease. Essays in biochemistry, 62(5))6433777 723, 2018
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583 589, 2021.
Arlind Kadra, Marius Lindaner, Frank Hutter, and Josif Grabocka. Well-tuned simple nets excel on tabular datasets. Advances in Neural Information Processing Systems, 34, 2021.
Dmitry Kazhdan, Botty Dimanov, Mateja Jamnik, and Pietro Liò. MEME: generating RNN model expla- nations via model extraction. CoRR, abs/2012.06954,, 2020a.
Dmitry Kazhdan, Botty Dimanov, Mateja Jamnik, Pietro Liò, and Adrian Weller. Now you see me (CME); concept-based model extraction. In Conference on Information and Knowledge Management (CIKM) Workshops, volume 2699 of CEUR Workshop Proceedings. CEUR-WS.org, 2020b.
Dmitry Kazhdan, Botty Dimanov, Helena Andrés-Terre Mateja Jamnik, Pietro Lið, and Adrian Weller. Is disentanglement all you need? comparing concept-based & disentanglement approaches. CoRR, abs//1104.06917, 2021.
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30, 2017.
Peter V Kharchenko, Lev Silberstein, and David T Scadden. Bayesian approach to single-cell differential expression analysis. Nature methods, 11(7):740 742, 2014.
Boen Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Inter- pretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In International conference on machine learning, pp. 2668 2677. PMT.R, 2018.
16

---

Published in Transactions on Machine Learning Research (07/2023)
Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T Schütt, Sven Dähne, Dumitru Erhan, and Been Kim. The (un) reliability of saliency methods. ar arXiv preprint arxi:: 111777110000000 1.006677 2017.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ar preprint arXiv:1412.690 6980, 2014.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning (ICML), volume 119 of Proceedings of Machine Learning Research, pp- 5338 5348. PMLR, 2020a.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Slephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept Bottleneck Models. In International Conference on Machine Learning, pp. 5338 5348. PMLR, 2020b.
Ron Kohavi and George H John. Wrappers for feature subset selection. Artificial intelligence, 97(1-2): 273 324, 1997.
Dylan Kotliar, Adrian Veres, M Aurel Nagy, Shervin Tabrizi, Eran Hodis, Douglas A Melton, and Pardis C Sabeti. Identifying gene expression programs of cell-type identity and cellular activity with single-cell ra-seq.. Elife, 8, 2019.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convolutional neural networks. Advances in newral information processing systems, 25, 2012.
Changhee Lee, Fergus Imrie, and Mihaela van der Schaar. Self-supervision enhanced feature selection with correlated gates. In International Conference on Learning Representations, 2021.
Bo Liu, Ying Wei, Yu Zhang, and Qiang Yang. Deep neural networks for high dimension, low sample size data. In IJCAI, pp. 2287-2293, 2017.
Huan Liu, Rudy Setiono, et al. A probabilistic approach to feature selection-a filter solution. In JCML, volume 96, pp. 319-327. Citeseer, 1996.
Samuele Lo Piano. Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward. Humanilies and Social Sciences Communications, 7(1):1-7, 2020.
Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled represen- tations. In International Conference on Machine Learning (ICML), volume 97 of Proceedings of Machine Learning Research, pp. 4114 4124. PMLR, 2019.
Iain C Macaulay and Thierry Voct. Single cell genomics: advances and future perspectives. PLoS genetics, 10(1)=e1044126, 2014.
Lucie Charlotte Magister, Dmitry Kazhdan, Vikash Singh, and Pietro Liò. Geexplainer: Human-in-Che-loop concept-based explanations for graph neural networks. arXiv preprint orXiv:2107.11899 2021.
Lucie Charlotte Magister, Pietro Barbiero, Dmitry Kazhdan, Federico Siciliano, Gabriele Ciravegna, Fabrizio Silvestri, Mateja Jamnik, and Pietro Lio. Encoding concepts in graph neural networks. arXio preprint arXiv:2207.13586 2022.
Anita Mahinpei, Justin Clark, Isaac Lage, Finale Doshi-Velez, and Weiwei Pan. Promises and pitfalls of black-box concept learning models. CoRR, abs/2106.13311 2021.
Andrei Margeloiu, Nikola Simidjievski, Pietro Lio, and Mateja Jamnik. Weight predictor network with feature selection for small sample tabular biomedical data. arXiv preprint arXiv:2211.15616 2022.
Jon Mcauliffe and David Blei. Supervised topic models. Advances neural information processing systems, 20, 2007.
17

---

Published in Transactions on Machine Learning Research (07/2023)
Leland Mclnnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection 03426, 2018.
for dimension reduction. arXiv preprint
Roger B Nelsen. An introduction to copulas. Springer Science & Business Media, 2007.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint Xrxiv:1913 19999...00 01 703, 2019.
Karl Pearson. Liii on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science, 2(11):559- 5722 1901.
Lutz Prechelt. Automatic early stopping using cross validation: quantifying the criteria. Newral Nelworks, 11(4):761-70) 767611777,,70 1998.
Giorgia Quadrato, Tuan Nguyen, Evan Z Macosko, John L Sherwood, Sung Min Yang, Daniel R. Berger, Natalic Maria, Jorg Scholvin, Mclissa Goldman, Justin P Kinney, et al. Cell diversity and network dynamics photosensitive human brain organoids. Nature, 545(7652):48 53, 2017.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "Why should trust you?" Explaining the predic- tions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data måning, pp. 1135-1144, 2016.
Andrew Rosenberg and Julia Hirschberg. V-measure: A conditional entropy-based external cluster evaluation measure. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP--CNTLL) pp. 410 410-420, 2007.
Andrew Ross and Finale Doshi-Velez. Benchmarks, algorithms, and metrics for hierarchical disentanglement. In International Conference on Machine Learning, pp. 9084 9094. PMLR, 2021.
Rahul Satija, Jeffrey A Farrell, David Gennert, Alexander F Schier, and Aviv Regev. Spatial reconstruction of single-cell gene expression data. Nature biotechnology. 3355):995-50,, 2015.
Mare Schmitt. Deep learning vs. gradient boosting: Benchmarking state-of-the-ard machine learning algo- rithms for credit scoring. arXiv preprint Sttt:22055115335 2022.
Eran Segal, Michael Shapira, Aviv Regev, Dana Pe'er, David Botstein, Daphne Koller, and Nir Friedman. Module networks: identifying regulatory modules and their condition-specific regulators from gene expres- sion data. Nature genetics, 34(2):166 176, 2003.
Ramprasaath R Sclvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pp. 2017.
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonogiou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastcring the game of go without luman knowledge. nature, 350(7676):354-33)) 2017.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic altribution for deep networks. In International Conference on Machine Learning, pp. 3319-3328. PMLR, 2017.
Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodologicaal) 58(1):267-288, 1996.
Talip Ucar, Ehsan Hajiramezanali, and Lindsay Edwards. Subtab: Subsetting features of tabular data for self-supervised representation learning. Advances in Neural Information Processing Systems, 34, 2021.
Bork Ustun and Cynthia Rudin. Supersparse linear integer models for optimized medical scoring systems. Machine Learning, 102(3):349-30) 0911399999999991 2016.
18

---

Published in Transactions on Machine Learning Research (07/2023)
Berk Ustun and Cynthia Rudin. Optimized risk scores. In Proceedings of the 23rd ACM SIGKDD interna- tional conference on knowledge discovery and data mining, pp. 1125-11344 2017.
Xi Wang and Junming Yin. Relaxed multivariate bernoulli distribution and its applications deep generative models. In Conference on Uncertainty in Artificial Intelligence, pp. 500 509. PMLR, 2020.
Zi Xiu Wang, Q Peter He, and Jin Wang. Comparison of variable selection methods for pls-based soft sensor modeling. Journal of Process Control, 26:56 72, 2013.
Martin Wattenberg, Fernanda Viégas, and lan Johnson. How to use t-sne effectively. Distill, 1110)ee2, 2016.
Yutaro Yamada, Ofir Lindenbaum, Sahand Negahban, and Yuval Kluger. Feature selection using stochastic gates. In International Conference on Machine Learning, pp.
PMLR, 2020.
Chih-Kuan Ych, Been Kim, Sercan Ömer Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. On completeness-aware concept-based explanations in deep neural networks. In Newral Information Processing Systems (NeurIPS), 2020.
Jinsung Yoon, Yao Zhang, James Jordon, and Mihaela der Schaar. VIME: Extending the success of self-and semi-supervised learning to tabular domain. Advances in Neural Information Processing Systems, 33:11033 11043, 2020.
Ming Yuan and Yi Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 68(1):49-67, 2006. doithtlpss:/doole /ttp:////0/////111111///1//11110
2005.005550 00532x)
URL httpp:://ssst http:////rraaoooonebbbbaaaaareee www.oelibbarrryy.e.e..am..aaaaal wtw.liibrrrrrrlrrrrr.........eee wiley. wimey.com/
1t5/abs/10.1111/5... 14111///.117-98888
Luke Zappia, Belinda Phipson, and Alicia Oshlack. Splatter: simulation of single-cell RNA sequencing data. Genome biology, 18(1):1-15, 2017.
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning, pp. 12310 12320. PMLR,
2021.
Nengfeng Zhou and Ji Zhu. Group variable selection via a hierarchical lasso and its oracle property, 2010. URL htttps://arxtiii .rg. httg//bs/10006 2871
A Relation between Tabular Concepts and Subscale Scoring Models
We notice that our definition of a tabular concept is similar to the core idea behind interpretable subscale or additive scoring models (Chen et al. 2022 2018a Ustun & Rudin 2016 2017). The main differences between the two are that (1) we interpret each subscale as a high-level concept in that domain (whereas subscale models do not necessarily assign semantics to subcales), (2) we learn these subsets, and their corresponding functions, via supervision from downstream task rather than using hand-crafted scoring scales, and (3) the function mapping each concept's features to its corresponding score can be highly non-linear (subscale models tend to assume linear functions between feature subsets and their scores).
B Multivariate Bernoulli Sampler Details
As indicated in Section 4 during training, we use TabCBM's learnt concept feature importance vectors {*())/ to generate per-sample binary masks {0(0)/ where each feature's probability of being selected by ((0) is proportional to their corresponding fealure importance in 2(0) Furthermore, we avoid assuming full independence between features when generating g(i) from (() by considering #(0) only as the marginal probability distributions of selecting each feature and introducing a non-trivial covariance matrix in the sampling process that considers inter-feature correlations. As argued by Lee et al. (2021), this has two advantages: (i) during the pre-training stage of the feature importance masks (see Appendix C this process
19

---

Published in Transactions on Machine Learning Research (07/2023)
encourages our latent code generator to avoid relying on information leaked from highly correlated unmasked features to solve the sell-supervised pretext tasks, therefore improving the quality of pre-trained learnt representations, and (ii) using a non-trivial covariate matrix to sample (60) encourages TabCBM's concepts to select features that are highly correlated together, aligning with our definition of tabular concepts.
In practice, this is achieved by sampling (69 from a multivariate Bernoulli distribution we construct using a
Gaussian copula (Nelsen 2007). A Gaussian copula multivariate cumulative distribution function (CDF) for random variables (X") M1=1 in [0, 1]" whose marginals are uniformly distributed (i.e., X Unif(0, 1)).
Given matrix C e [-1, 1| _|=Xa capturing the correlations of random variables (x()) JE=1 define the Gaussian Copula as:
Copula (x(() "",,,,,('))))))))))))))))))))))) == (x())
where we use 4 indicate the inverse CDF of a standard univariate normal distribution and De indicate the joint CDF of multivariate zero-mean normal distribution with correlation matrix C. This construction is extremely powerful as it allows Bernoulli distribution MultiBern(#%% (x))))ennn((()))))) using the copula's correlated random variables (x") )== by letting
sample binary mask vector g(3) from the multivariate
10:={a 1,
if x S #/00)
0/0
= [0,
0, otherwise
In TabCBM we construct this copula by building E [-1, 1|nxa via the empirical feature Pearson correlation coefficients computed in the task's training set.
make this sampling process differentiable by working with a relaxation of the multivariate Bernoulli distribution proposed by Wang & Yin (2020) and using the reparamcterisation trick on the copula's uniform random variables {X Specifically, given Copulag's 's uniform random l==1 and the marginal selection probabilities as dictated by a concept's feature importance mask ((),, we can generate relaxed masking vector g(0) by setting it. to
Finally, as in (Lee et al. 2021
variables {x
0f := ( a(() - log (1 a1(0) + log X log (1 (1- x(1))
where a the sigmoid function and o0) is a temperature hyperparameter which we fix to in our experiments. This allows the output relaxed mask g(() to be differentiable with respect to (1),, therefore enabling us to learn each concept's feature importance vectors via gradient descent. In practice, we implement this relaxation using deterministic transformation of a standard Gaussian noise vector whose entries encode the correlation structure in the features after being projected into the subspace L of the lower-triangular Cholesky decomposition LL" of C. For further details, see Section 4.3 of (Lee et al. 2021
C TabCBM pre-training
In practice, we speed up TabCBM's learning via two pre-training steps: In the first step, encourage the latent code generator Φ(x) to learn meaningful representations by training it to minimise the expected task loss ((x.y)>D[Lusee (fdownstremw (otee)ammm()))))) (o(x)).)) where 9dowustream : Rm > [0, 1]. is a learnable auxiliary linear layer used to predict a class ý from ((x). This encourages & to learn representations that are meaningful for the downstrcam task.
In second pre-training step, we use self-supervision to learn meaningful latent code generators using the pre-text task proposed by Yoon et al. 22020) and also used in (Lee et al. 2021). Specifically, we first randomly initialise each concept feature importance mask 200) by sampling it from Unif(0.4, 0.6). We chose to sample each initial mask from Uni,(0..4, 0.6) rather than from Unif(0, 1), as we found that conditioning their values to be closer to 0.5 at the start training helped with the training dynamics. For each training sample x and each learnt concept i, we then sample a binary mask g(i) {0, 13" from a multivariate Bernoulli distribution whose marginal probabilities are fixed to A and covariance matrix captured via Copulag. As opposed how we proceed in our training stages for TabCBM, in this pre-training stage we do not apply any relaxations
20

---

Published in Transactions on Machine Learning Research (07/2023)
when generating 8() meaning that it will be binary rather than continuous as in our training stages and that we will not update any feature importance vectors 2 This is done for us to be able zero out elements of input x and produce a masked vector X from which we train (xx) so that the latent code obtained from masked input X is informative enough for an auxiliary model Jmask : R - [0, 1]" to be able to predict mask 0. Simultaneously, we also condition our learnt latent code to be a meaningful representation by using it as the input of a separate simple auxiliary model Jentries IRm - I from which we aim to closely reconstruct the original sample x from o(x). Both of these objectives can be achieved by finding the parameters for o, Jmask; and Gentries that minimize the weighted loss independently for each concept
W
haw.:///inBeeeoogooooooooaeerere (ecooeeenttroyy( (gmask ((0)). + Aree Sheellgenn [elteetttis(((xxt B(9) x|1
where Arec R is hyperparameter controlling how much value perfect reconstruction over mask identification (we fix it to Arec 10 throughout evaluation). The purpose of this pre-training step is to precondition the latent code generator (0-) to capture meaningful semantics of the input features when some of its features are masked, as this will be done when learning different concepts for TabCBM.
For all of our experiments, we pre-train TabCBM by running the first pre-training stage for T pre-trrii epochs and then running the second pre-training stage for seel-seeerreeedd epochs (specific parameters are described in Section E.2p. Furthermore, we use a linear layer for the helper model Idownstream and a ReLU MLP with single hidden layer size of 64 for the Jmask and Pentries helper models. Finally, for the sake of fairness, we pre-train the backbones of all other competing models using the first stage of this procedure for T prostrain + sprrisupereee epochs to allocate equivalent computational budgets across all methods.
D Datasets
In this section, we give a detailed description of the datasets we use for our evaluation of TabCBM. A summary of each dataset's core characteristics can be found in Table
Table 4: Characteristics of all tasks used in this paper. Notice that we worked exclusively with classification tasks, so all downstream labels are categorical. The class imbalance ratio is defined as the ratio between the number of samples of the most represented class and the number of samples of the least represented class.
Dataset
of Samples
# of Features
# of Concepts
# of Classes
Class Imbalance Ratio
Lynehr
15,000
100
2
4
1.05
Syuth-Nonli
15,000
100
2
1.05
Syuth-Noulin- Large
15,000
500
5
32
7.4
Syuth-scRNA
7,500
5,000
11
20
2.6
Higgs
(without high-level)
11,000,000
21
n/a
2
1.13
Higgs (with high-level)
11,000,008
28
n/a
1.13
PBMC
20,738
6,212
n/a
1.01
FICO
10,459
23
n/a
1.09
Synthetic Tabular Datasets To evaluate our methods in a controlled environment, we propose three synthetic tabular tasks with increasing difficulty. The first two tasks, namely Synth- Linear and Synth-Noulin, consist of N = 15, 000, 100-dimensional samples {x E R 100 Ji generated from a corresponding latent vector h E 1000 ~ ~N I), where I is the identity matrix (each sample x has its own corresponding latent vector h). Then, we generate each sample from its corresponding latent vector h by applying a transformation let (h) h be the identity function while in Synth-
f R + R to h. In the case of Synth-Linear, Nontin, we let f(h)- sin(h) + h be non-linear function of h. To assign label y to each sample, we first annotate each sample with a binary concept vector e E {0, 1}*, with k = 2 for these two datasets, such that its j-th dimension Cj := 1 " hy)20 indicates the sign of adding a subset of consecutive features in
the latent vector h (we set the spacing to be 5 for these two datasets). This binary vector is then used to produce label y E (0, 1, 1,... 2 -1} by generating y from the decimal representation of vector e i.e.,
21

---

Published in Transactions on Machine Learning Research (07/2023)
y y= (- Notice then that the label assigned to each sample can be uniquely determined from its corresponding concept vector c, which, in turn, is function of only the first sk dimensions of sample x. Therefore, to properly learn the generative process for these datasets, a method must be able to capture the importance of the first sh dimensions and discover, either implicitly or explicitly, the concept vector c.
We further extend the generative process of Synth-Nonlin and generate a more complex dataset, which we refer to as Synth-Nonlin-Larye in which features have 500 dimensions rather than 100 and k 5 binary concepts с E {0, 1) are constructed to determine label y (0, 2$ - 1} instead of only two concepts. As in Synth-Nonlin, each dimension Cj of is computed using an indicator variable of subset of features in h. In contrast with Synth-Nonlin, however, we allow different dimensions of c to depend on overlapping and setting
sets of features in h. We do this by letting Cj be defined as Cj == 1 -immety-- 4))20
y y= (a ()))))))))))))))))))))) 10 as we did for Synth-Nonlin. For this dataset, we use a spacing factor of s = 20 to construct our concepts, given that we also have more features than in the previous two datasets. This means that only the first 100 features of the dataset will be useful for learning the downstream task.
Synth-scRNA As an exploration of applying TabCBM to meaningful bioinformatics tasks, we explore a synthetic RNAseq (Macaulay & Voet 2014) task where each sample contains gene expression counts for a total of 5, 000 genes at a single-cell resolution. To generate such datasct, we follow the approach suggested by Kotliar et al. (2019) and adapt Splatter (Zappia et al. 2017) to generate a ac-RNAseq dataset where we can control the number of identity and activity gene expression programs (GEPs) (Segal et al., 2003) in the cell population. In this context, identity GEPs are aligned in a one-to-one fashion with different cell types and represent groups of signature genes that are co-regulated for cach cell type differentiation. In contrast, aclivily GEPs represent groups of co-regulated genes which are not necessarily related to a cell's type and instead represent biological functions related to environmentalleppieeneee related to different stages of a cell's cycle). For our experiments, with 7,500 samples in it, with ten different cell types (i.e., ten identity GEPs) and one activity GEP which can be expressed across all cell types. Given that being able to disentangle and learn identity and activity GEPs from Co-RRNAser data is an important task in genomics (Macaulay & Voet 2014 Kotliar et al.) 2019
effects (e.g., genes generate a synthetic RNAseq dataset
Kharchenko et al. 2014 SSaija et al. 2015). ; this synthetic task is representative of potentially impactful application of our algorithm.
In practice, we generate this dataset using Splatter's Python implementation by Kotliar et al. 2019 In this framework, call Splatter by setting the number of genes to 5, 000, the number of cells to 7, 500 and the number of genes differentiating in the task's activity GEP to be 250 (we extended Splatter's implementation by Kotliar et al. (2019) to allow for a specific number of genes used for each GEP). All other Splatter parameters (e.g.; loc, lib.scale, elc), except for the number of doublets which we set to 0 (i.e., we do not simulate doublets), are set to the same values as those used for the synthetic scRNA-sey dataset in (Kotliar et al. 2 These values were estimated using Splatter on subset of the organoid dataset proposed by Quadrato et al. (2017).
Before training, we preprocess the generated dataset by (1) removing all cells with less than 200 gene expression counts, (2) removing all cells with less than 200 genes with non-zero counts in total, (3) removing all genes expressed in less than 50 cells, and (4) normalising all counts for all cells (normalisation is done on a per-cell basis) and transforming these normalised counts to a log space using a log(x: (x + 1) transformation (as commonly done for scRNA-soq data (Kotliar et al. 2019:).
To produce task in which GEPs are relevant for the class, assign each cell a label representing its cell type (which is in one-to-one correspondence with an identity GEP) and whether or not the dataset's activity GEP is on. This yields an annotated datasct where cach sample has one of 10 x 2 = 20 labels and cach label can be uniquely determined by knowing the cell's identity and activity GEPs (see Figure for a visualisatiion of this dataset). Therefore, in this task both identity and activity GEPs can be thought of as ground ground-truth high-level concepts which can fully describe the label our task of interest. Furthermore, as we know the genes that are differentially expressed for each GEP, we can evaluate how accurately our method can discover the set of differentiating genes for each GEP using these masks as ground-truth concept masks.
22

---

Published in Transactions on Machine Learning Research (07 (07/2023)
Type
Type
Cell Type
Cell Type #4
Cell Type
Activity
UMAP2
UMAP2
Cell Type #6
Cell Type #?
Cell Type #8
Cell Type #9
Cell Type #10
UMAP2
UMAP1
UMAP1
UMAP1
UMAP1
UMAP1
UMAP1
Figure UMAP visualisations the Synth-scRNA dataset with each identity GEP (i.e., cell type) and activity GEP highlighted.
We further explore application of our method in real-world single-cell transcriptomics dataset. With this aim, and inspired by the work by Lee et al. (2021) we construct a binary transcriptomics task from two subpopulations of peripheral blood monocytes obtained from the PBCM (10x Genomics 2016a b) database. Specifically, we construct a dataset with approximately 20,000 samples where cach sample con- tains transcriptomics measurements of nearly 6,000 genes a T-cell (i.e., RNA expression paiterns). We preprocess and normalised all counts as done for our Synth-scRNA gene counts and annotate each cell with binary task label describing whether the cell corresponds to regulatory naive T-cell.
PBMC
Higgs As real-world example of large tabular dataset in the physical sciences, make use of the Higgs dataset (Baldi et al. 2014). In this dataset, we aim to classify a signal process as being one from which a Higgs boson is produced or as being a background signal. All 11 million samples in this dataset were produced via a Monte Carlo simulation and each sample signal is formed by 28 continuous features. these fcatures, the first 21 features represent kinematics propertics whereas the latter 7 fcatures are high-level features hand-crafted by experts which allow easier discrimination between background and proggs-po signals. To explore whether hierarchical models such TabCBM can learn the information encoded into the hand-crafted features without extra supervision, evaluate all models both when using the hand-crafted high-level features (Higgs (with high-level)) and when none of the hand-crafted high-level features are used (Higgs (without high-level)..
Given that a large proportion of tabular data is seen within financial tasks, and the fact that these tasks include both numerical and categorical features, we include the Fair Isaac Corporation (FICO) loan default risk challenge dataset (Fair Isaac Corporation, 2201 as part of our evaluation. We choose this specific dataset it has been shown that simple handcrafted subscale models, whose relationship with high-level concepts was not made until this work, be used to construct interpretable classifiers for this dataset (Chen et al. 2018a). Each of the approximately 10,000 samples in this dataset is composed of 23 features, & mixture of numerical and categorical features representing several properties of a client's past loan and financial history, and is annotated with a label indicating whether this client defaulted on their loan. The task to predict the likelihood of new client defaulting on a new loan.
FICO
E Experimental Details
In this section, we provide details on each method's configuration and hyperparameters for cach task evalu- ated in this paper. All of our numerical results are produced by averaging given metric across 5 different random inilialisations of each model and showing the slandard error found across those runs.
23

---

Published in Transactions on Machine Learning Research (07/2023)
E.1 Training and Optimisation Hyperparameters
For each method, and across all lasks, we split each task's dataset into 80% training data and 20% test data and generate a validation set by randomly sampling without substitution 20% of the training data. When sensible, this validation set is used to search over a subset of possible hyperparamcters for cach method by running a simple grid search over a set of predefined hyperparameters (described when appropriate below). Once the best hyperparameters are selected, we report only on the test results obtained for the model with the best validation error.
As is commonly done for training DNNs, we train all neural models via stochastic gradient descent with the same batch size B for all methods across given task. We choose specific batch size to maximise GPU utilisation while remaining within our hardware's ' memory capabilities. We used an Adam optimiser (Kingma & Ba 2014) with learning rate 10-3, momentum 0.99, and standard hyperparameters B1 = 0.9 and B3 = 0.999, across all methods and tasks. The exception for this is TabNet, where the learning rate starts at 0.02 and decays with a factor of 0.9 every ten steps as suggested by Arik & Pfister (2021). All methods are trained for T epochs, stopping early (Prechelt) 1998) if no improvements have been observed in the validation loss for P epochs. Hyperparameter values used for each dataset are reported in Table 5
E.2 Model Selection
For the sake of fairness, we provide each method with the same capacity and training times when possible, With this aim in mind, we fix the architecture used across methods to be the same for given dataset. We select architectures that are simple to train, yet large and expressive enough to perform well in each task of interest; with the constraint that they should train in our GPU cluster within reasonable times. In this subsection, we list the architectures used for all methods across a given dataset and summarise their exact values used for all datasets in Table 5
TabCBM Model Selection Across all datasets, we construct TabCBM's latent representation generator using a simple ReLU MLP whose hidden layers have sizes Ohidden-sizes and whose output layer has size m (with m being the size of the learnt concept embeddings). To help reduce training-tine time covariance shifts, we also introduce batch normalisation (Toffe & Szegedy, 2015) layer before each linear layer of this MLP, including the first layer. We then construet TabCBM's label predictor f using a ReLU MLP with hidden layers whose sizes are determined by fuidden-siess The number of output activations of f will correspond to the number of output classes for the task of interest. Across all tasks, we fix the architectures of TabCBM's p() models to be ReLU MLPs with a single hidden layer with size 64 for all i E (1,..., K/} (the only exception for this are the FTCO dataset where, due to hardware constraints, we reduce this to 32 and the PBMC task where we use an MLP with units {32, 16) to avoid parameter growth). Moreover, when the number of ground truth concepts is unknown, lowest validation loss. Otherwise, we use k' = k, where k is the number of known ground truth concepts in the dataset. As for the loss term weights, we select the values for Aco, Adiv, and Aspec via a grid search (by training for a few epochs only) as we vary each parameter, independently of each other, over {0.1, 10). Finally, we set t, the hyperparameter of our coherence loss, to t = (batch size average class ratio)/2 as suggested by Yeh et al. (2020). We report the berf-performing parameters for each task in Table 5
train TabCBMs with k E KC and select the model with the
MLP Model Selection For simplicity, we construct our MLP baseline by constructing a ReLU MLP whose hidden layers have size Ohidden-sizes U {m) fuidden-sizes and where batch normalisation layer is introduced before the first Ohidden-sizes hidden layers, as in TabCBM. To ensure fair comparison against TabCBM, which has more parameters than this MLP due to its concept-generatinng models, we also consider larger-capacity MLP which has twice as many activations in each layer as those defined by chidden-sizes U {m) U fhädden-sizes- However, in practice, we observe that this did not bring noticcable performance benefits (in fact, it sometimes even decreases performance, possibly due to overfitting), and therefore we opted for the smaller ML.Ps across all datasets.
CCD Model Selection For CCD, we construct cancept-based explanations for the MLP model defined above using the latent representation at layer [dhidden-sizes U {m)) as the representation we will aim to recon-
24

---

Published in Transactions on Machine Learning Research (07/2023)
struct using CCD's learnt concept scores. Its K Syper-paramen (representing how many top-K neighbours we use to compute its coherency regulariser) is set to K = (batch size. average class ratio)/2, as suggested by the authors, and the thresholding hyperparameter 3, indicating the activation of given concept, is fixed to O. Similarly, we fix CCD's hyperparameters a and X to X = X = 0.1 after experimenting with values in [0.01, 1] and finding these values to work well across the validation sets. Finally, for the reconstructing model g of CCD's topic model (this the model that will take the concept scores as input and attempt to reconstruct the latent representation of the DNN), we use a ReLU MLP with a single hidden layer with 500 activations in it and whose output layer has m activations.
SENN Model Selection For our evaluation of SENN, we begin by constructing an autoencoder OSENN whose encoder uses the same architecture as TabCBM's o function with the exception that its output layer generates k' activations (selected in the same way as in TabCBM). The decoder of this autoencoder is selected to be a ReLU MLP whose hidden layers have size fhädden-sizes and whose output is the same size as the number of input features in the task. This autoencoder is trained by minimising its reconstruction loss for Trre-trsin + Tuel--supervie epochs. We then train a SENN using ØSENN as its concept encoder and a ReLU ML. with hidden layer sizes Ouidden-units and L outputs (one vector of size k' for each output class {1.... .L) as the relevance parameteriscr. Throughout training, we follow the suggestions by MlvarczzMMllis & Jaakkola (2018) and fix SENNs regularisation strength to X = 0.1 and its sparsity regularisation strength to = 2 x 10
TabNet Model Selection For TabNet we explore different architectures with different capacities. We do this by exploring setting hyperparameters Na, Na, Nsteps by iterating their values from a set of options in sets Na= = V. = {8, 16, 32, 64, 96} and Nsteps = (3 8), respectively. Nevertheless, as with MLPs, we observe that over-paramcterising TabNet led to worse-performing models, therefore in Table 5 we only report the values of these hyperparameters that resulted in the best validation error. The virtual batch size of all TabNet models is set to half of the batch size for our dataset (i.e., B/2). Furthermore, pre-train TabNet, using the sule--upe task described by Arik & Pfister 20221), for T pree-train + Taelf-supervised epochs.
TrbTranssormer Model Selection When constructing TabTransforner model, we use the default parameters defined by Huang et al. (2020a) while setting the number of hidden layers to Thidden-layere-
XGBoost and LBM Model Selection For both of our gradient boosting baselines, namely XG- Boost and LightGBM, we train them for T epochs, with the same carly stopping pationce as our neural baselines, while we vary the max depth of each tree between 10). All other hyperparameters are left to the default values suggested by their respective official implementations. In Table 5 we show the depth d of the model achieving the best validation accuracy.
Table 5: Parameters used to generate the different baseline architectures for different datasets.
Parsm
Synth-livorr
Syoth-Nomlis
Synth-Nenlin-Larg
Symth-oRRN
Higgs (with high)
Higss (no hight)
PRMC
F1OD
B
1024
1822
1024
1824
2018
2008
1024
21124
1,500
1,500
1,500
1,500
1,500
250
250
230
750
50
250
300
Tymstiin
50
50
30
50
25
25
25
50
Tasts experteed
50
20
25
{16. 16)
{16, 16)
[16, 16)
[128, 64, 64)
{1024, 512, 256, 128)
{ [1024, 512, 256, 128)
[128, 64.
[256, 128, 64, 64)
16
16
16
18
64
60
64
65
{16)
[16]
(16)
(64,
(32, 16)
(92,
(16,
(16)
(2)
(2)
(5)
{11)
(4. B, S:
[4. 6, 8)
(3) 4, 5, 03
(4. 6, S3
2
2
5
11
8
0.1
0.1
0.1
Anie
5
5
10
0.1
0.1
0.1
10
5
5
10
0.1
11.
0.1
1:1
(16, 16)
[16, 16)
(16, 16)
(61,
(900, 300)
[300, 300)
(300, 300)
(300, 300)
Ne
G4
64
32
96
16
812
No
8
64
32
32
16
32
Naupa
[16, 18, 16]
{1G, 16, 16)
[16, 10, 16)
(GL, 32)
(1024, 512 ... 10]
[1024, 512 .. 16)
[16, 16, 16, 16, 16)
192, 32, 32, 16, 16]
5
10
10
25

---

Published in Transactions on Machine Learning Research (07/2023) 2023)
F Predictive Performance Experiment Details
When evaluating the concepl-supervised baselines (i.e., TabCBM, CBM, Hybrid-CBM, and CEM), we vary the number of concopts provided during training as well as the fraction of training samples r == (Pemncopt-mmp that have concept annotations in them. When subsampling only kaclected concepts out of total of k available concepts in a dataset (e.g., k = 2 for Synth-Linert and Synth-Nonlin while it is 5 and 11 for Synth-Nonlin-Large and Synth-scRNA, respoctively), we select a subset of concepts C" C 21,2,... ... .k} of size [C' = kselerted; chosen uniformly at random from all subsets of size kiselected, before training begins. We then provide all concept-supervieed methods with concepts annotations с C {0, 10 fsoleotte for a fraction E [0. 1] of the training samples, where the entries in с correspond to the concepts selected in C². The hyperparameter r decides the probability that sample (x, y) in the training set has annotations c. If a sample is not provided with concept annotations, then we set its contribution to the concept-specifie loss to zero.
(2]
In terms of models used, for CBMs, Hybrid-CBMs, and CEMs we use the same architecture as TabCBM's ø model for their concept encoders (or latent code generator in case of CEM). Similarly, we use the same architecture as TabCBM's f model as the label predictor for all of these models, with the exception that its input layer's size may be altered for CBMs if they are provided with fewer concepts than k. Similarly, ø's output layer's size is extended for CEMs so that. it matches CEM's bottleneck size given by k MCEM (with MCEM being the embedding size of CEM's embeddings). Throughout all of these experiments, we keep the number of concepts discovered by TabCBM to k' = l regardless of how many concepts 0 S Rselected S k were given at training lime. Notice, however, that this implies that CBM's bottleneck size will a always be Rscloctod, which can be highly constrained when Asclected is small (a key limitation of CBMs). Nevertheless, for the sake of providing a fair evaluation against a CBM-like model and TabCBM, we compare against Hybrid- whose bottleneck sizes will always be fixed to k', where k' Rselected of its activations will be unsupervised. This enables Hybrid-CBMs to have much capacity in their bottlenecks as TabCBMs. Finally, we use sigmoidal activation for both the bottleneck of CBMs and Hybrid-CBMs, as done in (Koh) ct al. 2020b) and (Espinosa Zarlenga et al. 2022). and we set CEM's embedding size to 8 and its RandInt parameter to Pint = 0.25 as suggested by Espinosa Zarlenga et al. (2022). For all of these models, including TabCBM, we fix the weight of the concept loss to be sconcept-sup = 5 after trying values in f0.1, 1,5, 5, 10) and finding that the best validation results were obtained with Aconcept-sup in the range (1, 10).
G Extra Qualitative Results in Synth-scRNA
As done for TabCBM in Scction 5.2 we plot the same randomly selected GPEs in Synth-scRNA as in Figure 3 and show the scores of concepts learnt by CCD and SENN that are most linearly correlated with those GFPs. Our results, shown in Figures Ga and 6b highlight that the alignment we observe in Figure 3 between TabCBM's discovered concept scores and ground-truth GEPs is not found for concepts learnt by SENN and CCD. We reiterate that TabCBM was able to attain the alignment shown in Figure without any concept supervision. This indicates that, in contrast with SENN and CCD whose concepts do not seem to align to what one would expect in the Synth-scRNA task, TabCBM can discover concepts that align with human-interpretablee high-level concepts for the downstream task of interest.
H Concept Intervention Experiments
In this section, we complement our results in Section 5.4 by further exploring interventions in TabCBM and other possible models. We first explore the effect that the fraction of annolated samples has on the results observed in Figure Then, we investigate how the concept alignment threshold 3 affects TabCBM's receptiveness to interventions. Finally, we include a comparison between interventions on TabCBMs, CBMs, CEMs, and Slybrid-BBM showing how each of these models' performance varies as we intervene on more concepts.
26

---

Published in Transactions on Machine Learning Research (07 (07/2023)
Cell Type #5
Cell Type #2
Cell Type #6
Cell Type #1
Activity GEP #1
Known GEPs UMAP2
0.40
0.35
0.30
0.25
Cin (p 0.62)
Cg (p 0.38)
0.75)
Go (p 0.57)
č 0.46)
0.20
Learnt Concepts UMAP2
0.15
0.10
0.05
0.00
UMAP1
UMAP1
UMAP1
UMAP1
UMAP1
(a) CCD best aligned learnt concept scores.
Cell Type #5
Cell Type #2
Cell Type #6
Cell Type #1
Activity GEP #1
Known GEPS UMAP2
2.0
1.5
C1 (p = 0.45)
22 (p = 0.53)
č (p = -0.56)
? (p 0.54)
Čg (o = 0.47)
0.5
Learnt Concepts UMAP2
0.0
-1.0
UMAP1
UMAP1
UMAP1
UMAP1
UMAP1
(b) SENN best aligned learnt concept scores.
Figure 6: UMAP visualisations the Synth-scRNA datasot showing the activation a randomly selected selection of ground truth concepts (lop row) and the activation of the learnt concept score by (a) CCD and (b) SENN with the highest absolute correlation with the respective ground truth concept (bottom row).
H.1 Concept Interventions in Partially Concept-Superrisee Setups
In Figures 7a and 7b we show the test. task accuracy when intervening on a TabCBM that was trained with r 25% and r = 50% the training samples provided with concept annotations. We notice that TabCBM appears to be quite robust different levels of concept annotations, performing similarly what we observe in the fully concept-superviseed case in Figure ! where all samples were provided with concept annotations. There seems to be slight drop in accuracy when using only 25% of the concept annotations (see the right-most plot in the bottom More importantly, in all cases that we tested (as we varied the fraction of concept-annotated samples r across (0.25,0) 0.5, 1}) we observe that TabCBM can discover unsupervised concepts which strongly align to concepts not provided during training and that, when intervened on, they can boost TabCBM's downstream task performance.
of Figure 7a however, this does not represent significant drop.
H.2 Effect of Alignment Threshold
As discussed in Section 5.4 an important part of how we intervene on TabCBM's unsupervised concept scores involves building alignment map between these scores and known ground-truth concepts in the task of
27

---

Published in Transactions on Machine Learning Research (2722)
Synth Linear
Synth-Nealnn
Syeth-Noelln Large
Syath-scRNA
528- 10-
192-
COMPAN
1.0
336
102
1.0
concepte Inkervenos
concepts Intervenne
cxazepa Ittervonss
c.nooupps Iatervazed
isugersteed concepts
Isugerstaed concegts
Inspervised con.cepts
(a) Intervening when r 25% of training samples for TabCBM are provided with concept annotations.
SUPERNED (Nnorgoze oxt
Syoth
Syath
Syath Large
Syzth
100
96-
96-
22
328-
307
35
96
COMINN
128-
98-
cuacepta Inkervendd
concepta Inturvendd
Iskurvined
Caterwazed
TAOCRM Isugerstsed concepts
0%
TeBCRM Insgerstard concegts 5476)
TabCBM Imspervised conceett =n 100%)
(b) Intervening when 50% of training samples for TabCBM are provided with concept annotations.
Figure 7: Task accuracy of TabCBMs when randomly intervening on different number of concepts (x-axis) across multiple datasets (columns). Each row in each subfigure has the same semantics its corresponding one in Figure
interest. We argued in the same section that this map could be built by detecting strong linear correlations (which could be both negative or positive) and aligning discovered concept é with ground-truth concept Cj if their empirical absolute linear correlation coefficients are greater than a threshold B.. This threshold indicates how strong of correlation should we look for declare a discovered concept be aligned with ground-truth concept.
In our experiments reported in Figure we fixed 8 to 3 = 0.7, arguing that this indicates what we believe to be strong enough correlation between two variables. Here, we show in Figures Sal and Sb how these results change if we set ,3 to 0.25 and 0.85, respectively. As one would expect, we observe that when / is very low (i.e., B = 0.25) interventions in TabCBM can damaging to TabCBM's performance in complex datasets such as Synth-scRNA. We attribute this to the fact. that the constructed alignment map a may mark even slightly correlated concept scores and ground-truth concepts as aligned whose correlation can be attributed to the complexity and noise inherent to the dataset, rather than to true semantic alignment. Nevertheless, do not observe this in the case of the simpler synthetic datasets TabCBM seems to be lcarning correlation scores whose absolute values are close to binary in nature (i.c., discovered concepts seem to be very strongly correlated to one and only one ground truth concept). In contrast, we observe that if /3 is too large (e.g., 3 0.85), then we lose the ability to exploit slightly weaker correlations during intervention time (as we did when B 0.7 Figure B), resulting almost no discovered concopts identified as being aligned with ground-truth truth concepts in Synth-seRNA. This represents a cost as we know from Figure g that could improve the model's test performance through interventions that arise from weaker correlations.
28

---

Published in Transactions on Machine Learning Research (0722023)
(NneeDOr ONL3
Synth Linear
Synth.Noche
Syelh-Noolnn Large
Syath
100
Sa8-
9G
100
336
302
COMPA
concepte Inkervenes
concepts Intervenne
coazepa
cenoupts Intervazed
Isugerstsed concepts
Isugerstaed concegts
Insperviied con.cepts
Intervening when the strength for identifying aligned concepts is set to / 0.25%.
SUPERNEDD Unnireoo oXt
Syuth Synth
Syath Large
Syzth
100
338
307
1.00
96
1.00
COMEEN
95-
Inkerennd
Inturvenrd
Iskurvened
Lstervazed
TROCRM Isugersteed concepts
0%
TebCRM Insgerstard concegti 5.7%))
TRbCBM Inspervised concepts = 100%)
(b) Intervening when the strength for identifying aligned concepts set to 0.85%.
Figure 8: Task accuracy of TabCBMs when randomly intervening on different number of concepts (x-axis) across multiple datasets (columns). Each row in each subfigure has the same semantics its corresponding one in Figure
In summary, these experiments suggest that one can think of a parameter indicating how much risk we are willing to take when intervening with TabCBM. If it is very high, then will only intervene concepts that we are highly confident are aligned with ground-truth concepts, resulting in likely intervention gains observed throughout our simpler synthetic datasets. On the other hand, if B is very low, then we may incorrectly align a discovered concept with a ground truth concept which, when intervened on, may decrease the end performance of TabCBM duc to misaligned scmantics. Hence, we select & 0.7 as a value that leverages this risk by taking advantage of possible strong correlations for improving test-time interventions, while avoiding misidentifying alignments that can lead to detrimental interventions.
H.3 Intervention Comparison Across Cancept-Suuperisee Methods
In Figure we compare the effect that interventions have on the test accuracy across multiple concept- supervised methods (CBM, Hybrid-CBM, and CEM) for Nymth-Nonlin-Large and Synth-scRNA the number of concepts provided during training between 50% of all concepts (top row) and 100% of all con- cepts (bottom row). We show our evaluation only on these datascts as, from our pool of concept-annotated datasets, they have enough concepts (i.e., 5 and 11 concepls, respectively) for us to study interesting be- haviours emerging when we intervene on a varying number of concepts. As Figure 4 this study, we annotate all r 100% of the training samples with the selected concopt annotations and we use 0.7 when intervening on unsupervised concepts with TabCBM. The training setups for CEM, CBM, and Hybrid- are the same those used in Section 5.1 Notice that because TabCBM allows for interventions in unsu-
vary
29

---

Published in Transactions on Machine Learning Research (07 /023)
Synth
Synth-Nonling Lonth-Nonlin-Large
100
100
50% Cuncepts
Task Accuracy
80
80-
60
60-
40
40-
20-
20
0
100.0
100
100% Sup Concepts
Task Ancuracy(16)
97.3
90-
80-
02.5
70
87.5
50
40
82.5
30
0
2
10
0
3
Concepts Intervened
Concepts Intervened
X
TabCBM
CBM
CEM
Hybrid-CBM
Figure Task accuracy of different concept-supervised baselines in the Synth-Noolin-Larg and Synth- scRNA tasks we increase the number of concepts intervened testing time. During training, we provide 50% of available concepts (top row) and 100% of available concepts (bottom row). For TabCBM, we include interventions both on supervised and unsupervised concepts.
pervised concepts, we can intervene on more concepts for this model than for the rest of the basclines (only applicable when the number of training concepts is less than the number of known ground-truth concepts in the dataset). Furthermore, we point out that because Hybrid-CBM is the same as a CBM when using all available concepts (there is no need for extra capacity in this case), do not include this baseline in the results when intervening in models that were given training supervision for all concepts (bottom row figures).
Our results show that TabCBM can perform significantly better than the selected basclines in concept- incompleteness setup (i.e., when the number of training concepts is less than the number of ground truth concepts), as seen by the results displayed Figure D top row. These results suggest that TabCBM can be an effective model to be used in real-world scenarios when it is likely that the set of concept annotations available at training time is incomplete with respect to the downstream task (Espinosa Zarlenga et al. 2022 When facing concept-complete setups, shown in Figure D bottom row where all required concepts for task are given as training supervision, we see that TabCBM fares better or competitively against basclines. An exception for this can be found only in Synlh-sc/RNA, where TabCBM falls behind CBMs when the number of interventions is small. We believe this to be the case because, given the mutual exclusiveness of Synth- scRNA's identity GEP concepts, CBM can properly capture this property better than TabCBM as does not have any extra constraints such as TabCBM's mask discovery. Nevertheless, observe that as soon few concepts intervened on, TabCBM's performance becomes similar to that of CBM in this dataset.
Hyperparameter Ablations
In this section, we explore TabCBM's sensitivity to its hyperparameters by studying how they affect dif- ferent metrics and propertics of the resulting model. We first discuss how W, the number of discovered concepts, affects TabCBM's accuracy and quality of learnt representations. Then, we explore how the differ- ent. regularisation terms for TabCBM's loss affect its end performance. Finally, we conclude by giving some
30

---

Published in Transactions on Machine Learning Research (07/2023)
recommendations on what are some good values for these regularisers that, from experience, work well in practice without much fine-tuning.
1.1 Effect of the Number of Discovered Concepts on TabCBM
Selecting the number of discovered concepts for TabCBM is one of the most important parts of constructing model that is high- high-performing while maintaining high level of interpretability. While setting this value can be strongly aided by domain-specifie knowledge (e.g., setting = k when we know the value of k a priori), it is often the case and expectation that TabCBM will be trained on a task in which the true number of ground truth concepts is unknown. However; in Figure 10 we show that TabCBM's performance, is relatively robust changes in its k' parameter. We observe that only when it is severely sppacity-constriing as it is the case when k' = 1 in both Synth-Lincar and Synth-Nonlin, do observe significant drop in its accuracy.
Synthhunii
Sgrtth-scRNNA
Hap high lecel
Figure 10: Test accuracies for TabCBM as we vary the number of discovered concepts V for all our tasks.
Although similar behaviour as that obscrved with the task accuracy when vary the value of K can be seen for the G-AUC metric, the same cannot be said of the M-AUC metric. We see this in Figure 11 where we observe that TabCBM can correctly identify all globally important features regardless of W, provided it is not under-parammterised,, resulting in high G-AUC value for all variants evaluated. In contrast, we observe that the M-AUC metric seems to decrease as the value of k increases (particularly in the more complex synthetic datasets). We hypothesise that this the case because in both Synth-NNonlin-Tayy scRNAthe ground-truth taund concept masks of some concepts are overlapping (c.g., a GEP's set of differentiatod genes can overlap with the set of differentiated genes for another GEP). Therefore, when using smaller k', TabCBM's objective function can be minimised by fusing some of these concepts' masks and scores into single concept. This leads to a higher M-AUC score than when k' is larger because TabCBM does not need to correctly disentangle such fused concepts as it has to do when k' is larger.
and Synth-
1.2 Effect of TabCBM's Regularisers and General Recommendations
Besides k', the next set of hyperparameters of importance for TabCBM is its different regulariser strengths Deo. Adiv, Aspor). These control how much we value concept coherence, concept diversity, and concept speci- ficity, respectively, with respect to downstream task accuracy. In practice, when evaluating TabCBM we explored different combinations of these hyperparameters by varying them the set 1,5,, 10). We observe that, although TabCBM's performance seems be relatively robust to fluctuations in these pa- rameters, the specificity strength is particularly important to fine-tune for TabCBM to correctly identify and discover concepts it was not given supervision for. To see this, in Figure 12 we show how the test task accuracy of TabCBM in the Sonth-Nonlin-Larg dataset changes as we modify all three regulariscrs while fixing the others to the paramelers defined in Table 5 for this dataset. These results show that test accuracy fluctuates little for changes in both Aco and Adiv: but is sensitive to changes in depec- This is because there scems to be a threshold for Agpec which TabCBM puts more of its attention into discovering masks first, and then uses these masks simplify the problem of learning the downstream tasks. This helps in this particular example as by first finding meaningful concept masks, TabCBM eliminates lot of noise and
31

---

Published in Transactions on Machine Learning Research (07 (07/2023)
Synth-
Syath-Nonlin
Synth-Nonlin-Laue
Synth-scRNA
M.AUC
04
0.4
0.4
62-
0.2
0.2
5.
10
0.7
0.5
G-AUC
6/8
0.5
0.0
64-
0.1
0.4
0.3
0.2
62
02
10
K'
Figure 11: Test G-AUC and M-AUC scores for TabCBM vary the number of discovered concepts k' for all our tasks that have concept annotations.
redundancy in the data in the same way that feature selection methods do; hence improving the learning dynamics after these masks have been discovered.
Task Task Accuracy
0.6
0.5
Accuracy
0.4
0.4
6.4
0.2
0.2
0.0
0.0
Aca
Adv
Aspcc
Figure 12: Test task accuracy of TabCBM in the Synth-Nonllnn-aryy dataset vary its loss regularisers
(Neor Adivy } during training.
Recommendations for loss hyperparameters Our results above, together with our experience with TabCBM during evaluation, seem to suggest that the parameter that requires the most finc-tuning during training is Aspee- In practice, we observed that setting X = 0.1 while searching for values of Adiv and Aspec in the set {0.1, 1,5, 10} under the constraint Adiv = Aapec yielded good results. If such search is too expensive, then we found that sctting Adiv = Aepec = 5 resulted good performance across all of our tasks without much fine-tuning.
J Differences Between Scoring Functions
As discussed Section in this work we opt to use as our concept scoring function ( the sigmoidal unnormalised inner product between ((x(1) and (x(0)) We chose this scoring function rather than other commonly used alternatives such as the cosine similarity (used in CCD (Yeh et al., 2020 for two main to that its domain (i.e., [-1,1]) is clamped between 0 and (as these vectors are
reasons. First, using the cosine similarity for generating concept score between and requires threshold the similarity not guaranteed to be non-negative). This is done in CCD by selecting all similarity scores below some hyperparameter 3 to be clamped at O. While this enables one to have magnilude--ndeeeedeee scores, it introduces the need for an extra hyperparameter and leads to a gradient- blocking operation, forbidding gradients to backpropagate
the vector-generating modcls á and p when their concept scores are clamped zero. Second, as shown in Table 0 for Synth-Nonlin-Larte in practice we observe that using thresholded cosine similarities rather than our proposed activation function in TabCBMs lead to drops in downst
32

---

Published in Transactions on Machine Learning Research (07/2023)
performance. We hypothesise that this is the help it more easily express concept's 's activation or ivatiivation after using sigmoidal activation, something that normalised vectors may not be able to do. Because of these reasons, we leave the exploration of further scoring functions for future work and use the sigmoidal inner product in this paper.
model can utilise the magnitude of these vectors to
Table 6: Differences in task accuracy and concept alignment scores (CAS) when using for TabCBM's concept scoring function s(t) (1) a 3-thresholded cosine similarity, and (2) sigmoidal dot product function. For simplicity, we show only results on the Synth-Nonlin-Large dataset and all TabCBMs are trained without any concept supervision by setting k' = k and using the same hyperpameters as those described in Table
Method
Task Accuracy (%)
CAS (%)
TabCBM (sigmoidal inner product)
62.78 + 1.13
88.54 + 4.49
TabCBM (thresholded cosine similarity with 8 == -0.5)
53.38 + 2.01
74.73 + 2.15
TabCBM (thresholded cosine similarity with 3 = -0)
53.48 + 1.87
75.19 + 1.96
TabCBM (thresholded cosine similarity with 3 = 0.5)
53.53 + 1.47
74.87 1.57
K
Interpretablliy-acccraay Trade-off in TabCBMs
our experiments exploring the task accuracy of concept-supervised TabCBMs, shown in Figure 2 we observe that TabCBM's task accuracy generally does not significantly vary when concept supervision is introduced. Nevertheless, we an exception is seen only in the Synth-scRNA task where noticeable decrease in task accuracy is introduced soon as concept supervision is provided to TabCBM. This suggests that interpretabilit-acccree trade-off may exist when concept supervision is introduced in complex tasks. In this section, we take deeper look at this trade-off and show that, although it does indeed affect TabCBM in complex tasks, our models are still able to obtain competitive performance with respect to black-box basclincs.
In Table we summarise the performance of TabCBM across all synthetic tasks before and after receiving concept intervention. Our results suggest that its worst, TabCBM drops around ~7% in mean aver- age performance in Synth-scRNA compared to equivalent unsupervised version (i.c., a TabCBM trained without any concept supervision). Nevertheless, these differences are not necessarily significant ances are relatively large because of their high sensitivity to which concepts are provided with supervision. More importantly, when all concepts are provided supervision, we see drop of only <-3% in task accuracy. Such a drop is not significant when one considers that even with ~3% less accuracy than its unsupervised version, TabCBM outperforms black-box models such as ML.Ps and falls behind other black-box models such as TabNet and XGboost by ~0.5%. These negligible drops in performance, however, come with the increased benefit of TabCBM being able to generate faithful concept explanations for its predictions and being significantly more receptive to dest-time interventions that can boost their performance way above that of black-box models (as seen in Figure 4 Finally, this tradc-off may be corrected in practice by decreasing Aconcopt-sup during training, although this may lead to less accurate concept explanations.
the vari-
Table 7: Accuragy-interppetaalltty trade-off: effect of adding concept supervision to TabCBM across all synthetic datasets. We show the task accuracy (%) obtained as we change the number of supervised concepts given to the TabCBM during training. For reference, we include MLPs and TabNets as baselines to compare our model against.
Method
D&CRM (0 Sup Conorpts)
TebCBM Sup Concepta)
TalCBM (50% Sup Concepts)
TeLCRM (100% sup concepta)
ToloNes
Liwer
88.51
09.51 0.37
97.57
0.37
97.57
0.97
Syuth Nonlin
96.58 1 0.23
96,71
0.52
94.71 1. 0.52
$4.62
1.01
87.65
0.98
91.57
1. 0.48
Noulnn-
65.17 +:
65.30
63.27 0.79
65.51
1178
01.74
6.42
2.07
a
83.06
$7.40
L Details on representation learning metrics
In Table we provide brief summary of the metrics used in our experiments in Section 5.2
33

---

Published in Transactions on Machine Learning Research (07 2203))
Table 8: Summary of the metrics used as part of our evaluation in Section 5.2
Metric
Definition
Concept Alignment Score (CAS)
This metric measures how strongly learnt concept score è is aligned with corresponding ground-truth concept cy. It does this by clustering test samples based on è and measuring the homogeneity (Rosenberg & Hirschberg, 2007 (i.e., the coherence) of these clusters with respect to the ground-truth labels of ey. To compute this metric for a set of learnt concepts, we first match each learnt concept score with the ground truth concept with the highest absolute Pearson correlation and then average the homogeneity scores produced for each matching. A high CAS score of represents perfect alignment between learnt concepts and ground-truth concepts. A low CAS score of O indicates perfect misalignment.
Espinosa Zarlenga et al. 2022
Mutual Information Gap (MIG) Chen el al. 2018b
This metric provides quantilative mensurement of how disentangled, diverse, a set of learnt concepts is. It is computed by measuring how much more informa- tion about ground-truth concept G is encoded in the learut concept & with the highest mutual information [((g;e] than in the learnt coucept é with the sec- ond highest mutual information [(Gs; c,). If the mean gap across all ground-truth concepts is large, then it implies that each ground truth concept is being encoded by single learnt concept. Otherwise, it implies that multiple learnt coucepts encoding redundant information about a ground truth concept.
Rd
The R' metric measures whether there exists a bijective aligument function be- tween learnt concept scores and ground-truth concept labels. Therefore, al- templs to capture whether learut concepls are coherent and diverse enough to completely capture all ground-Iruth concepts. This metric operates by traiming two nonlinear univariate models for each (learat-coucep,, ground-truth concept)- pair, one model for cach direction, and using their coefficients of determination It to quantify how closely can these two variables be represented using a bijective mapping. final score is then produced by averaging the maximum H² values obtained for all ground-truth concepts. A high Rt score indicates the existence of a bijection between learnt concept scores and ground ground-truth concepts.
Ross& & Doshi Velez. 2021
DCI Disentanglement
This metric captures concept diversity by measuring whether each learnt concept is aligned with at most one ground truth concept. It computed by averaging the complement entropies of the probabilities that a learnt concept & is rendered "im- portant" when predicting ground-truth concept cj. The importance scores used to construct such a probability distribution are computed using a simple linear regression model between every learnt concept and every ground-truth concept. A high DCT disentanglement indicates that all learnt concepts capture one and only one ground-truth concept.
Eastwood Williams 2018a
DCI Completeness
When the set of ground-trui truth concepts is fully descriptive of the downstream task, this metric captures concept completeness by measuring the degree to which each ground-trut concept is captured by at least one learnt concept. It estimates this by computing the complement entropy of the probability of learnt concept & being the only "important" learnt concept to predict ground-trute concept & (with importance computed as above). A high DCI completeness indicates that all ground-truth concepts are captured by one and only one learnt concept.
Eastwood Williams/ 2018a
DCI Informativeness
This metric captures another aspect of concept completeness by looking at how predictive the overall set of learnt concept scores of each known ground-truth concept. This is measured using the average prediction error obtained when training a simple classifier to predict each ground concept from è. A high DCI informativeness indicates that all ground-trun truuh concepts can be accurately predicted using the set of learnt concept scores.
Eastwood & Williams/ 2018a
34

---

Published in Transactions on Machine Learning Research (07/2023)
M Code Used and Hardware Infrastructure
For the experiments reported in this paper, we built a code base on top of the code made available by Es- pinosa Zarlenga et al. (2023 and by Kazhdan et al. [021 (under an MIT and Apache 2.0 licenses, respec- tively) to train the CCD, SENN, and CBM baselines. For both the CEM and Hybrid-CEM baselines, we used the official implementation by Espinosa Zarlenga et al. (2022). For both XGBoost and LightGBM, we made use of the official implementations of both frameworks. For TabNet we extended the open-sourcod MIT-licensed implementation by DreamQuark A Similarly, for TabTransformer extended the open sourced MIT-licensed implementation by Phil Wang
We built our code base using combination of TensorFlow (Abadi et al. 2016) and PyTorch (Paszke et al. 2019) and implemented TabCBM in TensorFlow. All of the code needed to reproduce our results, and use our model through a simple API, has been made public at an MIT license.
/mttooospppiionn//tbbbbb via
35


Reviews:
[{"review_id": "WwCaXNYZ9p", "reviewer": "Reviewer_mG3r", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strength\n\nA rich amount of empirical studies is the greatest strength for this work. I appreciate the authors’ efforts on quantitative evaluation for the proposed work over the baselines. Results in Section 5 have answered the 4 research questions introduced in the beginning. Especially, I find the results in concept score alignment interesting and I am surprised to see the model learns aligned scores in Figure 3 without supervision. \n\n\n### Weakness\n\nI have a few concerns for motivation, the training and the presentation of the current manuscript. \n\nFirst of all, I have to admit I am not convinced that having this definition of “tabular concept” helps us to build models that are more interpretable. From my reading, this definition of (tabular) “concept” means some groups of input features are “concepts”, justified in one dataset where some features together are regarded as “delinquency” and can be used without other features to predict the default. Concept bottleneck model is a good fit for image data as pixels are not the unit for humans – we do not see pixels and we see hands, wings, etc. However, most tabular data has features with very concrete meanings, e.g. credit scores, income, heights. As an engineer with a computer science background, they are already “concepts” to me. However, grouping these features, without introducing domain knowledge in detail, are not creating a more interpretable “concept” to me. Transferring CBM ideas to tabular needs to answer two questions that: 1) given most tabular input features are already semantically meaningful enough and why one needs some “meta-concepts”; and 2) do these “meta-concepts” make sense to domain experts. \n\nSecondly, this training objective is very complex and I have no idea how many efforts one needs to input to do parameter turning. The authors are very ambitious to include all properties of the learned concept vectors into the loss. As a result, every property becomes a regularization term. Moreover, there is another hyper-parameter to balance between concept supervision and unsupervised learning. My experience in training both small and large models tells me that without that amount of hyper-parameters, it requires so much effort in tuning and sometimes the result is just an overfitting to some set of hyper-parameters. I think hyper-parameter searching processes and bags of training tricks need to be presented (maybe it is in the appendix already and please forgive me if I have missed the pointers). If you ask me for advice on this, I would decrease the number of regularizations and think about what are fundamentally useful regularizations.\n\n\nLastly, the paper is too dense in the empirical part (and probably in a bad way). The authors are ambitious (which is good!) to answer 4 research questions that can quantitatively the proposed method. However, many metrics are just citing references and do not define them in the main paper. Moreover, I am lost many times and asking myself why they are evaluating this and what I should expect when looking at these tables. In addition, I do not understand why these metrics show that TabCMB is more interpretable if we do not have ground-truth “meta-concepts” at hand, which I assume is often the case. Can you evaluate mostly on real datasets and provide more examples & visualizations to explain what are learned “meta-concepts”? \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "\nIn this paper, a new type of neural network, TabCMB, is proposed to train networks that capture “concepts” in tabular data and is able to ground its predictions on these “concepts”. The major contributions include the architecture of TabCMB itself, ways to train TabCMB and empirical studies showing TabCMB outperforms baseline works on the dimension of  “interpretability” for the task at hand. \n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "My recommendations are actually included in the weakness section. In summary, \n\n1) I want to see a concrete example of \"meta-concept\" found by the TabCBM network, what does that mean from domain experts' perspective and is that more interpretable to them. Maybe defining \"more interpretable\" is helpful to convince the audience this model is indeed more interpretable than just throwing things into a tabular transformer. \n\n2) The presentation of the empirical study should be improved to have focuses, clear definitions and metrics, and what to expect from the results. \n\n", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hmXXlUFThP", "reviewer": "Reviewer_ez7U", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\nThe paper is overall in a very good shape.\n\n- Concepts in tabular data are well motivated on previous work.\n  Thus, TabCBM is well motivated, relating to and building on previous work on\n  feature selection and concept embedding.\n\n- The desiderata of concepts in tabular data motivate the objective and\n  follow-up methods well.\n\n- The research questions in the beginning of the extensive experiments section\n  provides a good overview.\n\n- For all experiments, there are at least from two (CCD and SENN in the\n  unsupervised case), to 3 (CBM, CEM, Hybrid-CBM in the supervised case)\n  up-to-date baselines for which TabCBM is compared to.\n\n- The supplementary is very extensive and answers many questions, providing a\n  good foundation for high reproducibility.\n\n\n### Weaknesses\n\n### Method\n\n- The concept score is the sigmoid of the inner product of a concept vector and\n  a latent encoding, as opposed to CCD where the score is the cosine\n  similarity. Using the sigmoid of the inner product without normalization has\n  two weaknesses:\n  - The amplitude of the latent code/ concept vector influence the\n    activation stronger then the respective direction.\n  - Using a sigmoid activation on the inner product will result in a score\n    of 0.5 when the directions are orthogonal and approach 0 when they are\n    parallel with a different sign.\n\n- The training of $\\rho$ and $\\phi$ lacks some explanation. Are they trained in\n  parallel with the same concept objective?\n\n- The desiderata, while providing a good motivation, would be better to relate\n  to previous work.\n\n### Experiments\n\n- With the exception of the scRNA dataset, all concept related experiments are\n  only conducted on synthetic data. This does not provide a lot of information\n  on how useful TabCBM's concepts are on real world data.\n  It is hard to say how much semantic meaning is carried by the unsupervised\n  concepts. This is somewhat addressed by the intervention experiment, where it\n  is also discussed that there are only few concepts in scRNA aligned with\n  ground truth concepts. This is in important difference to the synthetic data,\n  and should be at least discussed in greater detail. It would be much better\n  if the experiments would include another realistic dataset with ground-truth\n  concepts, although I understand this may be hard to come by.\n\n- The interpretability trade-off of TabCBM is not discussed. TabCBM loses\n  accuracy when using supervised concepts, i.e. in scRNA Fig. 2a. How does it\n  perform in its worst case with supervised concepts when compared to\n  (non-interpretable) models?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work introduces TabCBM, a Concept Bottleneck Model for tabular data, which\ncombines a feature selection approach based on (Lee et al., 2021) to learn a\nconcept-level feature mask, with a concept embedding approach based on (Yeh et\nal., 2020).\nSpecifically, TabCBM produces a set of concepts that consist of a feature mask\nas well as a concept score function.\nA motivation for concepts in tabular data is provided, as well as a set\nof desiderata for meaningful concepts, which is subsequently used to motivate\nthe training objective of TabCBM, as well as the evaluation metrics in the\nfollowing experiments.\nThe performance of TabCBM without concept supervision (the a-priori knowledge\nof concepts) is empirically compared to 7 other models on 4 toy datasets and 3\nreal-world datasets, as well as 3 other models with concept supervision on the\nsame 4 toy datasets, on which TabCBM shows equal to superior accuracy on all\ndatasets.\nThe alignment of the identified concepts of TabCBM is empirically shown as\nsuperior compared to 3 baselines, including PCA, using 6 metrics from the\nliterature, related to the declared concept desiderata. Additionally, the\nalignment is qualitatively compared on one dataset, showing good alignment of a\nfew concepts, which is shown to not be the case for two other baseline concept\nlearning methods from the literature.\nIt is empirically shown that TabCBM's feature mask provides a global feature\nimportance, using 4 toy-dataset with known ground-truth masks.\nThe work also provides an empirical evaluation on the intervention of concepts\non a toy dataset, showing how the correction of a few concepts, regardless of\nwhether they were known a-priori, can increase the model accuracy of TabCBM.\nThe work concludes with conceptionally relating TabCBM to group feature\nselection, as well as discussing limitations of TabCBM, specifically the\ndifficulty of the selection of the number of concepts, computational growth\nwith a high number of concepts or the dimensionality of concept embeddings, and\nits architectural complexity and high number of hyper-parameters.\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "### Method\n\n#### Major (non-critical)\n\n- Address the weaknesses above about the concept score.\n\n- Explain the training of $\\rho$ and $\\phi$ as pointed out above.\n\n- Relate the desiderata to other work.\n\n#### Minor (non-critical)\n\n- Desiderata 1. Should be pointed out to be task specific, and it should relate\n  to the task in Eq. (1)\n- Desiderata 3. It should be pointed out here that the similarity is probably\n  with respect to some latent similarity.\n\n- $\\Psi$ is defined as the t-nearest-neighbors, but the metric is only implied to\n  be L2. This should be added.\n\n- In Eq. (1), the inner product should be normalized, otherwise this term may\n  only reduce the amplitude of the concepts, same for Eq. (2). This should be\n  either discussed, or changed.\n\n### Experiments\n\n#### Critical\n\n- Address the lack of realistic data and the alignment therein of unsupervised\n  TabCBM concepts with ground-truth concepts, or explicitly disregard the\n  interpretability claim in real-world settings without concept annotations.\n\n- Address the interpretability trade-off with supervised concepts.\n\n#### Minor (non-critical)\n\n- The number of trials for the experiments should be provided.\n\n- With respect to the performance, in the end of 5.1 paragraph \"Concept\n  unsupervised\" it is stated that *\"even without concept supervision, TabCBM\n  does not sacrifice the performance seen in state-of-the-art black-box models\"*.\n  My intuition would be that the trade-off is stronger *with* supervision, as\n  indicated by Fig. 2a, thus, this statement is misleading.\n\n- Fig. 3 could use some more explanation, especially for Cell Type 5/$\\hat\n  c_9$, which has its sign flipped.\n\n\n### Miscellaneous (non-critical)\n\n- In the Eq. below Fig. 4, maybe pre-define the argmax index to improve readability\n\n- In the Eq. below Fig. 4, maybe add directly after the percentiles that they\n  represent the positive/negative activations of the concept within the\n  training set\n\n- Fig. 2 is a little bit too small.\n\n- It may be beneficial to label all equations to make them easier to reference by readers\n\n- It would not hurt to write down $\\mathcal{L}_\\text{concept-sup}$\n  explicitly instead of describing it.\n\n- Use a full-line equation for the definition of $s^{(i)}$\n\n- The scatter plots take a long time to load, consider using a raster graphic format, e.g. PNG\n\n- Fig. 4 final sentence: ...regardless of whether the[ir -> y] received training...\n", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fPHjFZSHbB", "reviewer": "Reviewer_NmM7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength:**\n- The paper addresses the important topic of interpretable deep models and, even more importantly, networks that allow for human interventions. \n- The paper is very well written and easy to follow, especially the description of the original CBMs, the definition of tabular concepts, and the introduction of TabCBM.\n- Experiments seem to be well conducted, and TabCBMs are exhaustively on synthetic and real-world data, including ablations and the application of interventions. \n- The paper is well describing the limitations of future work.\n\n**Weaknesses:**\n- Limited novelty since the contribution is mainly the application of CBMs to tabular data and tasks. However, the paper provides important insights. Therefore this weakness is negotiable. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The present paper applies concept bottleneck models to tabular data. To this end, first, the authors define what concepts are in the domain of tabular data. \nWhich is the interaction between relevant features, including their activation. The proposed method includes, next to the original concept bottleneck, which was originally introduced in the vision domain, a concept generator to not only rely on supervised concept signals but also provide see ability to unsupervised train such a network. The proposed methodology is evaluated on synthetic as well as real-world tabular data, also highlighting human interventions, which are, next to interpretability, an essential motivation of concept bottleneck models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In general, the present paper seems to be ready for acceptance. The presentation of Fig. 2 can be improved by increasing the font size of the plots' labels. Further,  in some cases, TabCBM seems to outperform other approaches by a large margin (Synth-Nonlin). Can you further elaborate on that?\n\n", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)