Relational Concept Bottleneck Models

Authors: pietro  barbiero
francesco  giannini
gabriele  ciravegna
michelangelo  diligenti
giuseppe  marra
k u leuven
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 15:16:51

Abstract: The design of interpretable deep learning models working in relational domains poses an open challenge: interpretable deep learning methods, such as Concept Bottleneck Models (CBMs), are not designed to solve relational problems, while relational deep learning models, such as Graph Neural Networks (GNNs), are not as interpretable as CBMs. To overcome these limitations, we propose Relational Concept Bottleneck Models (R-CBMs), a family of relational deep learning methods providing interpretable task predictions. As special cases, we show that R-CBMs are capable of both representing standard CBMs and message passing GNNs. To evaluate the effectiveness and versatility of these models, we designed a class of experimental problems, ranging from image classification to link prediction in knowledge graphs. In particular we show that R-CBMs (i) match generalization performance of existing relational black-boxes, (ii) support the generation of quantified concept-based explanations, (iii) effectively respond to test-time interventions, and (iv) withstand demanding settings including out-of-distribution scenarios, limited training data regimes, and scarce concept supervisions.

Main paper: 
 claims: 
 Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The claimed contributions in the abstract and introduction reflect the main contributions of the paper in terms of model definition, analysis and experimental results. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
background: 
 Concept bottleneck models. A Concept Bottleneck Model (CBM) is a function composing: (i) a concept encoder g : X → C mapping each entity e with feature representation x e ∈ X ⊆ R d (e.g., an image) to a set of k concepts c ∈ C ⊆ [0, 1] k (e.g., "red","round"), and (ii) a task predictor f : C → Y mapping concepts to a set of m tasks y ∈ Y ⊆ [0, 1] m (e.g., "apple","tomato"). Each component g i and f j vehicle the prediction of the i-th concept and j-th task, respectively.Relational languages. A relational setting can be outlined using a function-free first-order logic language L = (E, V, P), where E is a finite set of constants for specific domain entities1 , V is a set of variables for anonymous entities, and P is a set of n-ary predicates for relations among entities. The central objects of a relational language are its atoms, i.e. expressions p(τ 1 , . . . , τ n ), where p is an n-ary predicate and τ 1 , . . . , τ n are constants or variables. In case τ 1 , . . . , τ n are all constants, p(τ 1 , . . . , τ n ) is called a ground atom. Examples of atoms can be male(Bart) and parent(u, v), with Bart ∈ E and u, v ∈ V. Given a set of atoms Γ defined on a joint set of variables V = {v 1 , . . . , v n }, the process of applying a substitution θ V = {v 1 /e 1 , ..., v n /e n } to Γ is called grounding, i.e. the substitution of all the variables v i with some constants x i , according to θ V . For example, given Γ = [parent(v 1 , v 2 ), parent(v 2 , v 3 )] and the substitution θ = {v 1 /Abe, v 2 /Homer, v 3 /Bart}, we can obtain the ground list θΓ = [parent(Abe, Homer), parent(Homer, Bart) ]. The set of all the ground atoms of a relational language is called its Herbrand base (HB). Logic rules are defined as usual by applying logic connectives {¬, ∧, ∨, →} and quantifiers {∀, ∃} on atoms.Graph neural networks. The architecture of a typical GNN for node-classification tasks consists of three primary steps. For every node i, 1) an incoming message M j→i is passed from a neighbor node j ∈ N (i) to i, where N (i) denotes the set of all the incoming neighbours of i, 2) the embedding representation of node i is updated by aggregating all the incoming messages from its neighbors, 3) a readout function is applied to the node embeddings to predict the class label ŷ(i). Steps 1)-2) are typically repeated multiple times to allow multi-hop information propagation.Question: Does the paper discuss the limitations of the work performed by the authors?[Yes] Justification: This is done in the discussion section (6) . Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
relational concept bottleneck models: 
 This work addresses a key research question: how can we bridge the gap between the interpretability of concept-based models and the reasoning capabilities of relational DL? To answer this question, we extend the notion of bottleneck to a relational setting (Section 3.1) and classic message-passing to also update atom predictions during the recursive steps (Section 3.2). Then we illustrate the learning problem that R-CBMs can solve (Section 3.3), and finally we discuss the connections of R-CBMs with both standard CBMs and GNNs (Section 3.4).The structure of the relational concept bottleneck can be defined as a relational structure, where each atom corresponds to a node. The dependencies among the atoms are represented as a directed hypergraph, where each hyperedge is positional, and can have multiple nodes as head, but only a single node as tail. In this regard, every hyperedge defines a relational concept bottleneck from (possibly) many source ground atoms to a destination ground atom. Moreover, each hyperedge is assumed to be sufficient to carry out the prediction for the destination atom, which can however be collectively improved by merging separate predictions. Formally, an atom dependency graph for a node A, is a positional, labeled hypergraph H = (HB, R), whose nodes are the atoms in the Herbrand Base HB of a relational language, and each hyperedge r ∈ R is such that r = ([A 1 , . . . , A m ], [A]), with A 1 , . . . , A m , A ∈ HB, meaning that in H there is a hyperedge with source [A 1 , . . . , A m ] and destination A. Each hyperedge is labelled with a type identifier l(r). Given an atom A, we indicate by R(A) the set of hyperedges with destination A, and by N r (A) the source of the hyperedge r if r ∈ R(A) or the emptyset otherwise. Figure 2 shows an example with two hyperedges with destination p 4 (b), wherex b x a x c p 1 (b, a) p 3 (b) p 2 (a, b) p 1 (b, c) p 2 (c, b) p 2 (a, c) p 3 (c)N orange (p 4 (b)) = [p 3 (b), p 2 (a, b), p 1 (b, a)] and N violet (p 4 (b)) = [p 1 (b, c), p 2 (c, b)],we used different colors to identify different hyperedges.Relational Concept Bottleneck Models (R-CBM) merge CBMs and GNNs into an interpretable relational setting. An R-CBM first processes the atoms of a relational language by an encoder, and then map them by a predictor (like a CBM). The final prediction is computed by aggregating all the ones from separate groups of neighbour atoms, according to a given dependency graph (like a GNN). The pipeline of R-CBMs can be described as follows: (i) the atom encoder and predictor embeds each atom into a concept embedding and prediction score, respectively (ii) message-passing is performed to refine the embeddings and scores according to the structure defined by the atom dependency graph, and (iii) the atom predictions are obtained by aggregating the predictions.Atom encoder. An n-ary ground atom A is defined by a predicate p and a tuple of entities e = (e 1 , . . . , e n ), such that A = p(e). Like in a GNN, the entities have a feature representation x e = (x e1 , . . . , x en ) ∈ R d•n , being d the representation size. For each atom A = p(e), the atom encoder g p computes the atom encoding g p (x e ) ∈ R H , being H the embedding size.Message-passing. Given the relational concept bottlenecks, the updating of the embeddings and predictions of the atoms can be expressed as a message-passing schema over the dependency graph. For each A ∈ HB, with A = p(e), the initial embedding and prediction for A are calculated by:h 0 (A) = g p (x e ),y 0 (A) = s(h 0 (A)) where g p is the atom encoder and s : R H → [0, 1] is a learnable predictor function working on the local (non-relational) embeddings, such as an MLP with sigmoid activation function. Assuming the message-passing is running for T time steps, for every r ∈ R(A), 1 ≤ t ≤ T , we have the updates:h t r (A) = u l(r) h t-1 (A), h t-1 (B) B∈Nr(A) y t r (A) = f l(r) y t-1 (A), h t r (B), y t-1 (B) B∈Nr(A) h t (A) = r∈R(A) h t r (A) y t (A) = r∈R(A) y t r (A)where u l(r) and f l(r) are edge-type specific functions implementing, respectively: a combine/update operation that provides a refined latent representation h t r (A), and a local readout operation that provides a candidate prediction based on a single neighbourhood h t r (A). The operator aggregates the predictions over all the neighbourhoods r ∈ R(A), e.g, by maximum or summation, whose selection criterion for interpretable models will be discussed in Section 3.4.Relational task predictor. The final task prediction for the atom A is given by y T (A), via the combination of the aggregation function and the local readout f l(r) at time T . We note that this formulation unifies and extends the role of the task predictor in CBMs and of a readout function in GNNs. In practice, each task predictor f l(r) can be implemented by any blackbox-like function, like the one used in GNN architectures [29] or CBM-Deep [13] . However, an interesting alternative is to use either a partially interpretable function, like in linear CBMs, or a fully interpretable function, like in Deep Concept Reasoners (DCR) [1] , which constructs a logic rule combining the predictions of the incoming atoms. Further details on selectable task predictors are in App. A.4.Example 3.1. Given a local neighbourhood for the atom grandparent(Abe, Bart), such that N r (grandparent(Abe, Bart)) = [parent(Abe, Homer), parent(Homer, Bart), parent(Homer, Lisa)] a non-interpretable f l(r) can compute the prediction y T r (grandparent(Abe, Bart)) based on the neighbourhood. An interpretable f l(r) , like the one used by DCR, can also provide an explanation for the prediction, like e.g. parent(Abe, Homer) ∧ parent(Homer, Bart) → grandparent(Abe, Bart).In this paper, we use a joint (end-to-end) SGD training of the atom encoder and predictor, as the original CBM paper [13] suggests for generalization. The learning problem can be stated as follows.Definition 3.2 (Learning Problem). Given: a relational language (E, V, P) with all the atoms collected in HB; a set of entities represented by their corresponding feature vectors in X (i.e. the input); a dataset composed of a subset of supervised atoms D = {(A i , l i ) : A i ∈ HB, l i ∈ {0, 1}}, where l i is the corresponding ground-truth value for A i ; models g p , s, u l(r) , f l(r) with parameters π and a maximum number of iterations T ; an atom dependency graph determining the relational structure of all the atoms; a loss function L. Find: min π (Ai,li)∈D L(y T (A i ), l i ).To derive specific instantiations of R-CBMs, we introduce the notion of templetized hyperedge for an atom dependency graph. A templetized hyperedge ρ is defined as a standard hyperedge, but where the source and/or the destination contain one or more variables. For example, ρ = ([p 1 (v, u), p 2 (u, e)], p(v, e)) is a templetized hyperedge, meaning that we have an hyperedge istance for each possible grounding to the variables occurring in the atoms (i.e. v, u in the example). All the hyperedges generated by the same template are associated to the same edge label l(r), so that both model functions u r(l) , f r(l) , are shared across all instances of the same template. We omit the subscript in case there is a single templetized hyperedge in the hypergraph (e.g. one single edge type in the dependency graph).Case #1: Standard CBMs. A standard (non-relational) CBM can be easily seen as an R-CBM, by making few assumptions on the relational language and the atom dependency graph it is based on: (i) all predicates are unary, and can be partitioned into two disjoint sets, i.e. the concepts c 1 , . . . , c k and tasks t 1 , . . . , t m , respectively, (ii) in the atom dependency graph any concept atom has no parents, i.e. for every atom of the form c(e), we have R(c(e)) = ∅, and for every task t there is exactly one templetized hyperedge ρ whose source is composed by all the concept atoms, i.e. N ρ (t(v)) = [c 1 (v), . . . , c k (v)], (iii) T = 1. Hence, the final prediction on an atom A = p(e) is obtained as y(A) = s(g p (x e )) if p is a concept predicate, and y(A) = f ρ (y(c 1 (e)), . . . , y(c k (e))), if p is a task predicate. For instance, given the unary predicates P = {red, round, tomato}, a possible concept bottleneck is given by the templetized hyperedgeN (tomato(v)) = [red(v), round(v)].Case #2: Node classification via GNNs. R-CBMs allow the modelling of simple relational structures, such as relation-entity graphs, which are typically used by GNNs to generate and update node embeddings. For instance, let us consider a node classification task wrt a class p, and let q denotes the relation in the graph. This can be represented by the templetized hyperedge N (p(v)) = [q(v, u)]. For a heterogeneous graph with q 1 , . . . , q k relations, we can instead consider the templetized hyperedge N (p(v)) = [q 1 (v, u), . . . , q k (v, u)]. Message-passing and readout in GNNs are special cases of R-CBMs, as embeddings do not depend on atom predictions and readout occurs only at step T : Case study #3: Templetized relational concept bottlenecks. The templatization of standard CBMs can be further generalized, as relational bottlenecks can represent more complex interactions. Definition 3.3 (Templetized relational concept bottleneck). Given an n-ary predicate p, and an integer w ≥ 0, we define a templetized relational concept bottleneck of width w as the expression:h t (A) = u h t-1 (A), h t-1 (B) B∈N (A) y T (A) = f h T (A)EQUATIONwhere v = (v 1 , . . . , v n ), ū = (u 1 , . . . , u w ) are variables and b(v, ū) is a list of atoms with predicates in P and tuples of variables taken from {v 1 , . . . , v n , u 1 , . . . , u w }. (v 1 , v 2 , u) = [parent(v 1 , u), parent(u, v 2 )] we get the templetized relational concept bottleneck:N (grandparent(v 1 , v 2 )) = [parent(v 1 , u), parent(u, v 2 )] Figure 3 illustrates the instantiation of a relational concept bottleneck N (U S(v)) = [U S(u), parent(u, v)], for E = [Willie, Homer, Bart].We notice that by replacing the variables v with an entity tuple in Definition 3.3 does not correspond to an univocal instantiation of a relational concept bottleneck, as the same destination atom is predicted for every substitution θ of the variables ū. For instance if E = [Abe, Homer, Bart, Lisa] in Example 3.4, we have different hyperedges having grandparent(Abe, Bart) as tail:N (grandparent(Abe, Bart)) = [parent(Abe, Abe), parent(Abe, Bart)] (θu = {u/Abe}) N (grandparent(Abe, Bart)) = [parent(Abe, Homer), parent(Homer, Bart)] (θu = {u/Homer}) N (grandparent(Abe, Bart)) = [parent(Abe, Bart), parent(Bart, Bart)] (θu = {u/Bart}) N (grandparent(Abe, Bart)) = [parent(Abe, Lisa), parent(Lisa, Bart)] (θu = {u/Lisa})Each separate grounding of a relational concept bottleneck corresponds to a separate predicate prediction, as the same destination atom can be predicted by different bottlenecks. Taking again Example 3.4, we can assume to have also the bottleneck:N 2 (grandparent(v 1 , v 2 )) = [grandparent(v 1 , u), sister(u, v 2 )] which also adds the hyperedge:N 2 (grandparent(Abe, Bart)) = [grandparent(Abe, Lisa), sister(Lisa, Bart)] (θ u = {u/Lisa})The final prediction is obtained by aggregating all the single predictions with the operator.Aggregation semantics. In standard CBMs the interpretation of the prediction solely depends on the task predictor f and there is a single templetized hyperedge from concepts to any task node. However, the same atom can be predicted by different bottlenecks in R-CBMs. Indeed, different bottlenecks can represent separate dependency paths for the same atom like:N 1 (t(v)) = b 1 (v, ū1 ), N 2 (t(v)) = b 2 (v, ū2), . . . Even when considering a single bottleneck, the same grounding for v corresponds to multiple dependencies for the same atom if w > 0. Each ground bottleneck corresponds to a separate hyperedge in the dependency graph and it plays a fundamental role the choice of the aggregation function . In this paper, we select as = max, as it guarantees a sound interpretation to R-CBMs' predictions. Indeed, the max aggregation corresponds to the semantics of an existential quantification on the variables ū. As a result, the final task prediction is true if the task predictor f fires for at least one grounding of the extra variables.Example 3.5. Following Example 3.4, we consider a task predictor f as a logic conjunction (∧) between concept atoms. If we use = max, then the final task prediction is true if at least one substitution for u is true, i.e. if there exists an entity that is parent of Bart and such that Abe is her/his parent. Hence, the final task prediction can be interpreted as the logic formula∃u parent(Abe, u) ∧ parent(u, Bart) → grandparent(Abe, Bart)In summary, assuming = max and that each f is realized as a logic rule φ, a relational concept bottleneck with N (p(v)) = b(v, ū) can be associated with the explanation:∀v ∃ū φ(b(v, ū)) → p(v)where ∀v = ∀v 1 , . . . , ∀v n and ∃ū = ∃u 1 , . . . , ∃u w , like done in logic programs [14] .Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]Answer: [No] Justification: We will make the code public after publication.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experiments: 
 In this section we analyze the following research questions: Generalization-Can standard/relational CBMs generalize well in relational tasks? Can standard/relational CBMs generalize in out-ofdistribution settings? 3Interpretability-Can relational CBMs provide meaningful explanations for their predictions? Are concept/rule interventions effective in relational CBMs? Efficiency-Can relational CBMs generalize in low-data regimes? Can relational CBMs correctly predict concept/task labels with scarce concept train labels?Data & task setup. We investigate our research questions using 7 relational datasets on image classification, link prediction and node classification. We introduce two simple but not trivial relational benchmarks, namely the Tower of Hanoi and Rock-Paper-Scissors (RPS), to demonstrate that standard CBMs cannot even solve very simple relational problems. The Tower of Hanoi is composed of 1000 images of disks positioned at different heights of a tower. Concepts include whether disk i is larger than j (or vice versa) and whether disk i is directly on top of disk j (or vice versa). The task is to predict for each disk whether it is well-positioned or not. The RPS dataset is composed of 200 images showing the characteristic hand-signs. Concepts indicate the object played by each player and the task is to predict whether a player wins, loses, or draws. We also evaluate our methods on real-world benchmark datasets specifically designed for relational learning: Cora, Citeseer, [30] , PubMed [23] and Countries on two increasingly difficult splits [28] . Additional details can be found in App. A.1 and App. A.5.Models. We compare R-CBMs against state-of-the-art concept bottleneck architectures, including CBMs with linear and non-linear task predictors (CBM-Linear and CBM-Deep) [13] , a flat version (Flat-CBM) where each prediction is computed as a function of the full set of ground atoms, but also with Feedforward and Relational black-box architectures. We also compared against DeepStochLog [33] , a state-of-the-art NeSy system, and other KGE specific models for the studied KGE tasks. Our relational models include an R-CBM with DCR predictor (R-DCR) and its direct variant, using only 5 supervised examples per-predicate (R-DCR-Low). We also considered a noninterpretable R-CBM version where the predictions are based on an unrestricted predictor processing the atom representations (R-CBM-Emb). In the experiments, the loss function was selected to be the 5 Key Findings 
generalization: 
 Standard CBMs do not generalize in relational tasks (Table 1 ). Standard CBM best task performance ∼ 55% ROC-AUC is just above a random baseline. This result directly stems from the architecture of existing CBMs, which can process only one input entity at a time. The experiments validate that this design fails on relational tasks that inherently involve multiple entities. Naive attempts to address the relational setting, like Flat-CBMs, lead to a significant drop in task generalization performance (-17% in Hanoi), and become intractable when applied to larger datasets (e.g., Cora, Citeseer, PubMed, Countries). In RPS, instead, Flat-CBMs performance is close to random as the linear predictor for this model can not well approximate the required non-linear combination of concepts. These findings expose the limitations of existing CBMs when applied to relational tasks and justify the need for relational CBMs.R-CBMs generalize well in relational tasks (Tables 1 and 2 . Relational concept bottleneck models match the generalization performance of relational black-box models (GNNs and KGEs) in relational tasks. For example, R-CBMs exhibit gains of up to 7% MRR (Countries S1), and at most a 1% loss in accuracy (Citeseer) w.r.t. relational black-boxes. In larger KGEs like WN18RR , R-DCR beats standard KGEs and is competitive against state-of-the-art custom logic-based solutions, while being more general. Non-interpretable solutions R-CBM-Emb are admitted by our relational formulation when f c is selected to be a generic MLP blackbox. Since R-CBM-Emb and R-DCR only differ for the selection of the predictor (interpretable for R-DCR), a comparison of the results of these models on WN18RR provides a direct measurement of the performance decay due to the additional interpretability. Relational CBMs employing a simple linear layer as task predictor (R-CBM-Linear) underfit tasks demanding on non-linear combinations of concepts (e.g., RPS). In such scenarios, a deeper task predictor (e.g., R-CBMs Deep) trivially solves the issue, but it also hampers interpretability. R-DCRs address this limit providing accurate predictions while generating high-quality rule-based explanations (Table 4 ). It also matches generalization performance of neural symbolic system DeepStochLog [33] , which is provided with ground truth rules.R-CBMs generalize in out-of-distribution settings where the number of entities changes at test time (Figure 4 ). R-CBMs show robust generalization performances even in out-of-distribution conditions where the number of entities varies between training and testing. To assess generalization in these extreme conditions, we use the Tower of Hanoi dataset, where test sets of increasing complexity are generated by augmenting the number of disks in a tower. We observe that a naive approach, such as Flat-CBMs, immediately breaks as soon as we introduce a new disk in a tower, as its architecture is designed for a fixed number of input entities. In contrast, R-CBMs are more resilient, as we observe a smooth performance decline from ∼ 100% ROC-AUC (with 3 disks in both training and test sets) to around ∼ 85% in the most challenging conditions (with 3 disks in the training set and 7 in the test set). CBM architectures allow human interaction with the learnt concepts to intervene on mispredicted concepts during testing to improve the final predictions. In our experiments we assess CBMs' response to interventions on the RPS and Hanoi datasets. We set up the evaluation by generating a batch of adversarial test samples that prompt concept encoders to mispredict ∼ 50% of concept labels by introducing a strong random noise in the input features drawn from the uniform distribution U(0, 20). In our findings, we note that R-CBMs positively respond to test-time concept interventions by increasing their task performance. This contrasts with standard CBMs, where perfect concept predictions are not enough to solve the relational task. Notably, the RPS dataset poses a significant challenge for relational CBMs equipped with linear task predictors, as the task depends on a non-linear combination of concepts.Expanding our investigation to DCRs, we expose another dimension of human-model interaction: rule interventions. Applying both concept and rule interventions, we observe that R-DCRs perfectly predict all adversarial test samples.Relational Concept Reasoners discover semantically meaningful rules (Table 4 ). Among CBMs, a key advantage of DCRs lies in the dual role of generating rules which serve for both generating and explaining task predictions. Table 4 shows instances of R-DCR explanations, confirming that R-DCR discovers rules aligned with known ground truths across diverse datasets (e.g., wins(X) ← ¬rock(X)∧paper(X)∧¬scissors(X)∧rock(Y )∧¬paper(Y )∧¬scissors(Y ) in RPS). Notably, R-DCR discovers meaningful rules even in low data regimes (R-DCR-Low) and when the correct rules are unknown, such as in Cora, Citeseer and PubMed. R-DCR accurately makes interpretable predictions with very few atom supervisions (Table 1 ). R-DCR-Low is able to learn an interpretable relational predictor when reducing the training data to 5 labeled atoms for each predicate. Indeed, the supervisions are crucial to establish an alignment between human knowledge and the model on the semantics of logical explanations. The alignment can be perfectly achieved in RPS, where the predictions are mutually exclusive. On the Hanoi dataset, learning the relational binary concepts larger and top from 5 examples is challenging, leading to slightly decreased overall performance.. correct(v) ← top(u 1 , v) ∧ top(v, u 2 ) ∧ larger(v, u 1 ) ∧ larger(u 2 , v) ∧ larger(u 2 , u 1 ) ∀v, ∃u 1 , u 2 . correct(v) ← top(v, u 2 ) ∧ top(u 1 , u 2 ) ∧ top(u 2 , u 1 ) ∧ larger(v, u 2 ) ∧ larger(u 2 , v) Cora ∀v, ∃u. nn(v) ← nn(u) ∧ ¬rl(u) ∧ ¬rule(u) ∧ ¬probM odels(u) ∧ ¬theoru(u) ∧ ¬gene(u) ∧ cite(v, u) PubMed ∀v, ∃u. type1(v) ← type1(u) ∧ ¬type2(u) ∧ ¬experimental(u) ∧ cite(v, u) Countries ∀v 1 , v 2 , ∃u. locatedIn(v 1 , v 2 ) ← locatedIn(v 1 , u) ∧ locatedIn(u, v 2 ) 
discussion: 
 Related work on CBMs. Concept bottleneck models [13] inspired several works focusing on improved generalization [15, 5, 31] , explanations [3, 1] and robustness [17, 10, 36, 11] . Despite these efforts, the application of CBMs to relational domains remains unexplored. Filling this gap, our framework allows relational CBMs to (i) effectively solve relational tasks, and (ii) generalize the explanatory capabilities of these models from propositional to relational.Related work on GNNs. R-CBMs and relational black-boxes (such as GNNs) share similarities in considering the relationships between multiple entities when solving a given task. The prediction computation of R-CBMs is based on a message passing paradigm which is similar to message-passing in graph neural networks [7] , which is a special case of the proposed architecture. However, relational CBMs can also define aggregations based on a semantically meaningful concepts, allowing the extraction of explanations, which can not be done by GNNs.Related work on ILP. Among our considered R-CBMs, R-DCR is the only one learning a set of logic rules, hence we can draw some parallels with some algorithmic solutions defined in Inductive Logic Programming (ILP) [21] . R-DCR bottlenecks are connected to ILP mode declarations or metarules [22] , which define the search space. However, while ILP involves searching through a hypothesis space of possible logical rules, guided by principles like consistency, coverage, and simplicity, R-DCR is different as it searches this space via gradient descent over a continuous relaxation of the logic, and the logic formulas are learnt by exploiting neural architectures.Related work on Neuro-Symbolic AI. Neural Theorem Provers (NTP) [27] and their more scalable variations [20] combine formal proof proving with neural networks for efficiency via a trainable heuristic search. However, R-CBMs are more general than NTP -or other specific rule learnersindeed their interpretability is not restricted to rule learning, but rather rely on concept interventions. Moreover, unlike NTP, when dealing with logic rules the R-CBM framework is not limited to Horn clauses, and it can be applied in classic CBMs setups where inputs are not symbolic, but images. Moreover, R-CBMs' templates can represent all the rules using a (subset of a) specified list of atoms in the body at the same time, and the embeddings will be used to determine which actual rule to instantiate in each given context (like in R-DCR). On the other hand, NTP's approach to rule learning is to enumerate all possible rules and let the learning decide which rules are useful. Please note that this approach is not scalable to larger KGs because of the combinatorial explosion of the number of rules when there are many predicates in the dataset.Limitations. A limitation of relational CBMs consists in their limited scalability to very large domains. This limitation is shared with all existing relational systems, and most relational models have to rely on simplifying heuristics to scale to large relational structures like knowledge graphs [37, 26] . Another limitation of R-CBMs lays in the need for the definition of a relational concept bottlenecks, which acts as an architectural inductive bias, restricting the search space. Future extensions of relational CBMs can relax the need of an external template definition by including an automatic calibration of template widths, the construction of reduced set of variables' substitutions, or the automatic generation of the relational templates.Conclusions. This work presents R-CBMs, a family of concept bottleneck models designed for relational tasks. The results of our experiments show that R-CBMs: (i) match the generalization performance of existing relational black-boxes, (ii) support the generation of quantified conceptbased explanations, (iii) effectively respond to test-time interventions, and (iv) withstand demanding settings including out-of-distribution scenarios, and low data regimes. R-CBMs represent a significant extension of standard CBMs, and pave the way to further investigations using CBMs to improve interpretability in GNNs and to explain KGE predictions.The DistMult Knowledge Graph Embeddings (KGE) [34] was used as BlackBox relational baseline for the Countries S1 and S2 datasets. We varied the embedding sizes in the set {10, 20, 50, 100, 200, 300} and selected the best results on the validation set. The DistMult KGE was used as a basic concept encoder for CMBs. The R-CBM-Linear computes the concepts via linear layer followed by the KGE output layer. The R-CBM-Deep computes the concepts via an MLP with 2 hidden layers followed by a KGE output layer. Activation functions are ReLu.All experiments have been carried out on a machine with a Intel i7 CPU, 128GB RAM. Running times for all experiments are within 1 hour, with the exception of the link prediction experiment on WN18RR, which took 14h:20m.In all synthetic tasks, we generate datasets with 3,000 samples and use a traditional 70%-10%-20% random split for training, validation, and testing datasets, respectively. During training, we then set the weight of the concept loss to λ = 0.1 across all models. We then train all models for 3000 epochs using full batching and a default Adam [12] optimizer with learning rate 10 -4 . KGE experiments have used the Complex and Rotate KGE encoder and scorer function for g r , s in the Countries and WN18RR datasets, respectively.As explained in Section A.2.2, the relational CBMs exploits the features obtained by pretraining on a GNN on the same data split. Such pretraining is beneficial only in high-data settings (i.e. 100%, 75% and 50%). On low data regime (i.e. 25%), pretrained features are worse than original features. In these cases, we train the different baselines from scratch by using directly the low level features of the documents.For training R-CBM models on the WN18RR dataset, we used the heuristic, commonly employed in the NeSy community, of instantiating only hyperedges where all source atoms are observed in the training set [37] .The task consists of predicting the unknown locations of a country, given the evidence in form of country neighbourhoods and some known country/region locations. The entities are divided into the C, R, W domains referring to the countries, regions and continents, respectively. The predicate locIn(v 1 , v 2 ) determines the location of a country in a region or continent, with the variables(v 1 , v 2 ) ∈ C × R ∪ W or (v 1 , v 2 ) ∈ R × W . The country neighbourhoods are determined by the predicate neighOf (v 1 , v 3 ) with the variables v 1 , v 3 ∈ C.The entities in the dataset are a set of countries, regions and continents represented by their corresponding feature vectors as computed by a DistMult KGE (see baselines). The concept datasets are respectively the setD c locIn = (C × R) ∪ (R × W ) and D c neighOf = C × C.The task dataset D y locIn = C × W is formed by queries about the location of some countries within a continent. The templetized relational concept bottleneck is defined as:N (locIn(v 1 , v 2 )) = [locIn(v 1 , u 1 ), locIn(u 1 , v 2 ), neighOf (v 1 , u 2 ), locIn(u 2 , v 2 )] .Finally, the cross entropy loss was used both for functions for concepts.R-CBMs can also be used to find easy/hard samples, similar to what done for standard CBMs in [6] . In our framework, we consider as hard examples the ones whose prediction is highly uncertain when using a CBM with a propositional template (see CBM-Deep rows in Table 6 . When using a relational template, instead, we verified that the distribution of the prediction uncertainty significantly decreases. We show this in Figure 5 , where the prediction uncertainty decreases when transitioning Relational Concept Bottleneck Model Linear (R-CBM-Linear) The most basic task predictor employed in standard CBMs is represented by a single linear layer [13] . This choice guarantees a high-degree of interpretability, but may lack expressive power and may significantly underperform whenever the task depends on a non-linear combination of concepts. In the relational context, we define it as following:EQUATIONDeep Relational Concept Bottleneck Model (Deep R-CBM) To solve the linearity issue of R-CBM, one can increase the number of layers employed by the task predictor (as also proposed in [13] ). In the relational context we can define a Deep R-CBM as following: Deep R-CBM:EQUATIONwhere we indicate with M LP a multi-layer perceptron. However, the interpretability between concept and task predictions is lost, since MLPs are not transparent. Further, the ability of a Deep R-CBM to make accurate predictions is totally depending on the existence of concepts that univocally represent the tasks, hence being possibly very inefficacy.Relational Deep Concept Reasoning (R-DCR) [5] proposed to encode concepts by employing concept embeddings (instead of just concept scores), improving CBMs generalization capabilities, but affecting their interpretability. Then [1] proposed to use these concept embeddings to generate a symbolic rule which is then executed on the concept scores, providing a completely interpretable prediction. We adapt this model in the relational setting: R-DCR:EQUATIONwhere φ indicates the rule generated by a neural module working on the concept embeddings. For further details on how φ is learned, please refer to [1] . Since the logical operations in R-DCR are governed by a semantics specified by a t-norm fuzzy logic [8] , whenever we use this model we require the aggregation operation ⊕ used in Eq. 3.2 to correspond to a fuzzy OR. The max operator corresponds to the OR within the Gödel fuzzy logic.Relational Deep Embedding Reasoning (R-CBM-Emb) [18] proposes a latent relational process, which computes the atom representations using the presentations of other atoms that co-occur in the same ground formula. The final readout is based on an MLP processing the final atom representation. This model can exploit the rich reletional representations developed as atom embeddings, but it acts as a blackbox in terms of explanations of how the decision process takes form. This model can be implemented in our general model structure by restricting the f c function to only process the h t c embeddings as input, such that:h t c (A) = u r(l) h t-1 (A), h t-1 (B) B∈Nc(A) y t c (A) = M LP h t c (A) .R-DCR-Low R-DCR-Low is a version of R-DCR that is trained by providing the atom supervisions for only 5 incoming hyperedges. Its architecture and learning is entirely identical to DCR except for two variants:• Since DCR strongly depends on crisp concepts prediction for learning good and interpretable rule, in absence of sufficient supervision, we need a different way to obtain crisp predictions.To this end we substitute the standard sigmoid and softmax activation functions for concept predictors g i with discrete differentiable sample from a bernoulli or categorial distributions. The differentiability is obtained by using the Straight Through estimators provided by PyTorch. • Since the backward signal from DCR can be very noisy at the beginning of the learning, we add a parallel task predictor (and a corresponding loss term), completely identical to the one of a R-CBM-Deep model. Such predictor only guides the learning of the concepts during training by a cleaner backward signal but is discarded during test, leaving a standard DCR architecture.Flat Concept Bottleneck Model (Flat-CBM) assumes each prediction to be computed as a function of the full set of ground atoms. This model has limited scalability but it is introduced for comparison reasons in the experimental section.Libraries For our experiments, we implemented all baselines and methods in Python 3.7 and relied upon open-source libraries such as PyTorch 1.11 [24] (BSD license) and Scikit-learn [25] (BSD license). To produce the plots seen in this paper, we made use of Matplotlib 3.5 (BSD license). We will release all of the code required to recreate our experiments in an MIT-licensed public repository.Resources All of our experiments were run on a private machine with 8 Intel(R) Xeon(R) Gold 5218 CPUs (2.30GHz), 64GB of RAM, and 2 Quadro RTX 8000 Nvidia GPUs. We estimate that approximately 50-GPU hours were required to complete all of our experiments.There are no ethical issues.NeurIPS Paper ChecklistQuestion: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: These details are given in the Appendix.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We provide the standard-deviations over multiple seedsall for most experiments.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: Our research does not raise any ethical issue.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA]Justification: The paper does not involve an immediate societal impact.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We mentioned and respected all the licences of software and data in the Appendix.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [No] Justification: The code and the related licenses will be released upon publication of the paper.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.Assuming a 1-to-1 mapping between constants and entities allows us to use these words interchangeably.App.discusses how to select the atoms input to a relational concept bottleneck.The code to replicate the experiments presented in this paper is available at https://github.com/ diligmic/RCBM-Neurips2024.https://github.com/mledoze/countries 
: 
 Chemistry, politics, economics, traffic jams: we constantly rely on relations to describe, explain and reason on everyday life problems. For instance, we can easily deduce Bart's citizenship if we consider Homer's citizenship and his status as Bart's father (Figure 1 ). While relational Deep Learning (DL) models [29, 19, 32, 16] can effectively solve such problems, the design of interpretable neural models capable of relational reasoning is still an open challenge. Among DL methods, Concept Bottleneck Models (CBMs) [13] are interpretable methods explaining their predictions by first mapping input features to a set of human-understandable concepts and then using such concepts to solve the given tasks. However, current CBMs are not well-suited for addressing relational problems as they can process only one input entity at a time by construction. To solve relational problems, CBMs would need to handle concepts/tasks involving multiple entities (e.g., the concept "parent" which depends on both the entity "Homer" and "Bart"), thus forcing CBMs to process more entities at a time. Moreover, the definition of a suitable relational bottleneck layer is generally not straightforward, as a task prediction may require complex connections among multiple relational concepts. On the other side, while existing relational DL methods, such as Graph Neural Networks (GNN), may effectively solve such problems (e.g., correctly predicting Bart's citizenship), they are still unable to explain their predictions as CBMs would do (e.g., Bart is a US citizen since Homer is a US citizen and Homer is the father of Bart). Hence, a knowledge gap persists in the existing literature: defining a DL model capable of relational reasoning (akin to a GNN), while also being interpretable (akin to a CBM).To address this gap, we propose Relational Concept Bottleneck Models (R-CBMs, Section 3), a family of concept bottleneck models where both concepts and tasks may depend on multiple entities, and that have both CBMs and GNNs as special cases. The results of our experiments (Section 4 and 5) show that R-CBMs: (i) match the generalization performance of existing relational black-boxes, (ii) support the generation of first-order logic explanations, (iii) effectively respond to test-time concept and rule interventions improving their task performance, (iv) withstand demanding test scenarios including out-of-distribution settings, limited training data regimes, and scarce concept supervisions. 
 
 Appendix:  


Decision: The paper introduces Relational Concept Bottleneck Models (R-CBMs). Roughly speaking, R-CBMs merges CBMs and GNNs into an interpretable relational setting. The reviewers agree that this is novel and interesting work, while also pointing our some downsides, see e.g. the discussion about "concept incompleteness" and adding simple GNN-based models as additional baselines. In any case, the rebuttal of the authors helped to clarify some of the issues, and the reviews lean towards accept. I fully agree.  It would be nice if you could clarify the difference to "relational bottlenecks" (https://arxiv.org/pdf/2309.06629).

Reviews: review: paper_summary: paper_summary: This paper introduces Relational Concept Bottleneck Models (R-CBMs), a family of relational deep learning models that can provide some degree of interpretability and explainability; R-CBMs generalise both CBMs and GNNs.

According to the authors, R-CBMs 1) match the generalisation performance of existing black-box models, 2) support the generation of logic explanations, 3) respond to test-time concept and rule interventions, and 4) generalise to out-of-distribution samples and in limited training data regimes.

R-CBMs represent a relational graph as a set of atoms and dependencies among atoms and a directed hypergraph: each hyperedge defines a relational concept bottleneck from several ground atoms to a destination ground atom. Fig. 2 provides an example: the atom p4(b) can be predicted from the atoms [p3(b), p2(a, b), p1(b, a)], and these atoms all belong to the same hyperedge. What is the difference between the hyperedge notation and writing this as a Horn clause, e.g. p4(b) :- p3(b), p2(a, b), p1(b, a)?

R-CBMs are composed of a few components: 1) an atom encoder/predictor, 2) a message-passing component that updates the atom representations based on the dependency hypergraph, and 3) a relational task predictor. Having GNN-like message-passing components for updating the embeddings and the predictions of the atoms, in my opinion, may invalidate the interpretability claims of R-CBMs since GNNs are intrinsically black-box neural models.

One of my concerns in this work is that there are already models that can learn and leverage Horn rules-like structures for interpretable relational predictions, such as Neural Theorem Provers (e.g., https://arxiv.org/abs/1705.11040, https://arxiv.org/abs/2007.06477) -- what's the delta between R-CBMs and NTPs?

NTPs are really similar to the proposed approach -- for example, R-CBMs use "max" as the aggregation function to decide which hyper-edge to use when making a prediction, which is exactly the same strategy used by NTPs for deciding which "proof path" to use to prove whether a given atom is true or not.

Experiments -- probably the most used dataset for relational link prediction in Knowledge Graphs is FB15k-237; why did you use WN18RR instead? Are there scalability issues due to the higher number of atoms or relational predicates in FB15k-237? Also, the paper introducing WN18RR was not cited (as well as several papers introducing the baselines).

For WN18RR, the paper proposes an apparently wide set of baselines. However, extremely simple but effective baselines like ComplEx-N3 (https://github.com/facebookresearch/kbc/, https://arxiv.org/abs/1806.07297, ICML 2018) produce more accurate results than the proposed method (0.58 Hits@10 vs 0.56).

I really loved Tab. 4 with the examples of learn (symbolic) rules; however, learning symbolic rules is something that NTPs can also do -- can you please expand on the delta between R-CBMs and NTPs? Also, it's not completely clear to me how rules are learned -- is it because the encoder/predictor can be used to learn a dependency hypergraph that can be used to learn hyper-edges (the Horn clause-like structures)?

Update after reading the rebuttal: the authors addressed most of my concern, I'm increasing my score!
strength_weakness: strength_weakness: 1) It is not clear if it's reinventing Neural Theorem Provers (e.g., https://arxiv.org/abs/1705.11040, https://arxiv.org/abs/2007.06477), which can also be used to learn FOL rules via back-prop
2) Why WN18RR instead of, e.g., FB15k-237? Missing very simple but effective link prediction baselines, e.g. https://arxiv.org/abs/1806.07297 (ICML'18)
3) Writing is a bit opaque -- why use "hyper-edges" to introduce Horn clauses?
4) Given all the GNN-like components inside R-CBMs, are they really interpretable?
questions: questions: 1) Can you clarify how the learning of the symbolic rules happens? Is it by learning a link predictor in the atom encoder module that can be used to extract hyper-edges/Horn clauses?
2) Any answer addressing the "Weaknesses" would be really helpful
limitations: limitations: 1) Are there scalability limitations of the proposed model? E.g., would you be able to evaluate on FB15k-237?
2) What if you need more than one application of the rules? NTPs can handle that by applying them recursively
paper_summary: paper_summary: The paper introduces Relational Concept Bottleneck Models (R-CBMs), which address the challenge of designing interpretable deep learning models that operate in relational domains. Existing Concept Bottleneck Models (CBMs) are interpretable but lack the capability to manage relational data, while Graph Neural Networks (GNNs) can handle relational data but are not as interpretable. R-CBMs integrate the strengths of both by allowing interpretability in a relational context. They achieve this by mapping input features to a set of human-understandable concepts, then using these concepts to make predictions. The paper evaluates R-CBMs across various experimental settings, demonstrating that they match or exceed the generalization performance of existing relational models, support quantified concept-based explanations, respond effectively to test-time interventions, and perform robustly in challenging scenarios like out-of-distribution testing and limited data availability
strength_weakness: strength_weakness: 1. Related work is insufficient but should have included the interpretable methods to fix this issue. Here are the two papers, the authors should include:
* [Posthoc based]

[1] Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat. Ghosh et al. ICML 2023

[2] POST-HOC CONCEPT BOTTLENECK MODELS. Yuksekgonul et al. ICLR 2023

* [CLIP based]

[1] Label-Free Concept Bottleneck Models. Oikarinen et al.

[2] Visual Classification via Description from Large Language Models. Menon et al. ICLR 2023

* [Relational CBM]

[1] Relational Concept Based Models. Barbiero et al. arXiv 2023.

[2] Interpretable Neural-Symbolic Concept Reasoning. Barbiero et al. ICML 2023.

2. Insufficient baselines. The authors should comapre with atleast one of the posthoc based baselines. Also original CBM is not SOTA anymore as there are multiple variants of CBM.

3. Do the authors assume to have concept annotation? this is expensive. Can it be aliviated with the LLMs to constuct the concepts. There are existing works which include the LLM to deduce the concepts. 

4. Currently the community is concerned about the incompleteness of the the concept annotations. So with the assumption of having concept annotation the authors fail to answer this question? How to address this incompleteness?

5. CBMs can be used to find easy/hard samples. With relational model this can be done. The authors can perform an experiment for finding out the easy/hard samples. They can see Route, Interpret, Repeat paper.

6. How to quantitatively evaluate the concepts extracted? The authors should have computed the concept completness scores (Yeh at al.) to do so.

7. How to extend this work for imaging datasets like scene graph understanding?
questions: questions: See Weaknesses
limitations: limitations: See Weaknesses
paper_summary: paper_summary: This paper proposed a Relational Concept Bottleneck Models(R-CBM), which merge CBMs and GNNs together. To be more specially, it encode atom into concept like CBM and did message passing afterwards like GNN.
strength_weakness: strength_weakness: 1. It seems like the CBM provide a initialization of GNN model, therefore, it is not enough to compare with only CBM model as baseline. Because essentially, this is a GNN model. I think we should provide baselines for different GNN + different initialization as comparison.

2. Concepts in CBM are clearly defined. Can the author clearly defined what are the concepts in each of your dataset? When the dataset become larger, how to define your concepts?

3. I think some good property that R-CBM learned like example 3.4 and 3.5 could be formalized better.

4. Table 4 list some easy relations that R-CBM learned. While I think the power of R-CBM should be learning relations that people could not identify. Could you show some other complex relations?
questions: questions: See the weakness above
limitations: limitations: NA
paper_summary: paper_summary: The authors propose Relational Concept Bottleneck Models, a family of relational deep learning methods that utilize concept bottleneck models to provide interpretable task predictions. R-CBMs are shown to predict well in various settings, matching the performanace of black-box models.
strength_weakness: strength_weakness: It might be nice to include a comparison to a simpler graph-based method that is not black box, e.g. [C&S](https://arxiv.org/abs/2010.13993) and a discussion of how the choice of relational task predictor and aggregation affect interpretability

Would be interesting to see how R-CBM compares with CBM on datasets that aren't specifically designed for relational prediction.
questions: questions: What is the explanation for why R-CBM-Linear does not effectively respond to interventions for RPS?
limitations: limitations: Adequately addressed

Comments: title: Further comments [Reviewer]
Post rebuttal comment
Thanks!
Thanks for your comments
comment: Dear Reviewer CYvd,

We are sorry for misinterpreting your comment regarding concept incompleteness.

**According to Definition 3.1 in Yeh, et al., the completeness score can be higher than 1 (100% in our table), whenever "the best accuracy by predicting the label just given the concept scores" (the CBM's accuracy) is higher than the accuracy of the black box prediction model.** Both the numerator and the denominator are normalized wrt the "accuracy of random prediction to equate the lower bound of completeness score to 0". However, this metric is not normalized to give an upper bound in 1 in its original formulation. 

Regarding concept incompleteness in the relational domain, consider that in the relational setting the issue of concept incompleteness is less demanding than in the non-relational domain. Indeed, the set of concepts is often complete by construction as both concepts and tasks can relate to the same ground predicates (this is a property of relational datasets, as we described in the example in the common answer "Concepts’ definition/annotation for each dataset"). **One of the main results of our experiments shows that in the nine relational datasets we considered, this is actually the case: relational concept bottlenecks are complete, while non-relational bottlenecks are not.**

**Regarding LLMs, we only wanted to point out that this paper aims to fill a different research gap with respect to the problem of incomplete concept annotation and the integration of LLMs with CBMs: to the best of our knowledge, this is the first paper that shows how to construct CBMs in the relational domain**. Our contribution fills this gap notwithstanding whether the set of concepts and their annotations are given from supervisions, extracted from an unsupervised method [2], or provided by an LLM [1,3]. However, we acknowledge that studying concept incompleteness and integrating LLMs with CBMs in relational domains are interesting directions for future research.

Thanks again for your feedback and your comments, they definitely helped us improving our work.
By concept incompleteness, i did not indicate concept completeness score. Concept completeness score indicates how good your concept can be a good predictor of the downstream labels. Concept incompleteness means what if your concept set is incomplete on the first hand. Look at the papers i refered.

Regarding concept completeness score, you provided in the pdf - is those numbers in percentages? usually concept completeness scores are b/w 0-1. I see some numbers are greater than 100 (e.g., 102.52). How is that possible? So, this part i consider is not rightly estimated

For the LLM point, LLM can be used to solve the question of incompleteness. and i believe at this point relational concepts can be obtained by using LLM as well, so these are not orthogonal. How to perform intervention with LLM concepts - there are papers like Language in a bottle (LaBo) (CVPR 2023). However i agree this can be explored in future. But that makes the contribution borderline.

Thanks again.
Please let us know if you have any further questions or things we could clarify further. If not, we would appreciate if you could consider updating your review based on our replies.
**We provided the completeness scores of our method and CBM baselines in Table D** (see pdf attached to the global answer to all reviewers) and **explained the reason why concept annotations are often complete by construction in relational domains** (see common answer "Concepts’ definition/annotation for each dataset"). Please let us know if you have further questions regarding incompleteness that we have not addressed.

**Yes, LLMs might be used to construct concepts, but our paper focuses on an orthogonal research question which is: "how can relational concepts be used in a CBM setting?"**. Moreover, it is still unclear how to perform effective concept-interventions when LLMs are used to generate concepts. Indeed, even in recent papers (including the mentioned "Concept Bottleneck Models Without Predefined Concepts" [published on arxiv the last month]), interventions are limited to intervening on the task predictor's weights and on a cherry-picked selection of samples. Further research might be required to understand whether concept bottlenecks and annotations constructed with LLMs provide the same guarantees in terms of interpretability and intervention effectiveness wrt other CBMs. We consider the integration of LLMs and relational CBMs a wide topic of interest for future works.
I would like to thank the reviewer for the rebuttal. My questions are mostly answered except the LLM and incompleteness.
Recent publications aim solve this:
[1] A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis. Yang et al.
[2] CONCEPT BOTTLENECK MODELS WITHOUT PREDEFINED CONCEPTS. Schrodi et al.
[3] LLM-based Hierarchical Concept Decomposition for Interpretable Fine-Grained Image Classification. Qu et al.

I would like to remain with my score.
Thanks, I will increase my score!
I thank the authors for their response, but maintain my rating of 7