Neural Concept Binder
Authors: Wolfgang Stammer
Antonia Wüst
David Steinmann
Kristian Kersting
Venue: NeurIPS 2024 poster (2024)
Keywords: Concept Discovery
Interpretable Artificial Intelligence
Interactive Machine Learning
Disentanglement
Primary area: interpretability_and_explainability

Abstract: The challenge in object-based visual reasoning lies in generating concept representations that are both descriptive and distinct. Achieving this in an unsupervised manner requires human users to understand the model's learned concepts and, if necessary, revise incorrect ones. To address this challenge, we introduce the Neural Concept Binder (NCB), a novel framework for deriving both discrete and continuous concept representations, which we refer to as "concept-slot encodings". NCB employs two types of binding: "soft binding", which leverages the recent SysBinder mechanism to obtain object-factor encodings, and subsequent "hard binding", achieved through hierarchical clustering and retrieval-based inference. This enables obtaining expressive, discrete representations from unlabeled images. Moreover, the structured nature of NCB's concept representations allows for intuitive inspection and the straightforward integration of external knowledge, such as human input or insights from other AI models like GPT-4. Additionally, we demonstrate that incorporating the hard binding mechanism preserves model performance while enabling seamless integration into both neural and symbolic modules for complex reasoning tasks. We validate the effectiveness of NCB through evaluations on our newly introduced CLEVR-Sudoku dataset.

Neural Concept Binder
Wolfgang Stammer 1,2
Antonia Wü
David Steinmann 1,2,
Kristian Kersting1.240 1.23311
CComputer Science Department, TU Darmstadt; "Hessian Center for Al (hessian.Al), German Research Center for Al (DFKI); *Centre for Cognitive Science, TU Darmstadt
Abstract
The challenge in object-based visual reasoning lies in generating concept repre- sentations that are both descriptive and distinct. Achieving this in unsupervised manner requires human users to understand the model's learned concepts and, if necessary, revise incorrect ones. To address this challenge, we introduce the Neural Concept Binder (NCB), a novel framework for deriving both discrete and continuous concept representations, which refer "concept-slot encodings". NCB employs two types of binding: "soft binding", which leverages the recent SysBinder mechanism to obtain object-factor encodings, and subsequent "hard binding". achieved through hierarchical clustering and retrieval based inference. This enables obtaining expressive, discrete representations from unlabeled images. Moreover, the structured nature of NCB's s concept representations allows for intu- itive inspection and the straightforward integration of external knowledge, such as human input or insights from other AI models like GPT-4. Additionally, we demon- strate that incorporating the hard binding mechanism preserves model performance while enabling seamless integration into both neural and symbolic modules for complex reasoning tasks. We validate the effectiveness of NCB through evaluations our newly introduced CLEVR-SSudoe dataset. Code and data at: project page
1 Introduction
An essential aspect of visual reasoning is obtaining a proper conceptual understanding of the world by learning visual concepts and processing these into suitable representation (ef. Fig. 1]. The majority of current machine learning (ML) approaches that focus on visual concept- soncept-based processing utilize
forms of supervised [34
68
32
[4]
weakly-supervised [4
69
49
63
g
82 or text-guided (29)
learning of concepts. These approaches all require some form of additional (prior) knowledge about the relevant domain. An attractive alternative, though much more challenging. is learn concepts in an unsupervised fashion. This comes with several challenges: (i) learning an expressive concept representation without concept supervision intrinsically difficult 40 and (ii) there is no guarantee that learned concepts align with general domain knowledge [36] 85 9 and (iii) can therefore be
utilized for complex downstream tasks. Moreover, (iv) trust that the learned concept representations are reliable for high stakes scenarios [5] it is necessary to make the model's concept representations
suman--nspectable and -revisable 68 69
31
(cf. Fig. (left)).
These challenges raise questions about the nature the unsupervised learned concept representations. easier to learn and more expressive. However, they are difficult to interpret and suffer from problems related to poor generalization [81] and information
Continuous encodings (60 59
79
leakage 47 50]. On the other hand, discrete encodings 69 79 26 4 hard to learn 44 72
but are easier to understand and thus align, e.g., to task at hand.
"Thee authors share equal contribution. Correpondence to: Wolfgang Stammer
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Learned concepts
Reason: A3 and B8
Unlabeled multi- object
CI:
Inspectable concepts
Utilizing concept symbols
Revisable concepts
[A3, BS, CZ [A1, B2, C2]
Interpretable neural
computations
NCB
Retrieve concepts
Symbalic computations
X A3 - Class (X)
Concept learner (NCB)
Figure 1: Unsupervised learning of concepts for visual reasoning. (left) Models that learn concepts from unlabeled data require inspectable and revisable concept representations. (right) Concepts obtained from the Neural Concept Binder (NCB) can be utilized both in (interpretable) neural and symbolic computations.
This work proposes the Neural Concept Binder (NCB) framework to learn expressive, yet inspectable and revisable, concepts from unlabeled data. NCB combines continuous encodings, obtained via bloc-sslt-bbsed soft-binding, with discrete concept representations, derived through retrieval-base hard-binding. NCB's s soft binding leverages the object-factor disentanglement capabilities of the recent SysBinder mechanism [65]. Subsequently, NCB's s hard binding mechanism utilizes HDB- SCAN 13 to cluster the continuous block-slot encodings, distilling structured corpus of discrete concepts from these clusters. This corpus enables the retrieval of discrete concept representations during inference by matching the continuous encoding with the closest entries in the corpus. Thus, to address the challenges of unsupervised concept learning, NCB integrates the strengths of both continuous and discrete concept representations. Moreover, NCB enables straightforward concept inspection and facilitates easy revision procedures, allowing alignment of the learned concepts with prior knowledge. In our evaluations, we demonstrate that NCB's discrete concept-slot encodings re- tain the expressiveness of their continuous counterparts. Moreover, they can be seamlessly integrated into downstream applications via symbolic and interpretable neural computations (cf. Fig. (right)). In this context, we introduce our novel CLEVR-Sudon dataset, which presents a challenging visual puzzle that requires both perception and reasoning capabilities (cf. Fig.
In summary, our contributions are the following: (i) we introduce the Neural Concept Binder framework (NCB) for unsupervised concept learning, (ii) we show the possibilities to integrate NCB with symbolic and subsymbolic modules in challenging downstream tasks, achieving performance on par with supervised trained models, (iii) highlight the possibilities of easy concept inspection and revision via NCB, and (iv) we introduce the novel CLEVR-Sudoku dataset, which combines challenging visual perception and symbolic reasoning.
Related Work
Unsupervised visual concept learning focuses on obtaining concept-level representations from unlabeled images [25] Some works have tackled this only for specific domains, such as extract-
ing "teachable" concepts for chess 62 or learning manipulation concepts from videos of task demonstrations [39 Others rely on object-level concept guidance through initial image segmenta-
(49 In contrast, Vedantam et al. (77) and Wüst al. [8]] focus on learning higher-level relational concepts, i.e., assuming that basic-level concepts have already been provided. Several approaches Iearn concepts from the training signal of an image classification task 778 14 38), often focusing on image-region-bee based concepts 222 More recently, several works have explored leveraging the knowledge stored in large pretrained models, such as combining large language models with CLIP embeddings [2 52) using weakly-supervise
tions (27 or "natural supervision"
queries to languae model []. These approaches still rely on some form of supervision, whether through text, class labels, or prompts. In contrast, this work focuses learning unsupervised concepts at both the object and factor levels, ensuring that these concepts remain inherently inspectable and revisable.
The motivation for inherently inspectable and revisable concept representations is to allow human stakeholders to investigate and potentially revise a model's 's internal concepts. Most research this area focuses on post-hoc approaches that distill concept knowledge from pretrained models [83 51

---

Neural Concept Binder (NCB)
Soft Binder ()
Hard Binder (v)
Downstream Task
Input
Slotes
0.01: Al
5x (=i)
C1
AB
0.58:
R
0.92: A1
D4
Block slot encodings
Black
(s(=)
Concept encodings
Recon.
A1)
distill:
A1)
A2)
D2>
Concopt exomplars
(x)
A8)
D4)
Concept
Inference Inspect
Extractor (III)
Retrieval Corpus (iv)
Train
Revise
Figure 2: The Neural Concept Binder (NCB) combines continuous, block-slot encodings via slot- attention based image processing with discrete, concept-slot encodings via retrieval-basee inference. The structured retrieval corpus (distilled from the block-slot encodings) allows for easy concept inspection and revision by human stakeholders. Moreover, the resulting concept-slot encodings can be easily integrated into complex downstream tasks.
18 23 In contrast, Lage and Doshi-Velez 35 explore learning inspectable concept representations
through human feedback, focusing tabular data and higher-level concepts. Similarly, Stammer et al. (69 develop inherently inspectable visual concepts using weak supervision and a prototype-based binding mechanism. However, no existing work addresses the development of inherently inspectable and revisable concept representations in the context of unsupervised visual learning.
The properties of discrete continuous encodings relevant to learning suitable concept representations. Continuous encodings allow for easier and
vibrant research topic that is highly
more flexible optimization and information binding [43
64 65 77. However, discrete representations
are considered essential for understanding AI models 311, mitigating shortcut learning [68] (3],
and solving complex visual reasoning tasks [26] 66) Despite their advantages, learning discrete
representations through neural modules remains challenging problem [4
24
20
74
While
some works have focused on inregoriallddistrbuutoo---bbb
discretization
28
46
others have
explored retrieval-baee based discretization of continuous encodings using various forms of inherent
"codebooks" " [73 (1]. Only a few studies have explicitly addressed how to bind semantic visual information to specific discrete representations [69]. Whereas previous works typically emphasize one of the two representation types, we see great potential in the recent trend of explicitly integrating both discrete and continuous representations [1 51.
3 Neural Concept Binder (NCB): Extracting Hard from Soft Concepts
In this work, we refer to concept "the label of set of things that have something in common" 2 This definition can be applied on different scales of a visual scene: on an image level (e.g., an image of park), object level (e.g., free vs. bird) or object-factor level (e.g., the color of a bird). Our proposed Neural Concept Binder (NCB) framework tackles the challenge learning inspectable and revisable object-factor level concepts from unlabeled images by combining two key elements: (i) continuous representations via SysBinder's block-sloo-attention 665 43) with (ii)
discrete representations via retrieval-based inference. Fig. 2 provides an overview of NCB's 's inference, training, concept inspection, and revision processes. Let us formally introduce these processes.
Overall, we consider a set of unlabeled images X == (x1,... .NN) E Nx with E RR,, N N and E N (for simplicity, we drop the image index notation the following). Briefly, given an image, x, NCB infers latent block-slot encodings, and performs retrieval- based discretization step on Z to infer concept-slot encodings, c. These express the concepts the objects the image, i.e., object-factor level concepts. We begin by introducing the inference procedure of NCB. We hereby assume that NCB's components have already been trained and will introduce details of the training procedure subsequently.
3

---

3.1 Inferring Concept-Slot Representations
Obtaining Continuous Block-Slot-Fncodingg Consider an image E X. The first component of NCB, the soft binder, is based on the systematic binding mechanism [65] and represented by block-slot encoder (ef. Fig. 2 (i)), 9e : - 2 E RRN where g is parameterized by 0 (for simplicity, this notation is omitted in the following). The soft binder transforms an input image into a latent, continuous block-slot representation, where Ns represents the number of slots, NB the number of blocks per slot, and DB the dimension of each block. The soft binder employs two key types of binding mechanisms: spatial and factor binding. Spatial binding ensures spatial modularity across the entire scene and is achieved through slot attention [43], allowing each object in the image to be represented in specific slot, 2/- Factor binding, introduced by Singh et al. [65], ensures that different object factors (e.g., attributes like color) are encoded in separate blocks of slot, i.e., z/- These two binding mechanisms work together to perform object- and factor-based image processing. We refer to Suppl. A.] for additional details on both systematic (factor) binding and slot attention. Overall, the resulting block-slot encodings represent continuous, object-centric representations of the input image, with objects encoded in slots and object factors encoded within the blocks of those slots.
Obtaining Discrete Concept-Slot- Encodiings. The role of NCB's s second processing component, the hard binder, is to transform the continuous block-slot encodings into expressive, yet discrete concept-slot encodings. Specifically, the hard binder is represented by a retrieval encoder, f (cf. Fig. 2 (v)), which processes the block-slot encodings, into set of discrete concept-slot encodings, c. In detail, f defines a function fR= 2 - C E N xNa, parameterized by a retrieval corpus R (cf. Fig. 2 (iv)). This retrieval corpus consists of a tuple of sets R == iR_... ..... R RNo,, where each set R² := {(enc/. vt) : I E {L.... 1}} contains tuples of block encodings. enc; R Ds and corresponding discrete values, i/ E {1, f1..... (Nc) Importantly, enc/ is a representative block encoding of a specific concept cluster, determined during NCB's training phase (cf. Fig. 2 (iii), detailed below). by serves as the symbol identifier for the concept cluster associated with enc' Each block can contain up to Nc E N different concepts. To infer the concept symbol for a sample's blot encoding, NCB compares z with the encodings in the corresponding block's 's retrieval corpus, R2, and selects the most fitting concept. Specifically, given a distance metric d(,;) -) and the block-slot encoding =/- the selection function $æ = - E Fig. 2 (v)) finds the index of the closest encoding in the retrieval corpus: &R (=() = argmin, d(enc) = ) such that (enc/, ur) E R This results in the concept representation for slot and block j. denoted as di == = (=)) For slot i,
and the final concept-slot encoding as c == fx(22 = [1,, .... cNsl We refer to Supppl. A.2 for details on alternative top-k selection function. We further note that NCB's 's flexibility, in principle, allows also to utilize the continuous
the full concept representation is denoted as c = let,...,c..
, ca\"]
encodings of its soft binder (Fig. 2 dashed arrow) in case downstream task requires it. Let us now move on to NCB's training procedure.
3.2 Unsupervised Concept Learning via NCB
The training procedure of the Neural Concept Binder is separated into two subsequent steps where provide an overview here and details in Suppl. A.3 We formally describe these steps using the pseudo- code in Alg. The first step consists of optimizing the encoder, 9, to provide factorrface block- slot encodings. It is optimized for unsupervised image reconstruction based on the decoder model, d6 z +
D (Fig. (ii) and utilizing a mean squared error loss: L = L (z, g (g(x))). The goal of NCB's second training step is to obtain the retrieval corpus, R. This procedure is based on obtaining an optimal clustering of block encodings via an unsupervised clustering model, h. and distilling the resulting information from h into explicit representations in the retrieval corpus. For each block j a clustering model has (Fig. 2 (iii) fit to identify a potentially overparameterised set of clusters within a set of block encodings (based on an unsupervised criterion, e.g., a density-based score (53). resulting in No E N clusters. Next, for each cluster, E {1. .Nc.. representative block encodings, enc', are extracted from h. Such an encoding represents either an averaged protorype or instance-base based exemplar encoding. The corresponding tuples (enc', v) are explicitly stored in the retrieval corpus R2 (Fig. 2 (iv)) where we use the index I to identify specific encodings in Rí, leading to Rd= = [(enc/, vz) E (1,..., IR']}). Thus, enc/ represents one block encoding of R' that has been assigned to cluster un. Finally, R = [R] ......... RR represents the final retrieval
4

---

Original Image
What i8
Why .
What ...?
concepts?
D5
B1
-B1
B3: "Cyllnder
Block
NS: "Brown"
Comcugta Block
FF5
bock u.
!12
B5-122
Inspection Type:
Implicit
(iI) Comparative
Interventional
(iv) Similarity
Figure 3: NCB's concept space is inherently inspectable. A human stakeholder can easily inspect the concept space by asking a diverse questions. For example, NCB answers interventional questions (iii) via generating images with selectively modified concepts.
corpus, i.e., the set corpora for each block. Through this training procedure, NCB learns to unsupervisedly categorize the object-factor information from the latent encoding space of the soft binder and stores this information in a structured, symbolic, and accessible way in the hard binder's retrieval corpus. We refer the resulting clusters of each block as NCB's concepts and denote concepts with a capital letter for the block and a natural number for the category id, e.g.. A3. We note that in practice, it is further possible to finetune the block-slot encoder, g. through supervision from the hard binder (cf. gray arrow in Fig. 2). e.g., once initial categories have been identified, and can be achieved via a standard supervised approach. Ultimately, this allows for dynamically finetuning NCB's concept representations. Let us now introduce how human stakeholders can inspect and revise NCB's learned concepts.
3.3 Inspecting and Revising NCB's Concepts
Inspection. NCB inherently enables: (i) implicit, (ii) comparative, (iii) interventional and (iv) similarity-based inspection (cf. Fig. 3). Where the first three aim at investigating NCB's explicit, symbolic concept space (stored in R), the last one aims investigating its latent, continuous concept space (stored in 0). (i) Implicit inspection queries the model to provide a set of examples for specific concept. Essentially, this answers the question "What are examples of this concept?". NCB answers this question in two ways: by providing samples from the retrieval corpus corresponding to exemplars of the concept or by identifying additional data samples belonging to the concept at hand. (ii) Comparative inspection, on the other hand, allows comparing two specifically different concepts, e.g. "Why does this object depict concept H5 and not concept HI?". NCB hereby provides examples for both concepts for the user to compare and potentially identify dissimilar properties. Ultimately, this form of inspection allows to answer questions of the form "Why not ...?" and represents valuable tool for in-depth and targeted concept inspection. (iii) Interventional inspection allows to answer questions such as "What if this object would have concept HI ?" To answer this question, NCB utilizes its decoder g'. Specifically, by swapping the block 2 data sample's ' block-slot encoding with that of representative sample, (enc/, v1) E R², NCB can provide an interventional image reconstruction, from which the effect of the swapped concept can be observed. Ultimately, this form of inspection allows to answer important questions of the form "What if ...?". . Finally, (iv) Similarity inspection allows inspecting NCB's continuous encoding space on a more global level (in comparison to more symbolic, sample-based inspection above), e.g., "What are similar concepts to this concept?". Specifically, NCB's 's distance metric directly provides information about the similarity between concepts in the continuous representation. Inspecting the block-slot encoding space thus allows to identify a suboptimal soft binding, e.g., when block encodings are similar according to g but not according to the human stakeholder. Overall, these inspection mechanisms allow human stakeholder to ask a diverse set of questions concerning a model's 's learned concepts (cf. Fig. 15 Fig. 16 and
Fig. 17 for additional examples of the inspection types).
Revise. Let us now describe how a human stakeholder can revise NCB's concept space. Below, we provide details the three main actions for symbolic revision (i.e., revision on the representations in R): (i) merging, (ii) deleting, or (iii) adding information. These actions can performed on single encoding concept level and essentially represent form of "rcorganization" of information
5

---

Table Comparison of different approaches for concept learning. Hereby, we differentiate based on the following categories: whether method (1) is learned in an unsupervised fashion, (2) provides object-level concepts (i.e. can explicitly process multiple objects), (3) provides factor-level concepts (e.g., the color green), (4) provides continuous concept encodings, (5) provides discrete concept encodings, (6) provides inherently inspectable and (7) revisable concept representations.
Method
Unsupervised Obj. level
Factor level
Cont. enes
Disc. enes
Inspectable
Revisable
CBM 34]
X
x
X
NeSyCI. 68
x
x
/
GlanceNets 50
x
/
/
VAE 33
x
8
VQ-VAE [75
/
x
/
/
x
SA [43
/
/
x
x
(5)
x
SysBinder 66
7
/
x
(5)
Neural Concept Binder
stored in R. Furthermore, we provide details on how to (iv) revise the continuous latent space, which essentially requires finetuning of g's parameters. (i) Merge Concepts: In the case that R contains multiple concepts that, according to additional knowledge (e.g. from a human or other model), represent joint underlying concept (e.g., two concepts for purple in Fig. 3 (right)) it is easy to update the model's internal representations by replacing the concept symbols of one concept with those of the second concept. Specifically, for block j if concept m should be merged with concept b where m, E {1. Nc then for all corpus tuples, (enci, 11) E R2, we replace et with b if vt = m. (ii) Delete Encodings or Concepts: If R contains an encoding, enc/. for a specific concept, m, that does not match the other encodings of that concept (e.g., misplaced exemplar) this encoding can simply be deleted from the corpus. Accordingly, if an entire concept, m, is identified as suboptimal, one can simply delete all corresponding encodings of that concept. I.e., for all corpus tuples. (enc), 01) E R2, we remove the tuple if eg = m. (iii) Add Encodings or Concepts: If a specific concept not sufficiently well captured via the existing encodings in R2, one can simply add a new encoding, eñc'+1- for the concept, m, the corpus. This leads to an additional entry in the corpus, (eñc? m.. Accordingly, it is also possible to add encodings for an entire concept. Hereby, one gathers block encodings of objects that represent that novel concept and adds these to the corpus as (eñc) b) with b N: + 1. (iv) Revise the (Continuous) Latent Space: Lastly, if the soft binder provides suboptimal object- and factor-level block-slot encodings, it is further possible to integrate revisory feedback on the soft binder's continuous latent space. This can be achieved via additional finetuning of the soft binder's parameters, 0, e.g. via standard forms of weak supervision
42 69
or interactive learning [68 61.
In summary, our novel Neural Concept Binder framework fulfills several important desiderata for concept learning (cf. [Tb.. Tp. Specifically, NCB learns concepts in an unsupervised fashion that are structured on both an object and factor-level. Furthermore, next to standard continuous encodings, NCB also provides discrete concept representations, which are crucial for interpretability and integration into symbolic computations. Lastly, NCB's 's concept space is inspectable and revisable, essential for unsupervised learned concept representations.
Experimental Evaluations
In our evaluations, we investigate the potential of NCB's soft and hard binding mechanisms in unsupervised concept learning and its integration into downstream tasks. Notably, NCB encompasses concept processing between both of its components (soft binder and hard binder) whereby the direction "soft binder <- hard binder" (cf. |Fg.. 2) represents a standard approach (i.e., supervised learning of the soft binder's 's encoding space via symbolic concept labels, e.g., [34 68)). Therefore, we focus our evaluations on NCB's 's more novel processing direction, "soft binder + hard binder". We aim to answer the following research questions: (QI) Does NCB provide expressive and distinct encodings? (Q2) Can NCB be combined with symbolic methods to solve complex downstream tasks? (Q3) Can NCB's s learned concepts be revised to improve suboptimal behaviour? (Q4) Can NCB be combined with subsymbolic methods to transparently solve complex downstream tasks?

---

Data. We focus our evaluations on different variations of the popular CLEVR dataset. Specifically, we investigate (Q1 & Q3) in the context of the CLEVR [30 and CLEvR-Easy 665 datasets. For investigating the integration of NCB into symbolic modules (Q2), we utilize our novel CLEVR- Sudoku puzzles introduced in the following. Finally, to evaluate the integration of NCB into subsymbolic modules (Q4), we evaluate on confounded and non-confounded variants of the CLEVR- Hans3 dataset [68]. We provide further details on these datasets in the supplements (cf. Suppl. C).
To investigate the potential of integrating NCB's s discrete concept represen- tations into symbolic downstream tasks, we introduce the novel CLEVR- dataset presents challenging visual puzzle that requires both visual object perception and rea- soning capabilities. Each sample in the dataset (ef. Fig. for an example puzzle) consists of Sudoku puzzle (partially filled) with CLEVR-based images [30] and additional example images depicting the mapping of relevant object properties to digits. Specifically, each digit in the Sudoku is replaced by an image of an object. All objects representing the same digit share a set of common properties, e.g., in Fig. 4 all objects replacing "1"s are yellow spheres. We introduce two variants of CLEVR-Sudoku: Sudoku CLEVR- -Esy and Sudoku CLEVR. In the first variant, shape and color are distinguishing properties for the digits. In Sudoku CLEVR, additional object attributes size and material tion. Moreover, up to 10 example images are provided per digit mapping: the fewer examples provided, the more difficult it becomes to learn the mapping. The ini- tial state and digit-attribute mappings vary across ples. One specific intricacy of TLLEVR-Sudoke is that the puzzle can only be solved if all subcell images are correctly mapped to their corresponding digits. Even single mistake can render the Sudoku unsolvable. Thus, compared to standard Sudoku puzzles, which primarily require deductive reasoning, solving CLEVR-Sudoku also demands complex object recognition and the abil- ity to map visual concept perceptions to the task con- cepts (i.e., the 9 digits of Sudoku). For further details, we refer to (Suppl. B
CLEVR-Sudoku,
dataset. This
relevant for the digit identifica-
Figure 4: Example from CLEVR-Sudoku. Each digit is represented by CLEVR objects with the same attribute combination. The objective is to solve the Sudoku only based on the initial grid of CLEVR images and the digit mapping of candidate examples.
Models. For our evaluations, we instantiate Neural Concept Binder based on the SysBinder model (65] for the soft binder encoder, g. and HDBSCAN [12 [3] for the clustering model, h. Further details about the instantiation can be found in Suppl. A.4 In the context of Q1, we compare NCB's results to four variations of the SysBinder model 665, as well as the recent Neural Language of Thought Model (NLOTM) [80]. We refer to the original SysBinder configuration as SysBinder (cont.), which provides continuous block-slot encodings. In SysBinder, SysBinder's continuous encodings are discretized at inference time via an argmin operation over its internal codebooks. SysBinder (hard) is trained from the beginning to produce discrete encodings using low codebook softmax temperature. SysBinder (step) is trained with step-wise decrease in temperature (cf. [Suppl. D) for details). For evaluations on BLEVR-Sudoke (Q2 and Q3), we first infer NCB's s discrete concept-slot encodings from the puzzle's ' candidate examples. These encodings, along with their corresponding digit labels, are then passed to a symbolic classifier, which is trained to predict digits from the encodings. The classifier subsequently infers the digits for each subcell in the puzzle's initial state. These predictions are used by a constraint propagation and search-bart based algorithm [55[[0 to solve the puzzle (cf. [Suppl. E.2 for details). We refer to the combination of the symbolic classifier and constraint solver as the solver. We compare the solver's 's performance when provided with ground-trum (GT) object- proeecttpppertty labels (GT concepts), encodings from supervised slot attention encoder 43 (SA (supervised)), and the discrete encodings from SysBinder (denoted as SysBinder (unsupervised)). For classification evaluations (Q4), evaluate configuration in which set transformer classifier (37] is provided with NCB's 's concept encodings (NCB+ + NN) to make final class predictions (ef. [Suppl. E.4). We compare this to SA + NN, where a supervised slot attention encoder (43 provides object-property predictions.
Metrics. We evaluate all models based on their accuracies on held-out test splits, each with seeded runs. We provide average accuracies and standard deviations over these. When assessing the expressiveness NCB's 's concept-slot encodings (QI), we evaluate the accuracy for object-property
7

---

Table 2: NCB's concept encodings are expressive despite information bottleneck. Classifying object properties from different continuous and discrete encodings. The classifier is provided with different amounts of training sample encodings. The best ("*") and runner-up (")")) results are bold.
Dataset
Train
SysBinder (cont.)
SysBinder
SysBinder (hard)
SysBinder (step)
NLOTM
Neural Concept Binder
N-2000
99. 2.22
62.49:16.45
22.92::0.00
95.761:..2
84.36:8.54
1:00
CLEVR-
N=200
99. 20:441
87 8.000
22 20.92
92, 7.2982
72. 22.99
98.50+11.50
Easy
N-50
9..133:4.21
78.41
22.92::0.00
70.64::1.80
49.941:1.97
95.87
N-20
664.88110.89
62.61 18
22.922.0.00
54.61 29.57
37. 07.05:24.111
944 224.222:1111
N=2000
98. 1.15
86, 266.22=10.66
36. 46:00.0
88, 18.90011..8
54. 18.77
97. 26:22.67
CLEVR
N=50
N-200
993.25:-402
12.6612258
81.13=12..9 12.59
61.67#8.61
38.46:00.00
36..66:0000
81. 17.17
68.81
50.17::1..26
13.600
94.67
996 80:.00
N-20
79.71
49.791..75
36.46..0.00
18.58116.09
41.12:12.90
88.57.11166
prediction. When evaluating the performance of the downstream tasks, we provide the percentage of solved CLEVR-Sudokus (Q2) and the classification accuracy on the test set of CLEVR-Hans? (Q4).
4.1 Evaluations
Discrete, yet expressive representations (Q1), First, we investigate how much valuable information NCB's discrete concept-slot encodings contain, despite NCB's ' inherent information bottleneck. To assess this, we train a classifier on NCB's s encodings to predict corresponding object-propers labels, e.g., the color green (cf. [Suppl. E. for details). In Tab. 2] we present the results for the CLEVR-Easy and CLEVR datasets, using classification training sets with 2000, 200, 50, or 20 encodings. Focusing first on the results for N 2000, we observe that, as expected, the continuous representation of the original SysBinder model contains more information compared to all discrete encodings. Remarkably, however, NCB's discrete concept representations are nearly on par with the continuous encodings. This is particularly notable given NCB's ' immense information bottleneck]] Additionally, we observe that NCB's is encodings significantly outperform all other forms of discrete representations. Shifting focus to the results when the classifier is trained on data subsets, we observe a substantial degradation in performance when using encodings from any of the discrete baselines or the continuous encodings. In stark contrast, when classifying based on NCB's 's encodings, the accuracy remains nearly constant, even with just 1% of the initial training samples. We provide additional ablations on the effect of concept encoding types and NCB's s selection function in (Suppl. F.T, as well as an ablation analysis on the effect of suboptimal behavior from NCB's s individual components in inppu F.2] Further analysis of NCB's ' concept space can be found in Suppl. F33 along with qualitative examples of learned concepts in Suppl. G Overall, our results demonstrate the expressiveness of NCB's s concept encodings despite their significant information bottleneck. Furthermore, our results suggest that NCB's 's encodings are casier to generalize compared to the baselines. Thus, we answer Q1 affirmatively.
Utilizing unsupervised concepts for solving visual Sudoku (Q2). In our following evaluations, we investigate the potential of NCB's representations for solving complex reasoning tasks through their integration into symbolic computations. These evaluations are based on our novel CLEVR- Sudoku dataset. The percentage of solved puzzles for CLEVR-Sudoke is reported in Fig. 5] It is important to note that the solver can only solve a puzzle if each image in the initial state has been classified correctly, meaning the results in Fig. Sjrepresent "all-or-nothing" outcomes. Focusing on the results to the left of the dashed lines, we observe that the symbolic module solves every puzzle with ground-truth (GT) concepts, even when only one example image is provided. Interestingly, performance drops significantly when using encodings from SA (supervised). This highlights the difficulty of the CLEVR-Sudol puzzles: minor errors in digit prediction can lead to major failures in solving the puzzle. When comparing the performance of encodings from the two unsupervised models, we observe that NCB's s concept encodings perform quite well. E.g., they enable solving approximately 50% of the puzzles for the 10-example configurations, compared approximately 61% for SA (supervised). In contrast, when using SysBinder's 's encodings, the solver fails across all Sudoku variations. This demonstrates the effectiveness of NCB's binding mechanisms over those of the SysBinder approach alone. We refer to Suppl. F44 for further discussions and quantitative digit classification results (Fig. 10). Overall, our evaluations highlight the potential of NCB's unsupervised concept encodings for solving complex symbolic downstream tasks. We therefore answer Q2 affirmatively.
SyaBinder (cont.) provides 2048-sized continuous encodings, whereas NCB provides discrete eucodings of size 16.
8

---

Sudoku CLEvR-Easy
Example
Examples
Examples
10 Examples
Solved Puzzles (1)
100
Sudoku CLEVR
Example
Examples
Examples
10 Examples
100
Solved Puzzls
76
50
Concepts
SA (supervieed)
SyeBinder (unsupervieed)
(unsupervised)
NCB revised (GPT-4)
NCB revised (human)
NCB revised (human)"
Figure 5: NCB's unsupervised concepts allow solving symbolic puzzles. Accuracy of solved Sudokus via different discrete concept encodings on Sudoku CLEVR-Fasy and Sudoku CLEVR (left sides). Additional revision on NCB's ' concepts leads to improved performances (right sides).
Table 3: NCB's unsupervised concept representations facilitate interpretable neural compu- tations. Explanations of a NN classifier trained on the unsupervised concepts of NCB. Via NCB's inherent inspection procedures human stakeholder can identify which concepts the classifier focuses on to make its predictions and thus interpret the NN's underlying decision rule.
GT Class Rule
NN Expl.
Human Inspection
Human Interpretation
Large, gray cube
C4 H5 A K5
(Grayl) A (Red V Gray2) A (Large) A
"A large gray object"
O13 A P6
(Gray3) A (Gray4)
Small, metal cube
B4 A D4 AHI
(Cube) A (Smalll) A (Small2) A (Small3) A (Small4)
"A small cube"
All K1
Large, blue sphere
BI A C7 H4
(Sphere) A (Bluel) A (Blue2) A (Small V Blue3) (Blue4 V Green Purple)
"A blue sphere"
A1 P2
Easily revising NCB's concepts (Q3). In our next evaluations, we illustrate the potential of NCB's revision procedures. Since revising the continuous latent space of NCB's soft binder is analogous to existing approaches (e.g., [61 58, 69),, we focus on the novel, NCB-specific forms of symbolic revision, i.e., revisions within the hard binder's concept space. We demonstrate two forms of symbolic revision (removing and merging concept information) using feedback from two sources: a pretrained vision-language model (here via GPT-4 (56)) and simulated human feedback. both cases, ask the revisory agent to identify which concepts in each block should be removed or merged based on exemplar images of each concept, i.e., implicit concept inspection (cf. [Suppl. E.3 for details). In Fig. 5) we show -LEVR--uuooku performance when NCB's 's retrieval corpus is updated by different revisory agents (i.e., NCB revised (GPT-4) and NCB revised (human)). Interestingly, while GPT-4's revisions improve performance in settings with few examples, they have negative impact when more digit examples are present. This is due to GPT-4's suboptimal consistency in object descriptions, leading to the removal or merging of too much concept information. This highlights the potential issue of "ill-informed" feedback (cf. (Suppl. F.5). contrast, human revisions provide a substantial boost in Sudoku performance, particularly in puzzle configurations with fewer candidate examples. Moreover, using NCB's similarity inspection mechanism (cf.. [Sec. 3.3), a human stakeholder can easily identify models that suffer from suboptimal soft binding processing. In such cases, these models can be excluded from further downstream evaluations (cf. NCB revised (human) and refined by finetuning parameters (e.g., via approaches from [61] 58 69].. In SuSppl.. F.6) we further explore concept revision by adding new information. Overall, our results demonstrate the potential and ease of revising NCB's concept space, allowing to answer Q3 positively.
Utilizing unsupervised concepts for understanding neural computations (Q4). our final evaluations, we investigate whether NCB's 's discrete concept encodings can make subsymbolic compu-

---

tations more transparent. We focus on the task of image classification using concept-bottleneck like approaches [68] 34 on variations of the benchmark CLEVR 3 dataset [6]. While the con-
cept encodings in NCB + NN are trained unsupervised, they perform on par with the supervised approach of [68] (ef. SSupll. F.7). More importantly, integrating NCB's inherently inspectable concept representations into neural computations leads to more transparent decision processes. We illustrate this in Tab. 3| where we provide class-
Confounded Classification
level explanations of the classifier in NCB + NN (ef Suppl. E.4 for details). Using NCB's inspection mechanisms, human stakeholders can easily iden- tify the classifier's 's internal decision rules for a class (e.g.. "a large gray object"). This is critical feature for deploying trustworthy Al models in real-world scenarios. The key result is that this transparency is achieved even with unsupervised concept encod- ings. In[Fig. 6] further investigate whether NCB- based neural classifier can be revised to mitigate con- founders in the CLEVR-Hans3 dataset (cf. SSuppl. E.4 SSppl F:8 for details). The confounding factor in the training set is the color gray, and we present the non-confounded test set accuracy in Fig. We observe that standard loss- based feedback via explanatory interactive learning (XIL) [68 on the NN classifier's 's explanations (+ XIL on NN) significantly reduces the effect of the confounder. A Alternatively, by simply zeroing the activations of the undesired concept gray (+ XII. on concepts), we achieve even better confounding mitigation results without the typical issues of joint optimization. Our results highlight the potential of integrating NCB's s unsupervised concept representations for eliciting transparent and trustworthy subsymbolic computations. We thus answer Q4 affirmatively.
100
NCB + NN
(%)
8
XIL on NN
NCB concepts
Figure 6: NCB's unsupervised concept rep- resentations facilitate shortcut mitigation. Test accuracy for classification via NN predic- tor when trained on confounded images.
Limitations. NCB largely benefits from high-quality initial block-slot encodings. If these encodings are suboptimal, the resulting concept-slot encodings also degrade in quality. An important next step to handle more complex visual inputs, such as video data, is the integration of recent approaches Additionally, due NCB's unsupervised training nature, further alignment of NCB's s concepts is inevitable for effective deployment in downstream tasks 19]. Further, to build trust in NCB's concept knowledge, human inspection is essential. Lastly, revisions are a critical aspect of NCB. However, they rely on humans to provide accurate feedback; a malicious user could manipulate NCB's concepts. Fortunately, by inspecting the concept space, it possible track and mitigate such manipulation effectively.
(e.g.. 16 19]
5 Conclusions
In this work, we introduce the Neural Concept Binder framework for learning visual object-factor concepts in unsupervised manner. Our evaluations suggest that NCB's specific binding mech- anisms facilitate the learning of expressive yet discrete concept representations. Furthermore, our results highlight the potential of integrating NCB's 's inherently inspectable and revisable concept-slot encodings into both symbolic and neural modules. Promising directions for future research include exploring the benefits of NCB's representations in continual learning settings (1I, high-level concept learning [81), and probabilistic logic programming approaches [66 67]. as well as investigating connections to object-centric causal representation learning (48). Lastly, incorporating downstream learning signals may be valuable (if present) for improving the quality of NCB's initial concept
encodings, e.g., through classification [5] 6 or differentiable clustering (76
Acknowledgments
The authors thank Gautam Singh for help with SysBinder and Cyprien Dzialo for preliminary results and insights. This work was supported by the Priority Program (SPP) 2422 in the subproject "Optimization of active surface design of high-speed progressive tools using machine and deep learning algorithms" funded by the German Research Foundation (DFG), the "ML2MT" project from the Volkswagen Stiftung and the "The Adaptive Mind" project from the Hessian Ministry of Science and Arts (HMWK). has further benefited from the HMWK projects "The Third Wave of Artificial Intelligence- 3AI", and Hessian.AI, as well as the Hessian research priority program LOEWE within the project Wox,eBox, and the EU-funded "TANGO" project (EU Horizon 2023, GA No 57100431).
10

---

References
[1] David Alvare7-Melis and Tommi S. Jaakkola. Towards robust interpretability with self- explaining neural networks. In Advances in Neural Information Processing Systems (NeurIPS), pages 7786-7795, 2018.
[2] E. James Archer. The psychological nature of concepts. In Analyses of concept learning, pages 37-49. Elsevier, 1966.
[3] Md Rifat Arefin, Yan Zhang, Aristide Baratin, Francesco Locatello, Irina Rish, Dianbo Liu, and Kenji Kawaguchi. Unsupervised concept discovery mitigates spurious correlations. In International Conference on Machine Learning (ICML). OpenReview.nee 2024.
[4] Masataro Asai and Alex Fukunaga. Classical planning in deep latent space: Bridging the Sabsymbolic-symbbo boundary. In Conference on Artificial Intelligence (AAAI), pages 6094- 6101. AAAI Press, 2018.
[5] Yaniv Aspis, Krysia Broda, Jorge Lobo, and Alessandra Russo. Embed2sym scalable neuro- symbolic reasoning via clustered embeddings. In International Conference on Principles of Knowledge Representation and Reasoning (KR), 2022.
[6] Yaniv Aspis, Mohammad Albinhassan, Jorge Lobo, and Alessandra Russo. Embed2rule scalable neuro-symbolic learning via latent space weak-labelling, In Neural-Symbolic Learning and Reasoning (NeSy), volume 14979 of Lecture Notes in Computer Science, pages 195-218. 18. Springer, 2024.
[7] Pietro Barbiero, Gabriele Ciravegna, Francesco Giannini, Mateo Espinosa Zarlenga, Lucie Char- lotte Magister, Alberto Tonda, Pietro Lio, Frédéric Precioso, Mateja Jamnik, and Giuseppe Marra. Interpretable neural-symbolis concept reasoning. In International Conference Machine Learning (ICML), 2023.
[8] Adrita Barua, Cara Widmer, and Pascal Hitzler. Concept induction using Ilms: a user experiment for assessment. CoRR, abs/2404. 1875, 2024.
[9] Aaron Bembenek and Toby Murray. Symbol correctness in deep neural networks containing symbolic layers. CoRR, abs/2402.03663, 2024.
[10] Christian Bessiere. Constraint propagation. In Foundations of Artificial Intelligence, volume 2, pages 29-83. Elsevier, 2006.
[11] Florian Peter Busch, Roshni Kamath, Rupert Mitchell, Wolfgang Stammer, Kristian Kersting, and Martin Mundt. Where is the truth? the risk of getting confounded in a continual world. CoRR, Abs/2400.006434 2024.
[12] Ricardo J. G. B. Campello, Davoud Moulavi, and Jörg Sander. Density-based clustering based on hierarchical density estimates. In Advances in Knowledge Discovery and Data Mining (PAKDD), 2013.
[13] Ricardo J. G. B. Campello, Davoud Moulavi, Arthur Zimek, and Jörg Sander. Hierarchical density estimates for data clustering, visualization, and outlier detection. ACM Transactions on Knowledge Discovery from Data, 10(1):511-::51, 2015.
[14] Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan Su. This looks like that: Deep learning for interpretable image recognition. In Advances in Neural Information Processing Systems (NeurIPS), pages 8928-8933 2019.
[15] Alex DeGrave, Joseph D. Janizek, and Su-In Lee. AI for radiographic COVID-19 detection selects shortcuts over signal. Nature Machine Inielligence, 3(7):610-619) 2021.
[16] Quentin Delfosse, Wolfgang Stammer, Thomas Rothenbacher, Dwarak Vittal, and Kristian Kersting. Boosting object representation learning via motion and object continuity. In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (PKDD/ / ECML), 2023.
[17]
Dinu, Claudiu Leoveanu-Condrei, Markus Holzleitner, Werner Zellinger, and Sepp Hochreiter. Symbolicai: A framework for logic-based approaches combining generative models and solvers. CoRR, abs/2402.00854, 2024.
[18] Maximilian Dreyer, Reduan Achtibat, Wojciech Samek, and Sebastian Lapuschkin. Under- standing the (extra-)ordinary: Validating deep model decisions with prototypical concept-based explanations. CoRR, abs/2311. 2023.

---

[19]
Gamaleldin F. Elsayed, Aravindh Mahendran, Sjoerd van Steenkiste. Klaus Greff, Michael C. Mozer, and Thomas Kipf. Savi++: Towards end-to-end object-centric learning from real-world videos. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[20] Jerome Feldman. The neural binding problem (s). Cognitive neurodynamics, 7:1-11, 2013.
[21]
Asma Ghandeharioun, Been Kim, Chun-Liang Li, Brendan Jou, Brian Boff, and Rosalind W. Pi- card. DISSECT: disentangled simultaneous explanations via concept traversals. In International Conference on Learning Representations (ICLR). 2022.
[22] Amirata Ghorbani, James Wexler, James Y. Zou, and Been Kim. Towards automatic concept- based explanations. In Advances in Neural Information Prucessing Systems (NeurIPS), pages
9273-9282, 2019.
[23] Shantanu Ghosh, Ke Yu, Forough Arabshahi, and Kayhan Batmanghelich. Dividing and conquering a blackbox
mixture of interpretable models: Route, interpret, repeat. In International Conference on Machine Learning (ICML), 2023.
[24] Klaus Greff, Sjoerd van Steenkiste, and Jürgen Schmidhuber. On the binding problem in artificial neural networks. CoRR, 2tss2201.052000 2020.
[25] Avani Gupta and P. Narayanan. A survey on concept-based approaches for model improvement. CoRR, abs/2403. 14566, 2024.
[26] Michael Hersche, Mustafa Zeqiri, Luca Benini, Abu Sebastian, and Abbas Rahimi. A neuro- vector-symbolic architecture for solving raven's progressive matrices. Nature Machine Intelli- gence, 5(4):363-3) 2023.
[27] Haiyang Huang. Zhi Chen, and Cynthia Rudin. Segdiscover: Visual concept discovery via unsupervised semantic segmentation. CoRR, abs/2204. 10926, 2022.
[28] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations (ICLR), 2017.
[29] Chen Jin, Ryutaro Tanno, Amrutha Saseendran, Tom Diethe, and Philip Teare. An image is worth multiple words: Learning object level concepts using multi-concept prompt learning. CoRR, abs/2310. 12274, 2023.
[30] Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross B. Girshick. CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning. In Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[31] Subbarao Kambhampati, Sarath Sreedharan, Mudit Verma, Yantian Zha, and Lin Guan. Symbols as lingua franca for bridging human-ai chasm for explainable and advisable AI systems. In Conference on Artificial Intelligence (AAAI), 2022.
[32] Eunji Kim, Dahuin Jung, Sangha Park. Siwon Kim, and Sungroh Yoon. Probabilistic concept bottleneck models. In International Conference on Machine Learning (ICML), 2023.
[33] Diederik P. Kingma and Max Welling. An introduction variational autoencoders. Foundations and Trends in Machine Learning, 1244)3307-392, 2019.
[34] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference Machine Learning (ICML). 2020.
[35] Isaac Lage and Finale Dossi-Veler. Learning interpretable concept-based models with human feedback. CoRR, abs/2012.02898, 2020.
[36] Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin De- tyniecki. The dangers of post-hoc interpretability: Unjustified counterfactual explanations. In International Joint Conference on Artificial Intelligence (IJCAI), 2019.
[37] Juho Lee, Yoonho Lee, Jungtack Kim, Adam R. Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for iatention-based International Conference Machine Learning (ICML), 2019.
ienpuutttionninvaiiatt neural networks. In
[38] Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin. Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions. In Conference on Artificial Intelligence (AAAI), 2018
[39] Ruizhe Liu, Qian Luo, and Yanchao Yang. Infocon: Concept discovery with generative and discriminative informativeness. 2024.
12

---

[40] Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Rätsch. Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. In International Conference on Machine Learning (ICML),
2019.
[41] Francesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem, and Michael Tschannen. Weakly-supervised disentanglement without compromises. In International conference on machine learning (ICML). 2020.
[42] Francesco Locatello, Michael Tschannen, Stefan Bauer, Gunnar Rätsch, Bernhard Schölkopf. and Olivier Bachem. Disentangling factors of variations using few labels. In International Conference on Learning Representations (ICLR), 2020.
[43] Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[44] Luca Salvatore Lorello and Marco Lippi. The challenge of learning symbolic representations. In International Workshop on Neural- Neural-Symbolic Learning and Reasoning, 2023,
[45] James MacQueen et al. Some methods for classification and analysis of multivariate observations. In Berkeley Symposium on Mathematical Statistics and Probability, 1967.
[46] Chris Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous re- laxation of discrete random variables. In International Conference on Learning Representations (ICLR), 2017.
[47] Anita Mahinpei, Justin Clark, Isaac Lage, Finale Velez, and Weiwei Pan. Promises and pitfalls black-box concept learning models. CoRR, abs/2106.13314 2021.
[48] Amin Mansouri, Jason S. Hartford, Yan Zhang, and Yoshua Bengio. Object centric architectures enable efficient causal representation learning. In International Conference Learning Representations (ICLR). OpenReview.net, 2024.
[49] Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B Tenenbaum, and Jiajun Wu. The neuro- symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In International Conference on Learning Representations (ICLR), 2019.
[50] Emanuele Marconato, Andrea Passerini, and Stefano Teso. Glancenets: Interpretable, leak-proot concept-based models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[51] Eleonora Misino, Giuseppe Marra, and Emanuele Sansone. VAEL: bridging variational autoen- coders and probabilistic logie programming. Advances Neural Information Processing Systems (NeurIPS), 2022.
[52] Mazda Moayeri, Keivan Rezaei, Maziar Sanjabi, and Soheil Feizi. Text-to-concept (and back) via cross-model alignment. In International Conference on Machine Learning (ICML), 2023.
[53] Davoud Moulavi, Pablo A. Jaskowiak, Ricardo J. G. B. Campello, Arthur Zimek, and Jörg Sander. Density-based clustering validation. In International Conference on Data Mining, 2014.
[54] Frank Nielsen. Hierarchical clustering. Introduction to IPCC with MPI Data Science, pages 195-211, 2016.
[55] Peter Norvig. Solving every sudoku puzzle. URL htpp:/norvvg.. com/sudoku. html, 2006.
[56] OpenAL. GPT-4 technical report. CoRR, abs/2303.08774, 2023.
[57] Bo Pan, Zhenke Liu, Yifei Zhang, and Liang Zhao. Surrocbm: Concept bottleneck surrogate models for generative post-hoc explanation. CoRR, 100.0/39,, 2023.
[58] Andrew Slavin Ross, Michael C. Hughes, and Finale Velez. Right for the right rea- sons: Training differentiable models by constraining their explanations. In International Joint Conference Artificial Intelligence (IJCAI), 2017.
[59] Adam Santoro, Felix Hill, David G. T. Barrett, Ari S. Morcos, and Timothy P. Lillicrap. Measuring abstract reasoning neural networks. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmüsssun, Stockholm, Sweden, July 10-15, 2018 volume 80 of Proceedings of Machine Learning Research, pages 4477-4486 PMLR, 2018
13

---

[60]
Yoshihide Sawada and Keigo Nakamura. Concept bottleneck model with additional unsuper- vised concepts. IEEE Access, 10:41758-41765 2022.
[61] Patrick Schramowski. Wolfgang Stammer, Stefano Teso, Anna Brugger, Franziska Herbert, Xiaoting Shao, Hans-Georg Luigs, Anne- Katrin Mahlein, and Kristian Kersting. Making deep neural networks right for the right scientific reasons by interacting with their explanations. Nature Machine Intelligence, 2(8):476 486, 2020.
[62] Lisa Schut, Nenad Tomasev, Tom McGrath, Demis Hassabis, Ulrich Paquet, and Been Kim. Bridging the human-ai knowledge gap: Concept discovery and transfer in alphazero. CoRR, abs/2310. 16.413, 2023.
[63] Rui Shu, Yining Chen, Abhishek Kumar, Stefano Ermon, and Ben Poole. Weakly supervised disentanglement with guarantees. In International Conference on Learning Representations (ICLR), 2020.
[64] Gautam Singh, Fei Deng, and Sungjin Ahn. Illiterate DALL-E learns to compose. In Interna- tional Conference on Learning Representations (ICLR), 2022.
[65] Gautam Singh, Yeongbin Kim, and Sungjin Ahn. Neural systematic binder. In International Conference on Learning Representations (ICLR), 2023.
[66] Arseny Skryagin, Wolfgang Stammer, Daniel Ochs, Devendra Singh Dhami, and Kristian Kersting. proball-roaaailistii answer set programming. In International Conference on Principles of Knowledge Representation and Reasoning (KR), 2022.
[67] Arseny Skryagin, Daniel Ochs, Devendra Singh Dhami, and Kristian Kersting, Scalable neural- probabilistic answer set programming. Journal of Artificial Intelligence Research, 78:579-611 617,, 2023.
[68] Wolfgang Stammer, Patrick Schramowski, and Kristian Kersting. Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations. In Conference Computer Vision and Pattern Recognition (CVPR), 2021.
[69] Wolfgang Stammer, Marius Memmel, Patrick Schramowski, and Kristian Kersting. Interactive disentanglement: Learning concepts by interacting with their prototype representations. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[70] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning (ICML), 2017.
[71] Alex Tamkin, Mohammad Taufeeque, and Noah D. Goodman. Codebook features: Sparse and discrete interpretability for neural networks. CORR, abs/2310. 173300,, 2023.
[72] Sever Topan, David Rolnick, and Xujic Si. Techniques for symbol grounding with satnet. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[73] Frederik Träuble, Anirudh Goyal, Nasim Rahaman, Michael Curtis Mozer, Kenji Kawaguchi, Yoshua Bengio, and Bernhard Schölkopf. Discrete key-value bottleneck. In International Conference on Machine Learning (ICML), 2023.
[74] Anne Treisman. Solutions to the binding problem: progress through controversy and conver- gence. Neuron, 24(1):105-123 1999.
[75] Aäron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
[76] Georgios Vardakas and Aristidis Likas. Neural clustering based on implicit maximum likelihood.
Neural Computing and Applications, 35(29): 2023.
[77] Ramakrishna Vedantam, Arthur Szlam, Maximilian Nickel, Ari Morcos, and Brenden M. Lake. CURI: A benchmark for productive concept learning under uncertainty. International Conference Machine Learning (ICML), 2021.
[78] Bowen Wang, Liangzhi Li, Yuta Nakashima, and Hajime Nagahara. Learning bottleneck concepts in image classification. In Conference on Computer Vision and Puttern Recognition (CVPR), 2023.
[79] Taylor Whittington Webb, Ishan Sinha, and Jonathan D. Cohen. Emergent symbols through binding in external memory. In International Conference on Learning Representations (ICLR), 2021.
14

---

[80] Yi-Fu Wu, Minseung Lee, and Sungjin Ahn. Neural language of thought models. In Interna- tional Conference on Learning Representations (ICLR). OpenReview.net, 2024.
[81] Antonia Wüst, Wolfgang Stammer, Quentin Delfosse, Devendra Singh Dhami, and Kristian Kersting, Pix2code: Learning to compose neural visual concepts as programs. Uncertainty in Artificial Intelligence (UAT), 2024.
[82] Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Conference Computer Vision and Pattern Recognition (CVPR), 2023,
[83] Chih-Kuan Yeh, Been Kim, Sercan Ömer Arik, Chun-L Li, Tomas Pfister, and Pradeep Ravikumar, On completeness-aware concept-base based explanations in deep neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[84] Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Zohreh Shams, Frédéric Precioso, Stefano Melacci, Adrian Weller, Pietro Lió, and Mateja Jamnik. Concept embedding models: Beyond the accuracy- explainability trade-off. Advances Neural Information Processing Systems (NeurIPS), 2022.
[85] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreason- able effectiveness of deep features as a perceptual metrie, In Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
15

---

Supplementary Materials
In the following, provide details on Neural Concept Binder, experimental evaluations well as additional evaluations.
Impact Statement
Our work provides new framework for unsupervised concept learning for visual reasoning. improves the reliability of the unsupervised concept learning by explicitly including both inspection and revision of the concept space in the framework. NCB thus makes an important step towards more reliable and transparent AI, by providing an interpretable symbolic concept representation. This representation can be utilized within reliable and proven symbolic methods, or to improve transparency of neural modules. However, the concepts learned unsupervised, one has to keep in mind that they are not necessarily aligned with human knowledge, and might require inspections to achieve this. As NCB features concept revision via human feedback, it is also necessary to consider user with malicious intents could modify the
that these revisions could have negative effects. memory and thus make the concept space incorrect. The fact that the learned representation of NCB explicitly inspectable can, however, prove to be helpful in limiting such malicious interventions.
A Details on Neural Concept Binder
A.1 Details on Systematic Binding and Slot Attention
The binding mechanism (SysBinder) of Singh et al. [65] allows images to be encoded into continuous block-slot representations and relies the recently introduced slot attention mechanism 43) In slot attention, so-called slots, 8 E R Ng Dg (each slot has dimension Dg), compete for attending to parts of the input via a softmax- baatma attention. These slot encodings are iteratively updated and allow to capture distinct objects or image components. The result is an attention matrix A E R x for input x E RDP. Each entry A; corresponds to the attention weight of slot for the input Based on the attention matrix, the input is processed to read-out each object by multiplying with the input resulting in a matrix U E Ns Da
SysBinder now performs an additional factor binding on the vectors u, of U. The goal of this factor binding mechanism is to find distribution over codebook memory for each block in u, i...
u². This codebook memory (one for each block), M RxxDg consists of a set of K learnable codebook vectors. Specifically, for each block j an RNN consisting a GRU and MLP component iteratively updates the j-th block of slot 8;, sy. based on ui and previous sy Finally, a soft information bottleneck is applied where each block &f performs dot-product attention over the codebook memory leading to the final block-slot representation:
s .MP))))))) )
5/-
softmax K
MP
VDB
This process is iteratively refined together with the refinement processes of slot attention. Overall, the encodings of SysBinder represent each object in an image by slot with Ng blocks where each block represents a factor of the object like shape or color.
Note that in the main text, the final of is denoted as zạ-
A.2 Selection Function
In the default setting, NCB selects that encoding from the retrieval corpus with the minimal distance to infer a corresponding concept representation. We further explore a top-k approach for the selection function with k 1. In this case, selects the values v,, for the k E N closest encodings in the retrieval corpus and the resulting of is obtained via majority vote over these values. Additionally, via this selection approach the probability of of based on the occurrence distribution over the top-k values can be estimated. We provide ablations regarding this in our evaluations in Suppl. F.I
16

---

Algorithm Training NCB: Given a set of images, X, a block-slot encoder, yo, an unsupervised
clustering model ha-
i < fit(go, X))
Step I: Optimize the block-slot encoder
2: Z + (x(
>> Step 2.1: Gather block-slots from optimized g
3: Z t saleet_object_sloe
Step 2.2: Filter out non-object slots
4: for j E (1,... NB) } do
5: S t fit(h, )
> Step 2.3: Obtain clustering of Z
6: Rd t distill(e), 73)
b Step 2.4: Extract clustering representation into Ri
A.3 Details on Training
The first step (cf. L.1 in Alg. 1) optimizes the encoder g provide object-fatooried block- encodings. It is optimized for unsupervised image reconstruction based on the decoder model, l6 :2 3 RR. (ef. Fig. 2) and mean squared error loss: I, LMsE( g' (g(x))). In practice, additional losses have been shown to be beneficial for further improving the obtained block-slot
encodings 65 64).
The goal of NCB's second training step is to obtain the retrieval corpus, R. This procedure is based on obtaining optimal clustering of block encodings via an unsupervised clustering model & and distilling the resulting information from A into explicit representations in the retrieval corpus. This step is divided into several substeps (cf. L.2-6 in Alg. I.. It starts with gathering set of block-slot encodings Z = (X) As can include slots which do not encode objects but, egg., the background, we first select the "object-slot" encodings from Z. This step results in 2 C Z and consists of heuristic selection based on the corresponding slot attention masks (described in the following section).
For each block j we next perform the following steps: (i) clustering model, موg (cf. Fig. 2), is fit to find a set of clusters within 2' thereby identifying No E N meaningful clusters. The learning of this optimal clustering is based on an unsupervised criteria, e.g., density based scores [53]. Ideally, this clustered together in the corresponding
leads to that objects that share similar block encodings latent block space, whereas objects that possess very different block encodings are associated with distant clusters. This resulting clustering is stored in h 's internal representation which
denote as od (e.g., the merge tree in a hierarchical clustering method 12 1 54 Importantly, has is optimized individually for each block. (ii) In the distill step representative block encodings of each cluster, enc?, are extracted from h 's internal representation, G¹. Hereby, every enc? can represent either an averaged prototype instance-based exemplar encoding of cluster. This is performed for every identified cluster, v E (1,.... Nc} and is based on Z and Φ² As a result, the tuples (enc', v) are explicitly stored in the retrieval corpus RS. The final retrieval corpus consists of the set of individual RN
corpora for each block, R = [R]
We note that in practice, it is further possible to finetune the block-slot encoder, g. through supervision from the hard binder, e.g., once initial categories have been identified and can be achieved via a standard supervised approach. Ultimately, this allows for dynamically finetuning NCB's concept representations.
Heuristic object-slot selection. In the following we describe the process of identifying the slot which contains an object. This is based on heuristically selecting slot ids based on their corresponding slot attention values. Importantly, this approach can select object-slot ids without additional supervision, e.g., via (GT) object segmentation masks.
In principle, our object-slot selection approach finds the slots which contain slot attention values above a predefined threshold, 8 (0, 1J. However, selecting such a threshold can be cumbersome in practice. In our evaluations we therefore select only single slot per image, i.e., that slot which contains the maximum slot attention value over all slots. Essentially. this sets the maximum number of selected slots per image to 1 and in images that contain one objects represent no loss of object relevant information. preliminary evaluations we observed that the consensus between object-slot selection based on GT object segmentation masks (matching object segmentation masks with slot attention masks) and our maximum-baat based selection heuristic is 99.45% over 2000 single object images.
17

---

A.4 Instantiating Neural Concept Binder
We instantiate NCB's soft binder via the SysBinder approach of Singh et al. [65] which has been shown to provide valuable, object-factor disentangled representations. Thus, the soft binder was trained as in the original setup and with the published hyperparameters. Furthermore we instantiate the clustering model, h, via the powerful HDBSCAN method [22 [3 [54 (based on the popular HDBSCAN library Hereby, h's internal representation, @, consists of the learned hicrarchical merge tree. In practice we found it beneficial to perform grid search over h's hyperparameters based on the unsupervised density-based cluster validity score [53]] The searched parameters are the minimal cluster size (the minimum number of samples in group for that group to be considered a cluster) and minimal sample number (the number of samples in a neighborhood for a point to be considered as utilize the excess of mass algorithm and allow for single clusters. We performed the training of the retrieval corpus, i.e., fitting h, dataset of images containing single objects for simplifying the subsequent concept inspection mechanisms of our evaluations. However, this can easily be extended multiple object images by utilising the soft binder's slot attention masks to identify relevant objects in an image. Finally, we instantiate the retrieval corpus as a set of dictionaries and, unless stated otherwise, we utilise a retrieval corpus which contains one prototype and a of exemplar encodings concept. Furthermore, S2 represents the argmin selection function and we utilize the euclidean distance as d(..-). is important note that does not make any assumptions about the number of clusters, Nc. Thus, although h fits a clustering to best fit the block-slot encodings of a block, it can potentially provide an overparameterized clustering, e.g. by representing one underlying factor such as "gray" with several clusters. This highlights the importance of task-alignment, e.g., for symbolic downstream tasks, and concept inspection for general concept alignment. We refer to our code for more detail 5 where trained model checkpoints and corresponding parameter logs are available.
point) each over the values [5, 10, 15, 20, 25, 30, 50, 80, 100). Moreover,
A.5
Computational Resources
The resources used for training NCB were: CPU: AMD EPYC 7742 64- Core Processor, RAM: 2064 GB, GPU: NVIDIA 1100-SXM4-40GH GPU with 40 GB of RAM. Hereby, training the SysBinder model [65] is the computational bottleneck of NCB where we utilised two GPUs per SysBinder run. Training for 500 epochs took 108 GPU hours. The fitting of (including the grid search over hyperparameters) was performed on the CPU and finished within few hours.
B Details on CLEVR-Sudoku
Sudoku CLEVR Examples
K=10
K=30
K=50
Figure 7: Examples of Sudoku CLEVR for different values.
CLEVR-Sudoke provides Sudokus based on the datasets CLEVR and CLEVR-Easy, Classic Sudokus have a 9x9 grid which is filled with digits from I to 9. In CLEVR- these digits are replaced by specific attribute combination, e.g., "yellow" and
images of objects. Hereby, a digit corresponds "sphere". Consequently, digits of the Sudoku are replaced by images of objects with these attribute combinations. These images each contain one object. To indicate, which attributes correspond to which digit, candidate examples of the digits are provided. The number of these examples is flexible
"Code available
18

---

parameter, in our evaluations we used N E {1,3,5, 5, 10). Further, the number of images provided in the Sudoku grid is flexible as well. In our main evaluations we only considered with K = 30, meaning that 51 of the 81 Sudoku cells are filled and 30 are left to complete. For additional investigation we considered values for K' (10,50) 50) as well. Examples of those Sudokus for Sudoku CLEVR are shown in Fig. 7| The dataset has number of 1000 samples for Sudoku CLEVR-Easy Easy and Sudoku CLEVR respectively for each value of K. Each sample has different puzzle and a distinct set of images, no image is used twice for one puzzle"
C Datasets
CLEVR. Briefly, CLEVR [3 image contains multiple 3D geometric objects placed in an illumi- nated background scene. Hereby, the objects can possess one of three forms, one of 8 colors, one of two sizes, one of two materials and random position within the scene.
CLEVR-Fasy. CLEVR-Fasy (65] images are similar to CLEVR images, except that in CLEVR-Fasy the size and material is fixed over all objects, i.e., all objects are large and metallic.
CLEVR-Hans3. The CLEVR-Hans3 (68) represents a classification dataset that contains images with CLEVR objects where the image class is determined based on the attribute combination of several objects (e.g., an image belongs to class 1 if it contains a large, gray cube and a large cylinder). Furthermore, we utilize confounded and non-confounded version of CLEVR-HanS' In the confounded case (i.e., the original dataset) the train and validation set contains spurious correlations among object attributes (e.g., all large cubes are gray in class 1) that are not present in the test set (e.g.. large cubes of class take any color). In our evaluations investigating only neural-based classification we utilize the original validation split as the held-out test split and select subset from the original training split as validation set. Thus, the non-confounded version corresponds to a standard classification setup in which the data distribution is identical over all three data splits. Lastly we provide evaluations on a single object version CLEVR-Hans3 (class I: a large, gray cube; class 2: small metal cube; class 3: a large, blue sphere; cf. Tab. 3) and the original, multi-object version.
D Baseline Models
We note upfront, that all SysBind configurations below were trained for many epochs as NCB, followed by an additional finetuning for 2 epochs on the same dataset that was used to distill NCB's retrieval corpus.
SysBind (cont.). This denotes the original SysBinder configuration which was trained in 65 and provides continuous block-slot encodings. We refer to the original work for hyperparameter details.
SysBind. This denotes SysBinder configuration that was trained in 665 However, inference time we perform discretisation via an argmin operation over the attention values to each block's prototype codebook.
SysBind (hard). This denotes a configuration in which the SysBinder model was trained via a codebook attention softmax temperature of 1e 4, resulting in a learned discrete representation.
SysBind (step). SysBinder (step) is trained by step-wise decreasing this temperature his denotes configuration in which the SysBinder model was trained via step-wise decreasing codebook attention softmax temperature (with a decrease by factor of 0.5 every 50 epochs, starting from 1.).
NLOTM. NLOTM [80] builds on the principles of SysBinder and incorporates a Semantic Vector- Quantized (SVQ) Variational Autoencoder along with the Autoregressive LoT Prior (ALP). The SVQ component facilitates discrete semantic decomposition of a scene by learning hierarchical, composable factors that correspond closely to objects and their attributes in visual scenes. We refer to the original work for details.
Supervised Concept Learner. This corresponds to slot attention encoder 43 that was trained for set prediction (i.e., in a supervised fashion) to predict the object-properties for every object in a CLEVR image. We refer Locatello et al. 43 and Stammer et al. (68) for details.
'The code for generating the dataset is available in our code repository, the already generated data files are accessible under http:::ttts::ss/te http:///gaaacccootttttttte//aaa. AUUA:///eteteAAMMMMUUAAAAAAAAALA STLALLLEATAAAAEEASSAAAASAE SuduuSu
19

---

E
Details on Experimental Setup
E.1
Classifying abject-properiee from concept encodings
For our evaluations in the context of (Q1) we utilise a decision tree as classification model that is trained set of concept encodings predict corresponding object properties, e.g., spher,, cube or cylinder. Importantly, we train a separate classifier for each property category, e.g., the categories shape, color, material and size in the case of CLEVR, and average accuracies over these. The classifiers parameters correspond to the default parameters of the sklearn library 5
E.2 CLEVR-Sudoku evaluations
For our CLEVR-Sudoku evaluations we use solver that combines symbolic classifier with a constraint propagation based algorithm. To solve CLEVR-Sudokus, it is at first required to detect the underlying mapping from the object attribute combinations to the digits via the provided candidate examples. For this, we require a symbolic classifier to learn this mapping, which in the case of our evaluations is achieved via decision tree classifier. For each evaluated model the concept encodings of the candidate example images of a CLEVR- classifier. Hereby, the corresponding digits are the labels to be predicted. With the predictions of the trained classifier the concept encodings of the images in the Sudoku grid are classified to get symbolic representation of the Sudoku, i.e., map the images in the cells their corresponding digits. Based on this numerical representation of the puzzle, we use an algorithm from (55) that uses combination of constraint propagation [10] and search. The algorithm keeps track of all possible values for each cell. Within each step, the Sudoku constraints are used to climinate all invalid digits from the possibilities. Then the search of the algorithm select digit for a non-fille non-filled cell. Based on this digit, the possibilities are updated for all other cells. When there is a constraint violation, the search-tree is traversed backwards and other possible digits for non-filled cells are explored. This process is repeated until the Sudoku is solved (in case the initial state inferred from the objects was correct) or until there is no possible solution left (meaning that the initial state was incorrectly inferred from the objects). The implementation of the algorithm is based on the code from Finally, to avoid errors due to random seeding of the classifier, for each puzzle we fit independent classifiers (each with different seeds) to predict the corresponding mapping. For the results in our evaluations we average the performance over these 10 classifier seeds.
are retrieved and provided as input to the
Lastly, the evaluations in the context of (Q2) are based on the trained (NCB) models of (QI).
E.3 Obtaining Revisory Feedback
We note that the evaluations in the context of (Q3) are based on the trained NCB models of (Q1).
Revisory feedback for downstream Sudoku task.
To revise its discrete concepts, NCB offers the possibility to delete or merge clusters in the blocks. In the case of merging, the prototypes and exemplars of the clusters to be merged get aggregated that they all map to the same concept symbol. For deletion there are several processing cases, depending on how many categories are in the block and how many are supposed be deleted:
Case if all clusters from a block should be deleted (or if there only one concept in the block, which should be deleted), we map all samples to the same concept. This results in the block containing no information (we keep the block to avoid issues with the dimensions of the concept representation).
Case 2: all clusters but one are to be deleted. In this case we still want to distinguish between the presumably "informative" cluster and the uninformative other clusters. Therefore map all the blocks to be deleted to one cluster id instead of deleting them completely.
Case at least two clusters should not be deleted. this case, we completely remove the encodings of the to-be-removed clusters. The cluster id for these clusters no longer exists in the retrieval corpus.
20

---

Feedback via GPT-4, We systematically prompt GPT-4 [56] for receiving revisory feedback. We provide example prompts in [isting First, we ask GPT-4 to name relevant object properties for a set of example images, e.g., "shapes: [cube, cylinder], color: [red, blue]". Based on these provided property lists we ask GPT-4 to provide a descriptive list of each exemplar object's image for each concept of each block, e.g., [Exemplarl: [cube, red], Exemplar2: [cube, blue). }". Based on these descriptions we identify whether all exemplar objects of one concept share a common subproperty, egg. "cube". If there is no common subproperty, the concept should be removed from the retrieval corpus. In a second step we evaluate whether all exemplar objects from two separate concepts share a common subproperty. In this case we decide to merge the concepts based on GPT-4's analysis. We finally integrate GPT-4's feedback into NCB's 's retrieval corpus via the procedures described above.
Feedback via simulated humans. To simulate feedback by a human user, we utilise decision tree (DT) classifier to classify attributes of objects based on NCB's discrete concepts (similar to Q1). For this, we transform the concept-slot encodings into multi-hot encodings. We then extract the importance of the concepts from the trained DT classifier. Based on this we select "unimportant" concepts to be deleted based on the procedures describe above. Note that in this setting we do not query for feedback considering the merging of concepts.
E.4 Neural Classification
We note that the evaluations in the context of (Q4) are based on the trained NCB models of (Q1).
Neural classifier. In the context of the classification evaluations (Q4) we utilize the setup of Stammer al. 68 Specifically, a transformer (37 is trained to classify images from the CLEVR-Hans3 dataset given encodings that are, in turn, obtained from either NCB or supervised trained slot attention encoder [43 (SA). In the case of utilizing NCB's s encodings we transform the concept-slot encodings into multi-hot hot encodings to match those of the SA-based setup. We refer to Stammer et al. and our code for additional details concerning this setup.
688
Obtaining explanations from the neural classifier. We provide the explanations in Tab. 3 for the single object version of Fig. 14 To obtain these explanations for the neural classifier utilize the approach of Stammer et al. [68] which is based on the integrated gradients explanation method [70]. This estimates the importance value of each input element (in this case input concept encodings) for a classifiers final decision. We remove negative importance values and normalise the importance values as in [68]. We then sum over the importance values corresponding to images of a class, normalise the values per block and binarize these aggregated and normalised importance values via the threshold of 0.25 (i.e., importance values above 0.25 are set 1, otherwise 0). This provides us with binary vector indicating which concepts are considered important per block. We illustrate these investigations via explanations from one model.
Explanatory interactive learning (XIL). Explanatory interactive learning (+ XIL on NN) is used to mitigate the confounder in the CLEVR-Hans dataset. Hereby, (simulated) human feedback on the explanation of the neural classifier is used to retrain the classifier via the loss based approach of Stammer al. (6). The feedback annotations mark which of NCB's concepts should not be used for the NN's classification decision. This is integrated into the NN by training the model to provide (integrated gradients-based explanations that do not focus on these concepts. We refer to Stammer et al. (68] for details. The second form of interactive learning (+XIL on NCB concepts) is directly applied on the NCB's concept representation. Specifically, concepts from NCB that encode information concerning the irrelevant, confounding factors are simply set to zero, corresponding to not being inferred for the object in the image. E.g., if the NCB infers concepts concerning the color "gray" to be present in an object and the underlying confounder is the color "gray" the corresponding concept activations of the NCB's 's prediction are set to zero, i.e., no gray. Then the neural classifier is retrained on the new concept representations. Next to a better performance, the advantage of this approach is that does not require the more costly loss-based XII. training loop. We illustrate these investigations via interactions on one model.
21

---

Table Ablation of NCB's selection components for classifying attributes from concept representa- tions. Best results are in bold.
Train
NCB (P)
NCB (P+E)
NCB (P+F, topk)
CLEVR-Easy
N=2000
98. 70=1.0
09.02+1.00
98.93+11.10
N=200
97.11=2.16
98.50+1.80
98.42+1.31
N=50
94.31=4447
95.8712.93
95.7243.04
N=20
90.50=7.09
94.22+4.11
94.15+4.14
CLEVR
N=2000
96.77=2.63
97.26
97.17+2.68
N=200
96. 96.41=2.64
96. 80
96.80+3.04
N=50
94,29=4.78
94.67+4.65
94.10+5.25
N=20
87.55=5.35
88.5714.68
88.42+4.63
Table 5: Ablation: Classifying attributes from concept representations with sub-optimal NCB components. The left column serves as reference and represents the configurations used in the main evaluations, ie.. where the soft binder was trained for 600 epochs and the clustering model represented the HDBSCAN approach that was optimized via a grid-search over its corresponding hyperparameters.
NCB (50 epochs)
NCB (100 epochs)
NCB (w/o grid search)
NCB (kmeans)
Train
NCB
CLEVR
N=2000
97.26
1..19:112
94.91:33.5
97. 97.69
97.26:12.60
N=200
96.80=3.01
03.69=1.08
93.83:22900
96.80=3.00
96.01+3351
N=50
94.67
89. 39.10#:4.29
89.67.+6.95
94.46%55.00
$7.654+8.79
N=20
88.5724.68
83.46:26.08
88.48:12.36
90.51+4.40
73.52+11002
F
Additional Quantitative Results
F.
Encoding Expressivity
In our evaluations in Tab. it appears that training for discrete encodings via SysBinder (hard) leads to no learning effect of the model altogether. In contrast training step-wise via SysBinder (step) provides better results, even slightly above the encodings of SysBinder (i.e., training for continuous representations and then discretising via argmin). Lastly, we observe that NCB's ' encodings lead to much lower performance variance compared all baselines. Particularly SysBinder (step)'s high variances, hint towards issues with local optima.
We further provide ablations in the context of (Q1) different component choices of NCB in Tab. Specifically, we investigate the effect of a top-k selection function as well as the influence of using only prototype encodings in the retrieval corpus (NCB (P)) versus using prototype and exemplar encodings (NCB (P+E)). Unless noted otherwise, the NCB configurations in Tab. 4jutilize the argmin selection function. We note that when using prototypes, the average encoding of all elements in cluster is formed, resulting in one prototype encoding per cluster in R³, In the second variant, we extend the prototypes with exemplars for each cluster. Exemplars are representative encodings for this cluster added to the corpus, resulting in larger corpus, which potentially provides an improved structure of the encoding space. Indeed, we observe that NCB provides the best performances via the argmin selection function and utilizing both prototype and exemplar encodings. This was the setting used in all evaluations of the main paper.
F.2 Ablation Analysis of Suboptimal NCB Components
Lastly, in the context of (Q1) we further refer to ablations in Tab. 5 on the specific implementation choices of the NCB instantiation our evaluations. We hereby investigate the effect of sub-optimal soft and hard binder components concept encodings. Specifically, we investigate (i) the effect when the soft binder, i.e., SysBinder encoder, trained for fewer epochs, resulting in less disentangled continuous representations, and
a classifier's ability to identify object attributes from NCB's
22

---

(ii) when the HDBSCAN model of the hard binder was not optimized via parameter grid search or replaced with more rudimentary clustering model, i.e., k-means clustering approach [45]
In the leftmost column of [Tb. 5] we provide the performances of the NCB configuration of our main evaluations reference. As reminder, hereby, NCB's soft binder was finetuned for 500 epochs, and its hard binder component contains a clustering model based on the HDBSCAN approach that was furthermore optimized via grid search over its corresponding hyperparameters. Focusing on the next two columns right of the baseline, we observe that when the soft binder component is trained for fewer epochs than the baseline NCB we indeed observe a decrease in classification performance. Notably, however, we still observe higher performances in comparison to the discrete SysBinder configurations (cf. [Suppl. F.T), but also when compared SysBinder's s continuous configuration (for N = 20). Focusing next on the rightmost column of Tab. 5 where NCB's 's clustering model was replaced with the more rudimentary k-means clustering approach, we observe strong decrease in classifier performance. This is particularly true in the small data regime (N = 50 and N = 20). Surprisingly, focusing on the second to the rightmost column, we observe that when we select the default hyperparameter values of the HDBSCAN package (rather than performing grid-search over these), the classifier reaches slightly improved performances than via the baseline NCB configuration (particularly for N = 20). Thus, in this particular case, the default values seem practical. However, this cannot be guaranteed in all future cases, and we still recommend performing a form of grid search if no prior knowledge can be provided upfront the specific density- -based cluster validity score used for selecting the optimal cluster parameters has been sub-optimal and leave investigating other, more optimal selection criteria for future work.
optimal parameter set. We postulate that
Overall, our ablation investigations indeed indicate that we obtain less expressive concept encodings via NCB with less powerful sub-components. However, we also observe certain amount of robustness of our NCB instantiation towards sub-optimal components.
F.3 Analysis of Learned Concept Space
We here provide a brief analysis NCB's learned concept space. These evaluations were performed on the models that were trained in the context of (Q1). Specifically. in Fig. 8, we provide the number of obtained concepts over all blocks (averaged over the initialization seeds) both for CLEvR-Easy and CLEVR. We observe a much larger number of concepts overall for the CLEVR dataset but also a much larger variance in the number of concepts. This is largely due to that in CLEVR- Easy NB more overparameterizeed concept space in the case of CLEVR. Further, in Fig. 9 we present the distribution of the number of concepts per block over all 3 NCB runs, both for CLEVR-Easy and for CLEVR. We observe that while most blocks contain maximally 20 concepts for CLEVR-Easy and 50 for CLEVR, there are several block outliers which contain much greater set of concepts.
whereas in CLEVR NB- 16. Thus, the models are able to learn
These represent cases in which the initial block- slot encoding space was uninformative to begin with and, therefore, difficult to find some form of useful clustering via h. Where some of these blocks only contained irrelevant information in general, some blocks encoded positional infor- mation, which represents continuous variable to begin with and is thus unlikely to be well represented via clustering.
Number Concepts
Concepts
500
CLEVR-Easy CLEVR
1.0
1.5
2.0
Figure 8: Average number of concepts (over all blocks) in NCB's 's retreival corpus.
F.4 CLEVR-Sudoku Evaluations
In our evaluations on (Q2) we observe that, interestingly, for Sudoku CLEVR the supervised object classifier shows better results than for CLEVR-Easy, This seems counter-intuitive, however, in BLEVR-Eaay-Sudoe
digit labels are mapped to combinations of attributes that only stem from two categories, shape and color (in contrast to four categories CLEVR-Sudoki thus making it more likely to obtain recurring attributes over several digits (e.g., digits 3, 4 and 5 of Fig. 4|al depict green objects). Thus, an error occurs the digit classification due to errors concerning one attribute the effect of this error will have a larger effect. Moreover in the case of CLEVR-EaSS we observe that in comparison to the supervised model, whose property misprediction errors can lead to large issues in the downstream module. NCB's 's unsupervised and somewhat overparameterised concept space
23

---

Number Concepts per Block
250
#onccpps per Block
200
150
100
50
CLEVR-Easy
CLEVR
Figure 9: The distribution of number of obtained concepts per block both for CLEVR-Easy and CLEVR. These values computed over all seeds.
Sudoku CLEVR-Easy
Example
3 Examples
5 Examples
10 Examples
100
Error (%)
Sudoku CLEVR
1 Example
3 Examples
5 Examples
10 Examples
100
100
100
100
Error (%)
T
T
Concepls
(supervised)
SysBirder (unsupervised)
(unsupervised)
NCB revised (GPT-4)
NCB revised (human)
NCB revised (human)"
Figure 10: Error ratios (%) of the digit classification in CLEVR-Sudoku based different symbolic concept encodings.
appears to dampen this issue, thus leading to a higher number solved puzzles, e.g., for 3, 5 or 10 examples.
In Fig. we report the emror in predicting the underlying digits of the CLEVR-Sudokus We observe that the errors of SysBinder (unsupervised) are drastically higher than the errors of the other methods. These high classification errors further explain this method's 's low performances, i.e. did not allow to solve any Sudoku. It can further be seen that for one example per digit the digit classification emrors are much higher. This reasonable as hereby the difficulty for the classifier is also higher. However, with increasing number of examples the classifier's s errors decrease. The relations between the errors in the digit prediction and the overall performance CLEVR-SSuolf are similar which sensible since the error is decisive for the number of solved puzzles.
24

---

Sudoku CLEVR-Easy with 5 Examples
K=10
K=30
K=50
100
100
100
Solved Puzzles (%)
40
40
GT Concepts
SA (supervised)
SysBinder (unsupervsed)
NCB (unsupervised)
NCB revised (GPT-4)
NCB revised (human)
NCS revieed (human))
Sudoku CLEVR with Examples
K=10
K=30
K=50
100
Solved Puzzies (%)
(supervised)
SysBinder (unsupervsed)
(unsupervfsed)
NCB revised (GPT-4)
NCB revised (human)
NCB revised (human)"
Figure 11: Solved Sudokus (%) of Sudoku CLEVR-Easy and Sudoku CLEVR with different values for K (empty cells).
We further evaluate the influence of the number of missing images per Sudoku. For this we consider Sudokus with K E {10, 30,500 50). The results on these variations with 5 candidate example images are reported in Fig. 11 We see that the more empty cells there are in a Sudoku's initial state (higher K',, the more Sudokus are solved. This is due the lower probability misclassifying an image inside the Sudoku cells, there are less images to classify. This pattern is observable for all of the different concept encodings we compared.
E.5 Revision Statisties
Deletions during Revision
CLEVR-Easy
CLEVR
1000
We provide statistics of the number of
resulting removal requests per agent iniF 12 For the revision of CLEVR- Easy concepts we can see that GPT-4 detects only few concepts delete while via simulated human revision more concepts get deleted. In our ini- tial evaluations (cf. Fig. 4h we had observed that human revision leads
Deletions
Revision by GPT-4
Figure 12:
Average number cluster deletions over all blocks via GPT-4 and simulated human user revision.
substantial improvements while GPT-4's revision even reduces performances slightly. For CLEVR- Sudoku Fig. 12] we specifically observe that the overall number of deletions via GPT-4 is sig- nificantly higher. Interestingly, GPT-4 detects on average more blocks to delete here but also has higher variance over the different NCB runs. We hypothesize that this very "conservative" revision leads to the removal of concepts that actually contain valuable concept information, thus leading to less expressive concept encodings overall. Ultimately, this is due to mistakes in GPT-4's s analysis of provided images (cf. Suppl. E.3
25

---

F6 Dynamically Discretising Continuous Factors via Symbolic Revision
In our second set evaluations in the context of (Q3) we investigate the third form of symbolic revision as introduced in [Sec. 3.3) adding concept information the hard binder's retrieval corpus. Hereby, we focus on the task learning a novel concept that had only been stored implicitly in the soft binder's representations, but not explicitly in the hard binder's representations. Specif- ically, we focus on positional concepts of CLEVR objects where the underlying GT position is represented via continuous values. Overall, it is debatable whether one, in principle, should or even can represent such a continuous underlying feature via a discrete concept representation.
In this of evaluations which it is necessary to identify coarse categorisa- tions of object's position, e.g., whether the object is placed in the left or right half of an image. We hereby simulate human stakeholder that, having identified the block that generally encodes position information, revises the corresponding concept en- codings. This revision is performed in two ways: (i) by iterating over all of the block's concepts and merg- ing concepts into left and right concepts or (ii) by replacing all information in with encodings from selected set of positive example images for the two relevant positions. Fig. 13 presents the results of training a classifier predict the attributes "left" and "right" from NCB's s encodings (we here focus only on one seeded run for illustrations) with different types of revision. We observe that both allow to easily retrieve relevant information from NCB's 's newly revised concept space. These results illustrate the important ability to easily adapt the hard binder's concept representations by dynamically re-reading out the information of the soft binder's ' representations in use-case based manner. The results further illustrate the effect of adding prior knowledge to NCB's concept representations, thereby potentially reducing the amount of inspection effort required on the stakeholder's side, egg. in comparison to the merge revision.
investigate setting
Left vs. Right Position
100
80
Acc. (%)
60
20
NCB
NCB w. exem. NCB w. add 20 exem.
NCB w. merge
Figure 13:
Test accuracy (%) for classifying objects as placed left or right in a scene.
F.7 Classifying CLEVR-Hans3
In our final evaluations (Q1) we highlight the advantage of NCB's concept encodings when combined with subsymbolic (i.e., neural) modules for making their decisions transparent. Specifically, while a discrete concept representation is technically not required for neural modules, it has a key advantage: discrete and inspectable representa- tion allows for transparent downstream computations. We highlight this property in the context of image classification on variations of the benchmark CLEVR- Hans3 dataset [68). For these evaluations we revert to training a set transformer [37] (denoted as NN in the following) for classifying images when provided the unsupervised concept encodings of NCB image representations. We denote this configuration as NCB NN and compare set transformer is provided concept encodings from supervised slot attention encoder, denoted as SA + NN. In Fig. 14 we obseve that NCB's 's concepts per- form par with those learned supervisedly, each reaching held-out test accuracies higher than 95%.
Classification via NN
Single Otj Multiple Ob).
SA (supervised)
(9)
60
NCB
(unsupervised)
70
Figure 14: Test accuracy (%) for classifying CLEVR-HansS images with a neural classi- fier that is provided concept representations of NCB and of a supervised trained slot atten- tion encoder. We differentiate here between class rules based on one object and multiple objects.
configuration in which the
F88 Confounding Evaluations
For the confounding mitigation evaluations in the context of (Q4) we train the NCB + NN configuration on the confounded version of CLEVR-Hans3,
where we hereby focus the single object class rules similar to those in Fig. 14 In this case all
26

---

Concept
Concept:
Concept
Further examples of interventional inspection. By swapping the encoding of block 2 with different exemplar encodings from different concepts, the shape (which is encoded by block 2) is changed. When swapping the encoding with an exemplar of the same concept, the shape remains unchanged.
Figure 15:
large cubes of class one images posses the color gray at training time, but arbitrary colors at test time. We observed accuracies of NCB NN the confounded validation set of 99. 22% against the non-confounded test set 79.29%. This very high validation accuracy versus significantly reduced test accuracy indicates that the classifier is strongly influenced by the datasets underlying confounding factor.
G Qualitative Results
Fig. 15 further exemplifies the inspection types Sec. 3.3. Fig. 16 and Fig. 17 represent qualitative inspection results of NCB's learned concepts. We specifically present implicit inspection via exem- plars of concepts from two blocks from NCB when trained on CLEVR-Fasy. One observe that block 2 (Fig. 16) appears to encode shape concepts, however contains one ambiguous concept. We further observe that Fig. 17 appears to encode color concepts, whereby it contains one ambiguous concept (concept 8) and two concepts that appear to both encode the color purple (concept 9 and 10) which could potentially be merged.
27

---

Block
Concept 1
Concept
Concept 3
Concept
Concepts of Block 2 for NCB with CLEVR-Easy. We here provide implicit inspection examples (i.e., via exemplars of each concept). We observe that block 2 appears encode shape information (concept 1-3) and contains one ambiguous concept (concept 4).
Figure 16:
28

---

Block 8
Concept
Concept 2
Concept
Concept 4
Concept 5
Concept
Concept
Concept
Concept
Concept 10
Figure 17: Concepts of Block 8 for NCB with CLEVR-Easy. We here provide implicit inspection examples (i.e., via exemplars of each concept). We observe that block appears to be encoding color information, contains ambiguous concept (concept and concepts that appear to both encode the color purple (concept 9 and 10).
29

---

Table 6: Percentage of solved CLEVR-Sudokus for different number of example images.
Sudoku CLEVR-Easy
Example
3 Examples
5 Examples
10 Examples
GT Concepts
100.0 = 0.00
100.00 + 0.00
100.00 + 0.0
100.00 0.00
SA (supervised)
35.22 5.63
40.07 = 3.76
42.07 = 3.14
44.36 = 2.54
SysBinder (unsupervised)
0..00 + 0.00
0.00 + 0.00
0.00 0.00
0.00 = 0.00
NCB (unsupervised)
6.13 + 4.42
47.30 + 33.06
54.95 + 31.86
63.21 + 27.45
NCB revised (GPT-4)
34.61 + 43.46
48.25 + 34.80
54.31 + 30.85
61.83 = 26.10
NCB revised (human)
54.41 +: 37.40
64.00 +: 28.53
67.07 = 24.83
70.10 = 21.52
NCB revised (human)
50.81 + 45.38
62.00 1 34.78
66.40 + 30.38
70.43 = 26.35
Sudoku CLEVR
GT Concepts
11 0.00
100.0 0.0
100.0 0.00
100.0 0.00
SA (supervised)
54.9 1 5.99
69.99 1.
1
77.71 0.38
SysBinder (unsupervised)
0.00
0.00
0.00
1
NCB (unsupervised)
0.01
8.46
26.62
38.24 26.97
NCB revised (GPT-4)
1.11
16,18
25.86
27.75 = 20.14
NCB revised (human)
3.23
12.1963..66
44.8
48.97 = 37.13
NCB revised (human)*
4.84 4.82
54.0 + 25.43
66.69 21.46
72.68 = 19.53
Table 7: Error ratios on digit classification of CLEVR-Sudokus for different number of example images.
Sudoku CLEVR-Easy
Example
3 Examples
5 Examples
10 Examples
GT Concepts
0.00 = 0.00
0.00 0.00
0.0 0.00
0.0 0.00
SA (supervised)
7.23 = 0.89
5.25 + 0.16
4.89 + 0.09
4.55 - 0.09
SysBinder (unsupervised)
88.69 + 0.05
48.12
87.66 0.31
87.30 + 0.40
NCB (unsupervised)
23.57 + 1.02
3.55
2.16 1.74
1.35 1.13
NCB revised (GPT-4)
13.21 + 9.89
3.21 + 2.56
2.1 1.64
1.36 1.06
NCB revised (human)
4.94 5.68
1.45 1 1.49
1.12 + 1.05
0.95 0.83
NCB revised (human)*
6.55 1 6.37
1.79 1: 1.73
1.31 + 1.24
1.06 + 1.00
Sudoku CLEVR
GT Concepts
0.00 0.00
0.00 0.00
0.00 1 0.00
0.00 0.00
SA (supervised)
3.78 0.79
1.85
1.52
1.30 1: 0.08
SysBinder (unsupervised)
08.8110.00
88.67
88.68
8.17
NCB (unsupervised)
4.25
11.61
6.30
4.49
NCB revised (GPT-4)
34.10
9.85
8.11
7.16
NCB revised (human)
42.32 = 14.56
5.84
4.80 4.94
3.91 + 4.38
NCB revised (human)*
39.85 + 17.31
3.69 +
1.34 0.97
0.84 + 0.66
H Numerical Results
In our evaluations we presented the results on CLEVR-Sudoke in the form of bar plots. We refer to Tab. 7 and Tab. for the numerical values for the different variations of the dataset.
Tab.
30

---

Table 8: Percentage solved CLEVR-Sudokus for different values of with example images.
Sudoku CLEVR- --asy
K=10
K=30
K=50
GT Concepts
100.0 + 0.00
100.0 0.00
100.0 = 0.00
SA (supervised)
39.02 + 3.25
42.07
47.89 = 3.37
SysBinder (unsupervised)
0.00 0.00
0.00
0.00 0.00
NCB (unsupervised)
49.64 + 35.07
54.95 31.86
64.91 + 25.50
NCB revised (GPT-4)
48.62 33.92
54.31 30.85
63.62 25.52
revised (human)
60.23 28.77
67.07
24.83
75.05 20.51
NCB revised (human)*
60.50 + 35.24
66.40 + 30.38
73.39 24.96
Sudoku CLEVR
GT Concepts
100.00 + 0.00
100.00 0.00
100.00 0.00
SA (supervised)
68.96 + 0.65
73.92 1.69
78.46 1.72
SysBinder (unsupervised)
0.00 0.00
0.00
0.00 0.00
NCB (unsupervised)
21.18 + 14.85
28.62
34.79 21.84
NCB revised (GPT-4)
17.76 12.86
21.76
29.12 20.49
NCB revised (human)
40.30 34.34
44.80 + 35.58
35.05
NCB revised (human)*
60.10 + 24.36
66.69 +
74.54 16.02
31

---

Listing I: Prompts for GPT-4.
Property List Prompt:
You are provided images. image contains subimages. Each subimage depicts one object. Each object represents reflective geometric solid that is placed in a neutral gray background scene with light souree. Furthermore. each object has multiple properties.
e.g., color. shape. size material.
Each property can be subdivided into several sub-properties, e.g., brown is sub-property of the property color.
obect properties and subproperties that are depicted in all images. Ignore the background and the object 's luminance and reflectivity. Use the following answer template:
Please provide
property: [sub-property, sub-property. ...] property: -sub-property, sub-properly, ...]
Description Prompt:
You are provided an image. The image contains at most 25 subimages. Each subimage depicts one object. Each object represents a reflective geometric solid that is placed in neutral gray background scene with light source. Furthermore each object has multiple properties, c.g., color. Each property can be subdivided into several sub-properties, c.g., green is sub-property propertics and sub-propertics are the following:
the property color. The possible
INSERT PREVIOUSLY OBTAINED PROPERTY LIST
Focusing only on these properties please perform the following tasks. First, for every object in the image please list the sub-properties from the given lists that the object depicts Only name the sub-properties that given. Please the following format:
Object2: [sub-property. :::]
Objectl: [sub-property,
32

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We demonstrate empirically that NCB generates expressive encodings compa- rable supervised learned concepts, among others the novel CLEVR-Sudodk dataset. Additionally we highlight the inspectability and revisablility of the learned concept space.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer:
[Yes]
The paper contains an explicit limitation section, discussing potential limita- tions of this work.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
33

---

Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Experimental setup and training details are provided in the appendix. Ad- ditionally, the setup is available in the provided code, together with the
dataset.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
34

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Access to the code repository is provided. For the CLEVR- RuSuoou dataset, the generation files are provided in the code and the full datafiles will be made public upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
y /nips. /nips.cc//ppp http:/pbblcgguudeeeeeeeee www.eeebbiseiioooooll..oooo
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/ettails
Question:
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
The experimental details are provided in the appendix and the provided code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper provides results over multiple seeds. In all experiments, average and standard deviation are reported.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
35

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide details about the used computational resources in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttp:://eeurppe: httppp//ccc...cicccciiiiieeoeeee
Answer: [Yes]
Justification: The relevant parts of the code of ethics are discussed the impact statement and the remainder of the checklist.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The paper contains an explicit impact statement, discussing potential societal impacts.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
36

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific clusters), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not involve pretrained models of any kind. The released dataset is not scraped from the internet and does not require safeguards.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The authors of existing models and datasets used within the paper are cited and their licenses respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
37

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thooe cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The released code and the new dataset are both documented, including training and license information.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper did not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper did not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
38

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
39


Reviews:
[{"review_id": "U0HGZEIJjj", "reviewer": "Reviewer_edmj", "summary": "This paper introduces Neural Concept Binder, a framework for obtaining discrete concept representations from images without any supervision. The method is an extension of Neural Systematic Binder (SysBinder), adding a clustering step on top of the block-slot representations to obtain discrete concept representations. The resulting representations are interpretable and modifiable, as shown in the experiments. The model is additionally evaluated on property prediction and downstream tasks on modifications of the CLEVR dataset and shown to be able to leverage fewer training samples than SysBinder.", "questions": "1. For SysBinder (hard) and SysBinder (step), do the models train well with this temperature adjustment? E.g. do they exhibit decent reconstruction and slot decomposition?\n2. I’m not sure if I completely understand the analysis for Q4, but since this is done on the concept encodings, and not the discrete representations, can the same analysis be done with SysBinder representations? If so, does NCB offer any additional benefits here?\n3. How important is the choice of clustering algorithm to the results? What if we use a simple k-means clustering as is done in the original SysBinder paper?", "limitations": "Yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper investigates an important problem in learning discrete, interpretable concepts from images in an unsupervised way. The model is a logical extension of SysBinder in clustering the representations to obtain discrete concepts. The experiments show improvements in sample efficiency of these discrete representations over SysBinder’s continuous representations.", "weaknesses": "1. Since the solver used in the Sudoku experiments is the same across all baselines, it seems the determining factor of performance is in how well the digits are classified. Therefore, I do not believe framing this evaluation in the context of Sudoku adds any insight—in fact it seems to add unnecessary noise to the evaluation. The evaluation in the appendix (Figure 8) seems more informative and sufficient for determining the benefit of NCB. \n2. This paper is missing several related citations: Unsupervised Concept Discovery Mitigates Spurious Correlations (https://arxiv.org/abs/2402.13368) and Neural Language of Thought Models (https://arxiv.org/abs/2402.01203). NLoTM is particularly relevant and can be an additional baseline since it also extends SysBinder to learn discrete representations, except it is trained in an end-to-end way.\n3. The discussion in section 3.3 is interesting, but it would be informative to tie each point with corresponding experimental evidence.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1** (regarding RQ2 (CLEVR Sudoku)): \n\nA determining factor for the performance on CLEVR-Sudoku is the classification of the digits. We agree that this has, to some degree, already been investigated in the context of Q1, where we tested the suitability of NCB’s concept representations for few-shot GT attribute classification. However, in Q2 we wanted to extend these findings into a much more challenging downstream task (than just GT attribute prediction) and specifically focus on the integration into purely symbolic systems that require correct, task-specific symbolic mappings. Importantly, as NCB is trained unsupervised, there is no guarantee that the learned concept symbols will be directly translatable to the symbols required for a downstream task. It is, therefore, potentially necessary to learn a translation from NCB's concept symbols to the task symbols, e.g., individual object attributes to specific object compositions. Overall, this is a very valuable problem setting concerning real-world applications, e.g., in planning (e.g., navigating through a world of objects). In Q2, we investigate NCB's potential for integration in such settings and consider the evaluations on CLEVR-Sudoku to be a helpful illustrative example. Additionally, the evaluations in Q2 also serve as baselines for investigating the ability to revise NCB's concepts for such a complex downstream task in the context of Q3.\n\nNonetheless, we do acknowledge that Figure 8 also provides valuable insights that fall a bit short in the main paper (thanks for the hint!). We have updated the main text to put more focus on these results within the discussion of Q2. \n\n**W2** (additional citations and baseline): \n\nThank you for pointing out the related works, we were not aware of them. We consider Arefin et al. orthogonal to our work as they focus on learning object-level concepts in comparison to our object-factor level concepts and have included it in our related works section. \nIndeed, Wu et al. is relevant to our problem setting. Unfortunately, it was published so close to the NeurIPS deadline that we seem to have missed it; thank you for pointing it out! We have provided the results of this model in our general response and included it as an additional baseline in our paper. We agree this helps highlight our work. \n\n**W3** (Evidence for discussion 3.3): \n\nThank you for raising this point! We agree that the paper would definitely benefit from more illustrations of the results that can be obtained by inspection. We therefore added example visualizations (c.f. additional pdf). Further, in the context of Q3, we do investigate revising models. Specifically, we apply $\\texttt{merge}$ and $\\texttt{delete}$ revision; the corresponding models are denoted as \"NCB revised (GPT-4)\" and \"NCB revised (human)\". We also investigate the application of $\\texttt{add}$ in the \"left-right\" experiment in the appendix (F4). \n\n\n**Q1** (Training SysBinder (hard) and (step)): \n\nGood question! The SysBinder (hard) model did not train well, i.e., the reconstruction was terrible. The SysBinder (step) model, on the other hand, had stable training and provided reconstructions of the quality of the vanilla SysBinder. \nThese results support recent findings [2] that training from the start with hard discretization results in badly performing models. I.e., it is very difficult for the model to learn good representations at all (e.g., for reconstruction) with such a strict bottleneck. \nOn the other hand, learning step-wise can more easily lead to issues with local optima than for models without any bottleneck. \n\n\n**Q2** (RQ4 evaluations): \n\nWe are sorry for the confusion here. Within Q4, we investigate the popular setting of concept-bottleneck-like approaches [1,2] in which a model predicts (discrete) high-level concepts from an image and a second model makes the task prediction based on these concept activations. This leads to models with more transparent and high-level explainability properties. In principle, one can also utilize continuous concept encodings rather than fully discrete concept representations. However, this makes their understandability more ambiguous. In Q4, we investigate the case in which the prediction model is given discrete concept representations. We compare to supervised discrete concept representations (c.f. F7) and particularly investigate the ability to inspect and revise such models that utilize NCB's concepts. Thus, the used concept encodings for the evaluation are actually the discrete concept-slot encodings of NCB. Overall, interpreting and revising models that utilize continuous embeddings is not as straightforward, we have therefore omitted such comparisons here. We have further clarified this now in the text of Q4.\n\n**Q3** (importance of hdbscan): \n\nThanks for the valid questions! We refer here to the general response above with novel ablation evaluations in the additional pdf.\n\n---\n[1] Koh et al. \"Concept bottleneck models.\" ICML, 2020.\n\n[2] Stammer et al. \"Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations.\" CVPR, 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gWhD5j0YWx", "reviewer": "Reviewer_VMaX", "summary": "The authors introduced a pioneering framework that combines an object-centric learning module with a retrieval-based module to address visual reasoning tasks and a new visual reasoning task, CLEVR Sudoku. The proposed method demonstrated significant potential in effectively acquiring inspectable and revisable concepts via human or machine feedback in various scenarios.", "questions": "Please check out the Weakness section first. I listed the following questions and suggestions that would be helpful for authors' future works:\n- Q1: The recent method [1] in object-centric learning literature is linked to causal representation learning and the identifiability of slot representations. How can this be integrated into your framework? \n- Q2: Object-factor learning can be interpreted as learning atoms in logic, and the NN explanations in Table 2 can be seen as the simple form of propositional logic in a neuro-symbolic framework. How can an object-centric learning framework be extended to represent logical rules, such as in the form of first-order logic?\n\nReference\n- [1] Mansouri, Amin, et al. \"Object-centric architectures enable efficient causal representation learning.\" arXiv preprint arXiv:2310.19054 (2023).", "limitations": "Please check out the Weakness and Question sections.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- S1: The proposed method offers significant novelty in that it has the potential to serve as a building block for concept learning, which can be leveraged as a core module in other frameworks. The author's well-structured experiments provided compelling evidence in support of these claims.", "weaknesses": "- W1: The proposed approach can be interpreted as directly integrating SysBinder and HDBSCAN. Because the initial concept detection fundamentally depends on the complete functionality of SysBinder, this framework may not circumvent specific inherent challenges of object-centric learning, including inductive bias resulting from choosing the proper object-factor encoder and identifiability issues.\n- W2: Using HDBSCAN is intuitive in the proposed method, but it would be beneficial to include an additional experiment that compares different clustering methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1** (dependence on continuous encoder): \n\nIndeed, the quality of the initial continuous concept encodings is important for the resulting discrete concept representation. We had remarked on this in the context of our ablations. We have now added an ablation study to highlight this empirically (c.f. Tab.1 (middle) in additional pdf). Specifically, via earlier checkpoints of the sysbinder encoder we observe that the weaker the encoder is, the less expressive are NCB's final discrete concept representations. However, as better models are being developed (e.g., that are shown to also process natural images), one can easily integrate these into the NCB framework and replace the SysBinder encoder that we used for our evaluations. \n\n**W2** (ablation clustering): \n\nThanks for the hint! We have added an ablation evaluation on the CLEVR dataset in the context of Q1 where we compare the expressiveness of concept representations when utilizing k-means rather than HDBSCAN (see corresponding table in additional pdf and general response above). We observe that for generalization, the HDBSCAN approach performs significantly better. \n\n**Q1** (causal representation learning): \n\nThanks for the pointer! This work is indeed interesting, as it connects the more “informal” object-centric detections with the guarantees (and also assumptions) of causal representation learning. While this work and NCB could be considered somewhat parallel, some possibilities exist to combine both. One could use the weakly supervised perturbations to finetune NCB's initial soft binder encoder if one wants to move toward weak supervision. On the other hand, one could also utilize the unsupervised trained NCB with its interventional inspection to generate sparse perturbation of images. This could potentially be possible without specific human supervision but might be limited to the concepts NCB discovers unsupervised. Overall, there are many possibilities for future research in this direction. We have added the reference accordingly into the main text.\n\n**Q2** (OC-learning and logic): \n\nThat's a very important question for the future development of object-centric and neuro-symbolic learning in general. In fact, one motivation for our proposed framework is to be able to extract unsupervised discrete representations from object-centric representations that can be integrated into symbolic AI approaches such as different logic systems. Specifically, NCB's concepts can already be utilized, to some extent, for first-order logic as properties about different objects are encoded into symbols, i.e., unary relations. Thus, first-order logic formulas containing quantors can be expressed out of the box. However, the NCB framework is currently not aimed at encoding arbitrary n-ary relations. We consider utilizing NCB in approaches like [1] could mitigate this issue to learn more complex logic programs based on the unsupervised learned concepts of NCB. \n\n---\n[1] Wüst et al. \"Pix2Code: Learning to Compose Neural Visual Concepts as Programs.\" UAI, 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OnehlmRqIN", "reviewer": "Reviewer_U7bj", "summary": "This paper introduces neural concept binder, a neural symbolic framework that utilizes both soft and hard binding. Building on top of the sysbinder model, it can additionally do exemplar-based hard binding and revise concepts. Evaluations made on CLEVR and the proposed CLEVR-Sudoku dataset proved the method's validity.", "questions": "- How generalizable can this method be? Can it serve as a part of the closed-loop reasoning in CLEVR (I mean, the original CLEVR questions, as tackled in NS-CL and NS-VQA line of work)?\n\n- Can relational concepts be similarly represented via soft/hard binding?", "limitations": "Yes, the authors have adequately addressed the limitations. I do not see any potential negative societal impact.", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "- The paper is well-written and easy to read. Connections to previous works are clarified nicely.\n\n- It's good to see the incorporations of both hard and soft bindings to existing neural-symbolic frameworks.\n\n- The model achieved good performance on the proposed CLEVR-Sudoku task and can do satisfactory concept revision and inspection, which is a neat proof of concept that hard binding works.", "weaknesses": "There are several weaknesses I can foresee that may lead to the rejection of this paper.\n\n- Limited contribution: After so many years of developing neural-symbolic methods in visual reasoning, from the earliest modular approaches to unsupervised concept learners, code-based reasoning models, and recent visual programming-like frameworks, the goal of neural-symbolic modeling has dramatically changed. In this work, the neural concept binder still focuses on one of the earliest task categories designed for visual reasoning (CLEVR attribute classifications or unsupervised concept learners). It's also built on top of sysbinder, in other words, it's merely an incremental improvement by adding a retrieval-based library.\n\n- I don't see any generalizability of this method beyond extremely toy tasks (attribute classification). The proposed CLEVR-Sudoku is strange and does not correspond to any of the real-world visual reasoning tasks. Relational tasks are also not tackled in this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.1** (NeSy contribution): \n\nWe agree that the field of neuro-symbolic AI is rapidly evolving with different focuses, e.g., visual reasoning in real-world images. However, works that are focussing on higher-level neuro-symbolic problems still heavily rely on mapping raw input images to symbolic representations, whether that is by using pre-trained concept extractors [1, 2, 3] or foundation models [4, 5, 6] or forms of weaker supervision [7]. \n\nHowever, how to obtain meaningful symbols from images in a fully unsupervised way remains a core challenge for the community. With our work, we address exactly this open problem, and to our knowledge, only [8] has additionally attempted to tackle this task. We hereby specifically focus on the importance of being able to inspect and effectively revise learned symbols, particularly due to the unsupervised nature of the learning setup. These aspects are most often overlooked in the majority of research that is in the pursuit of performance-driven models. Overall, we believe that our work offers a unique contribution to the field of concept learning, as also all other reviewers agree upon. If the reviewer is aware of specific works that are unknown to us yet relevant for this topic, we would be grateful if they could share them so we can better assess the reviewer's remarks.\n\n**W1.2** (utilizing SysBinder):\n\nWe strongly disagree that our work represents only an incremental improvement over SysBinder.\nWhile in this work, we have focused on utilizing the SysBinder encoder as NCB's soft binder, our method is not limited to it. Instead, this encoder can be replaced by any current or future models that process images into object-factor encodings (e.g., also models that can process natural images). \nWith our framework, we propose a general method for discovering discrete symbols from continuous representations that offers inspection and revision possibilities for human stakeholders. This is not trivial and a very important aspect for unsupervised learning that, to our knowledge, has not been addressed by other works. \n\n**W2.1** (attribute classification as \"toy task\"): \n\nWe disagree and consider extracting symbols from (unlabeled) images as a cornerstone for a variety of neuro-symbolic methods. Many neuro-symbolic approaches rely on extracting symbolic representations from the input, whereas most recent works specifically rely on pre-trained models [1, 2, 3, 9]. NCB, on the other hand, allows to learn such symbols **without** supervision and a main goal of our work is to introduce NCB as a core module for other frameworks (as particularly reviewer VMaX has acknowledged), e.g., for more complex visual reasoning methods. With the evaluations in the context of Q2-Q4, we have highlighted this ability. \n\n**W2.2** (CLEVR Sudoku): \n\nWith CLEVR Sudoku, we are proposing a very relevant challenge that is combining complex visual inputs with explicit reasoning on a symbolic level. \nSpecifically, the underlying reasoning skills required to solve CLEVR Sudoku are very much relevant for real-world scenarios.\nIn fact, many works agree that reasoning with abstract representations (i.e., concepts) are essential for robust generalization [10, 11, 12]. Overall, there is a great deal of interest, and thus, a variety of benchmarks that investigate exactly this form of required intelligence [13, 14, 15, 16, 17]. These specifically exclude language priors and common knowledge priors of real-world images to asses a model's learning and reasoning abilities.\n  \n**W2.3** (Relational tasks): \n\nThe goal of our NCB framework is to extract meaningful symbolic representations from the input, which can be beneficial for a variety of tasks. Hereby, NCB focuses on learning discrete object-level concept representations from images, i.e., focusing on unary relational concepts. The aim overall is not to propose a framework to solve relational tasks. However, NCB can be used as an easily integrable module for models that perform relational reasoning, e.g., which usually require pre-trained, (semi-)supervised concept extractors. \nOverall, we agree that it is important for future work to integrate NCB into more complex relational tasks, as we had noted in our conclusion.\n\n**Q1** (closed-loop reasoning): \n\nYes, exactly, NCB can be easily integrated into closed-loop reasoning. This is exactly the intended application of our method. \nE.g., NS-VQA had relied on the extraction of the scene representation via a supervised trained scene parser with ground truth object attributes. Instead of this scene parser, one can employ NCB, which discovers attributes without supervision and use the object’s attention masks (from the slot attention component) to determine their position. \nIn NS-CL, the concepts are learned via the question-answering task. This is an alternative approach to concept discovery that, however, requires question-answer pairs for the image domain that target those concepts. \n\n**Q2** (relational concepts): \n\nIn principle, we do think it is possible to represent relational concepts via soft/hard binding, though we do not investigate this here. As mentioned above, our approach is intended to be integrated into other methods, such as visual programming or program synthesis, where the learned relations would be based on NCB’s concepts. Hereby, some current work focuses learning relations from images neurally, and others focus on symbolic approaches. Thus, whether to perform relational concept learning via neural, soft binding principles or symbolic, hard binding principles, or a combination of these is still up for investigation. We further refer to our response above concerning relational tasks.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TJSL21C0tb", "reviewer": "Reviewer_6FRw", "summary": "The paper proposes a novel approach to unsupervised concept learning based on both continuous and discrete encodings. Neural Concept Binder (NCB) allows humans inspecting and revising the learnt concepts. In the experiments, NCB’s discrete concept encodings result as expressive as the continuous encodings. Also, NCB can be integrated with symbolic and sub symbolic module. Finally, to support the experimental evaluation the paper introduces a novel dataset CLEVER-Sudoku, very suitable for neuro-symbolic benchmarking.", "questions": "A few questions to try to understand better the notation employed to define the revision operations.\n- What do the authors mean with $v_l \\rightarrow v_m$? \n- What does the add operation work and how one can provide an encoding for a concept and be sure the network employs it as intended?", "limitations": "The method limitations are well addressed by the authors.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "-\t**Novelty**: the proposed approach, although based on existing works SySBinder and Slot attention, is surely novel in the field of concept learning and potentially very relevant as it may strongly facilitate the extraction and discovery of unsupervised concepts. Particularly the possibility to revise concepts is completely novel to the best of my knowledge and very useful to improve human-computer interaction.\n-\t**Novel resource** presented: CLEVER-Sudoku will be surely an important resource for the Neuro-symbolic literature.", "weaknesses": "## Major issues:\n  * Method presentation:\n    - The way in which block-slot-encodings are obtained, is badly presented. Although it is based on previous literature, since it is a key architectural component, it should have been presented more in details. I suggest the author to employ a background section to report the way in which slot attention and sys binder work, in order to make the paper self-contained. \n    - Figure 2 which illustrates the core of the method is quite confusing: it is not clear how the discrete concepts are actually represented (the concept slot encodings reported are positive continuos representations). Also, the reminders to the figures in the text do not help as they generically refer to the entire figure and not to a specific block. A color coding of the different parts of the model could help understanding. \n    - How does the $\\texttt{enc}_l^j$ work is not clear. What does it receive in input? Where it is extracted from?\n    - All the revision operations are definitely not clear. The formal operation to be executed is often confusing.\n  * Experimental evaluation:\n    - Models: NCB has been compared only against SysBinder. While it is a very novel and innovative method, there is a complete lack of benchmarking against standard unsupervised concept-based approaches such as SENN[1], BotCL[2], ACE[3]. Comparing against supervised approaches such as CBM[4] or CEM[5] could have been also useful. \n    - Datasets: NCB is only tested on variants of CLEVER. While it is surely an interesting benchmark, real-world benchmarks are missing. Experiments on CUB or CELEBA, for instance, would have been very appreciated to better understand the scalability of the approach.\n\n## Minor issues\n  * Related work:\n    - The unsupervised concept learning literature does not review several important concept-based paper working both post-hoc and explainable by design. Some examples are SENN[1], ACE[3], VAEL[6], as well as notorious prototype-based approaches such as Prototype layer [7] and ProtopNets[8]. \n    -  Unlike you state, continuous and discrete representations have been combined in recent literature for supervised concept learning. Some examples are CEM[5] and ProbCBM[9].\n  * Unclear sentences:\n    - “Briefly, given an image x, NCB derives a symbolic representation, c, which expresses the concepts of the objects in the image, i.e., object-factor level concepts. Herefore, NCB infers a block-slot encoding, z, of the image and performs a retrieval-based discretization step to finally infer concept-slot encodings, c”. The consequentiality of the inference process is misleading from this sentence. \n  * Method Inspection. What the authors refer as implicit, comparative, interventional and similarity-based inspections are normally referred to as example-based explanations (implicit and similarity-based) and counterfactual explanations (comparative and interventional). Sticking to well-known terms in literature is a good choice to avoid further confusion in the reader. \n\nOverall, I think it's an interesting paper proposing a novel approach to unsupervised concept learning. However, I think it will benefit from a further revision to deeply improve method presentation and expand the experimental campaing including other standard unsupervised concept-learning approaches and datasets.\n\n[1] Alvarez Melis, David, and Tommi Jaakkola. \"Towards robust interpretability with self-explaining neural networks.\" Advances in neural information processing systems 31 (2018).\n\n[2] Wang, B., Li, L., Nakashima, Y., and Nagahara, H. “Learning bottleneck concepts in image classification”. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023).\n\n[3] Ghorbani, Amirata, et al. \"Towards automatic concept-based explanations.\" Advances in neural information processing systems 32 (2019).\n\n[4] Koh, Pang Wei, et al. \"Concept bottleneck models.\" International conference on machine learning. PMLR, 2020.\n\n[5] Espinosa Zarlenga, Mateo, et al. \"Concept embedding models: Beyond the accuracy-explainability trade-off.\" Advances in Neural Information Processing Systems 35 (2022): 21400-21413.\n\n[6] Misino, Eleonora, Giuseppe Marra, and Emanuele Sansone. \"Vael: Bridging variational autoencoders and probabilistic logic programming.\" Advances in Neural Information Processing Systems 35 (2022): 4667-4679.\n\n[7] Li, Oscar, et al. \"Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 32. No. 1. 2018.\n\n[8] Chen, Chaofan, et al. \"This looks like that: deep learning for interpretable image recognition.\" Advances in neural information processing systems 32 (2019).\n\n[9] Kim, Eunji, et al. \"Probabilistic Concept Bottleneck Models.\" International Conference on Machine Learning. PMLR, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1** (More background):\n\nWe agree that adding more information on Sysbinder and Slot Attention can help the reader and make the paper overall more self-contained. We have added an additional section with details for the camera-ready version and provide it in a comment below.\n\n\n**W2** (Clarity of Figure 2):\n\nWe agree and have modified the Figure: (i) we have updated the digits (that had previously represented the concept symbols) in the \"Hard Binder\", \"Retrieval Corpus\" and \"Concept-slot encodings\" components to the notation used throughout the experiments. Specifically, we now denote concepts in Figure 2 with a capital letter for the block and a natural number for the category ID, e.g., A3 for the third concept in the first block (A). (ii) Furthermore, we have added Roman numerals (I-VI) to each component in the figure and reference these specifically within the main text. (iii) We have updated the text accordingly for more clarity. Overall, we agree this greatly helps in understanding the individual components. Apologies for having missed that in the initial version.\n\n**W3** ($\\texttt{enc}_l^j$):\n\nLet us go through this question step by step. We base this on the more detailed training descriptions in A.2 (appendix).\nFirst, NCB infers the block-slot encodings of a set of input images. These are denoted as $\\bar{Z}$ in Alg. 1 in the appendix. NCB then performs clustering of these encodings per block, i.e., it clusters the blocks j of each encoding (denoted $\\bar{Z}^j$) and assigns each of them one cluster label, $v \\in \\{1, \\cdots, N_C\\}$. Per cluster, $v$, NCB next identifies exemplar and prototype encodings (i.e., representative encodings identified via the clustering method and cluster averages).\nEach of these exemplar and prototype encodings correspond to one $\\texttt{enc}^j$ in the tuples, $(\\texttt{enc}^j, v)$, that are stored in the retrieval corpus $\\mathcal{R}^j$ of one block $j$. We now use the index $l$ to identify specific encodings out of $\\mathcal{R}^j$, leading to $\\mathcal{R}^j := \\{(\\texttt{enc}_l^j, v_l) : l \\in \\{1, ..., |R^j| \\} \\}$. In conclusion, $\\texttt{enc}_l^j$ represents one block encoding that is stored in $\\mathcal{R}^j$ and has been assigned to a specific cluster $v_l$.\nWe agree that highlighing these additional steps benefit the understanding of the main text and have updated it accordingly. We apologize for the brevity in the initial submission. \n\n\n**W4** (Revision operators):\n\nIndeed, we agree that the revision operators are somewhat confusing. It seems the attempt to formalize these steps has overcomplicated things. We have now removed the formal operations from section 3.3 and described these in words. E.g. we have posted the updated $\\texttt{merge}$ description in a comment below.\n\n**W5** (Additional baselines):\n\nWe fully agree on the value of additional baseline models. We have, therefore, added the recent NLOTM model as a novel baseline for our evaluations. We refer here to our general response and hope the reviewer can agree on the relevance of NLOTM as a baseline model.\n\nHowever, we politely disagree that the methods suggested by the reviewer represent valuable baselines. Specifically, [1] cannot handle object-level concept learning, nor are the concept assignments discrete (an image is represented as a continuous mixture of concepts). [2] on the other hand, focuses on image regions as concepts. Similar issues hold for [3], i.e., they consider concepts to represent segments of an image and do not provide object-level concepts. Moreover, [3] focuses on post-hoc learning of concepts, which makes potential model revisions tricky and the inspection of the model ambiguous. Overall, we fully agree that the references to unsupervised concept learning are related to our work and we have noted them accordingly in our paper. However, none of them tackle the same problem setting as NCB aims to, i.e., learning discrete, object-level concept representations without supervision. We therefore consider the benefit of running these particular baselines as minor.\n\nConcerning supervised models ([4,5]): in fact, we had compared against supervised concept-based models in the context of Q2 and Q4 (denoted as SA (supervised), c.f. also F.7) whereby we observe that indeed NCBs unsupervised concepts are competitive in comparison to the GT supervised concepts of a slot attention-based CBM despite any supervision in training NCB.\n\n\n**W6** (Additional datasets):\n\nWe kindly refer here to the general response above.\n\n \n**W7** (Related work):\n\nAgreed, we have added the missing references. We note that we had already referenced ProbCBM, though not in the context of discrete and continuous. We have now added this to this respective section.\n\n**W8** (Unclear sentence):\n\nWe apologize for the confusion here. We have updated the sentence (see comment below).  \n\n**W9** (Inspection terminology):\n\nWe agree that connecting the terms example-based and counterfactual explanations can help to understand the outcome of the different inspection forms of NCB. We have now denoted comparative and interventional inspection as two ways to obtain forms of counterfactual explanations and implicit and similarity-based inspection as two ways to obtain forms of example-based explanations. Thanks for the hint! \n\n**Q1** ($v_l$→$v_m$):\n\nWe have removed this notation. It had previously described replacing $v_l$ with $v_m$. We refer here to the response above concerning revision operators.\n\n**Q2** (add operator): \n\nWe have updated the add revision as in a comment below. \nOverall, this operation can be used when the hard binder has missed concepts of the original encoding space. However, if NCB's soft binder has not learned to encode a specific concept at all (i.e., does not represent it in its block encodings) or the concept that a user wishes to add is not present in the dataset at all, one has to revert to the fourth type of revision, i.e., an additional finetuning of the soft binder is necessary.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their time and valuable feedback. We are especially happy to receive so much positive feedback concerning the importance of the tackled problem ("**investigates an important problem**" - edmj), the contribution of our work overall ("**pioneering framework**" - VMaX, "**novel in the field of concept learning**" - 6FRw) and the novel CLEVR Sudoku dataset ("**important resource for the Neuro-symbolic literature**" - 6FRw). We are also happy to hear our writing is of high quality ("**well-written and easy to read**" - U7bj).

With the Neural Concept Binder, we introduce a framework to **learn discrete, interpretable concepts from images** in an **unsupervised** way. We specifically focus on **object-factor-level** concepts and the ability to **inspect and revise** these concepts. This is achieved by combining an object-centric learning module with a retrieval-based module, where the retriever component is beneficial both to obtain discrete concepts and effectively integrate revisory feedback from a human or second model. In our evaluations, we highlight the expressiveness of NCB's concepts and their potential to be integrated into various model settings (both symbolic and neural downstream modules). Moreover, we introduce CLEVR-Sudoku, a challenging, neuro-symbolic benchmark dataset.

### Additional baselines:
Thanks for the suggestions! We have indeed run novel baseline evaluations in the context of Q1 based on the recent SOTA model NLOTM [5]. To our knowledge, this is the only approach that also allows us to learn discrete, object-centric concept representations from unlabeled images. Two seeds are still pending (due to long training durations). We provide preliminary results for one seed in Tab. 2 (additional pdf) and will provide the missing seeds for the final version. We observe a strong decrease in the ability to infer GT object attributes from NLOTM's discrete representations over those of NCB. This suggests that despite NLOTM's explicit training approach for discrete representations, NCB's approach has its advantage regarding representation expressiveness. However, future investigations are necessary to make conclusive remarks. Overall, these results highlight the difficulty of the task and the ability of NCB to tackle it. 

Additionally, our work focuses on the ability to inspect and revise learned concepts. NLOTM, on the other hand, emphasizes the ability to generate novel images. It would be interesting to investigate how both approaches can be combined, e.g., to aid NCB's inspection mechanisms via NLOTM's generation abilities.


### Clustering ablations:
We agree and have added an ablation to investigate the effect of using k-means rather than the more powerful HDBSCAN (c.f. Tab.1 (right) in additional pdf). We observe that, particularly in the small-data regime, the concept representations obtained via k-means are less expressive (though still better than the discrete SysBinder baselines). Overall, the HDBSCAN approach performs better in terms of generalization. However, in principle, the NCB framework is implementation-agnostic to the choice of clustering method, e.g., one can also integrate more powerful recent neural approaches for clustering [1].

### Real-world datasets:
While we fully agree that natural-image datasets are important, we note that the scalability and ability to handle these via NCB depends on the object-centric encoder's (NCB's soft binder) ability to handle natural images. Importantly, this is a general challenge for object-centric learning research and a big topic in the current research community [2,3]. While this line of research is extremely important, it is also somewhat parallel to our work. As object-centric encoders improve, concept discovery via NCB based on these encoders will also be able to handle more natural images. For our evaluations, we mainly focused on instantiating NCB's soft binder with the SysBinder encoder (currently the SOTA model for obtaining object-factor representations without supervision). To the best of our knowledge, SysBinder has only been evaluated on CLEVR-based datasets. Our work focuses on how to extract discrete concept representations from such an encoder (irrespective of the kind of images it can process) and, importantly, how to inspect and revise these representations. Therefore, we have not investigated the scalability of SysBinder to more natural images and consider it out of the scope of this work. 

Concerning the scalability of the hard binder component, we expect that learning the hard binder should not lead to particular scaling issues (depending on the number of block encodings to be clustered and the scope of the grid search). In addition, NCB's retrieval can be sped up, e.g., via FIASS [4].

### Additional figures for inspection types:
We have added qualitative images in the appendix (e.g., Fig.1 in add. pdf) to further exemplify the inspection types of section 3.3.


---
[1] Vardakas et al. "Neural clustering based on implicit maximum likelihood." Neural Computing and Applications, 2023.

[2] Singh et al. "Guided Latent Slot Diffusion for Object-Centric Learning." arXiv, 2024.

[3] Elsayed et al. "Savi++: Towards end-to-end object-centric learning from real-world videos." NeurIPS, 2022.

[4] Johnson et al. "Billion-scale similarity search with GPUs." IEEE Transactions on Big Data, 2019.

[5] Wu et al. "Neural Language of Thought Models." ICLR, 2024.

Meta-review: This work proposes Neural Concept Binder (NCB), a novel framework for unsupervised concept learning. Specifically, the goal is to learn expressive yet inspectable and revisable concepts from unlabeled data. Further, the work introduces new dataset CLEVR-Sudoku dataset to show the utility of the proposed approach.

Summary Of Reasons To Publish:

1) Proposed approach has the potential to serve as a building block for concept learning.

2) Solid experiment results.

3) Introduces challenging dataset representing challenging visual puzzle requiring both perception and reasoning capabilities.


Summary Of Suggested Revisions:

I spent quality time going over reviewer’s concerns and the authors rebuttal. Reviewers pointed out concerns on missing experiment evidence, ablations, and presentation, clarity issues. Authors addressed these major concerns to some extent in the rebuttal.
Decision: Accept (poster)