From Tokens to Lattices: Emergent Lattice Structures in Language Models
Authors: Bo Xiong
Steffen Staab
Venue: ICLR 2025 Poster (2025)
Keywords: Masked Language Models
Formal Concept Analysis
Interpretability
Primary area: interpretability and explainable AI
TL;DR: We investigate the conceptualization of language models from the pespective of formal concept analysis and discuss the inductive bias of  masked language models in lattice structure learning.

Abstract: Pretrained masked language models (MLMs) have demonstrated an impressive capability to comprehend and encode conceptual knowledge, revealing a lattice structure among concepts. This raises a critical question: how does this conceptualization emerge from MLM pretraining? In this paper, we explore this problem from the perspective of Formal Concept Analysis (FCA), a mathematical framework that derives concept lattices from the observations of object-attribute relationships. We show that the MLM's objective implicitly learns a formal context that describes objects, attributes, and their dependencies, which enables the reconstruction of a concept lattice through FCA. We propose a novel framework for concept lattice construction from pretrained MLMs and investigate the origin of the inductive biases of MLMs in lattice structure learning. Our framework differs from previous work because it does not rely on human-defined concepts and allows for discovering "latent" concepts that extend beyond human definitions. We create three datasets for evaluation, and the empirical results verify our hypothesis.

Published as a conference paper at ICLR 2025
FROM TOKENS TO LATTICES: EMERGENT LATTICE STRUCTURES IN LANGUAGE MODELS
Bo Xiong
Steffen Staab
Stanford University, United States University of Stuttgart, Germany x Longoosttanford.ed
University of Stuttgart, Germany
University of Southampton, United Kingdom ste staabeki. uni-stuttgart.de
ABSTRACT
Pretrained masked language models (MLMs) have demonstrated an impressive capability to comprehend and encode conceptual knowledge, revealing a lattice structure among concepts. This raises a critical question: how does this conceptual- ization emerge from MLM pretraining? In this paper, we explore this problem from the perspective of Formal Concept Analysis (FCA), mathematical framework that derives concept lattices from the observations of object-attribute relationships. We show that the MLM's objective implicitly learns a formal context that describes objects, attributes, and their dependencies, which enables the reconstruction of a concept lattice through FCA. We propose a novel framework for concept lattice construction from pretrained MLMs and investigate the origin of the inductive biases of MLMs in lattice structure learning. Our framework differs from previous work because it does rely on human-defined concepts and allows for discov- ering "latent" concepts that extend beyond human definitions. We create three datasets for evaluation, and the empirical results verify our hypothesis.
INTRODUCTION
Masked language models (MLMs) (Devlin et al., 2019) are pretrained to predict a masked token given its bidirectional context. This self-supervised pretraining enables MLMs to encode vast amounts of human and linguistic knowledge (AlKhamissi et al., 2022; Petroni et al., 2019), making them tasks (Van Aken et al., 2019). A key feature of MLMs is their ability capture conceptual knowledge (Wu et al., 2023; Peng ct al., 2022; Lin & Ng. 2022), including understanding concepts and their hierarchical relationships, which are essential aspects of human cognition. Despite substantial evidence (Wu et al., 2023; Peng et al., 2022; Aspillaga et al., 2021; Dalvi et al., 2022) supporting MLMs' ability to conceptualize, the emergence of this capability from their pretraining objectives remains unclear.
valuable and implicit knowledge bases for a variety of knowledge-intensee
To interpret this phenomenon, one pitfall of work in this field is the tendency to focus exclusively on human-defined concepts and investigate how MLMs identify and learn these predefined concepts. This is typically achieved by either classifying (Wu et al., 2023; Peng et al., 2022) or clustering (Aspillaga et al., 2021; Dalvi et al., 2022) terms and then mapping them to established human- defined ontologies, such as WordNet (Miller, 1995). However, this approach often overlooks "latent" concepts that extend beyond human definitions. As a result, while these approaches may illuminate the what- the discovery of human-defined concepts -thee do not adequately explain the how the underlying mechanisms by which conceptualization emerges from the MLMs' pretraining.
In this work, we adopt mathematical formalization of concepts inspired by philosophical frameworks (Ganter al., 2005). In this definition, each concept defined as the abstraction of a collection of objects that share some common attributes. For example, bird is defined as set of objects that can fly, have feather, lay eggs, and so on, while eagle is defined as a subset of these objects that share superset of these attributes. By characterizing concepts as sets of objects and attributes, the partial order relations between concepts are naturally induced from the inclusion relations of the corresponding sets of objects and attributes, which is independent of human-define' structures.
To investigate the question- how does the conceptualization emerge from MLM pretraining? One might argue that ML are trained explicit definitions of concepts (e.g., through handbooks),

---

Published as a conference paper at ICLR 2025
The eagle is a subclass of raptor The owl is a subclass of raptor
The eagle can can hunt... The owl can fly... can hunt...
BERT
BERT
Raptor (human-defined)
eagle owl
eagle owl
flyAhuntA...
(a) Definitional hypothesis
(b) Distributional hypothesis
Figure I: A comparison of two hypotheses of conceptualization language models. (a) Defini- tional hypothesis assumes that concepts are learned directly from the definitions (i.e., concepts explicitly defined in the texts); (b) Distributional hypothesis assumes that concepts are learned from the observations of their attributes (i.e., similar concepts have similar attributes).
which we call as the definitional hypothesis. While this hypothesis may seem possible, as argued by (Cimiano al., 2005), such grounding or conceptualization is typically not frequently found in non non-wikipedia and non-handbook texts or conversation. Moreover, as shown in Fig. I(a), the definitional hypothesis relies on human-defined concept names the text "definition" and cannot handle "latent" concepts whose names are not explicitly defined by human. For example, there is no popularly used term specifying birds that can swim and hunt, but they are indeed a formal concept in the context of FCA. Another interpretation is based on the distributional hypothesis, which assumes that terms are similar to the extent that they share similar linguistic contexts (Cimiano et al., 2005). Therefore, the distributional hypothesis of formal concepts to the extent that they share similar contexts describing their attributes. Fig. shows a comparison of the two hypotheses of concept learning for eagle and owl. Intuitively, the human-defined concept raptor is a rarely used term in texts, while the attributes fly and hunt are more frequently used in texts.
be interpreted concepts are similar
In this paper, we follow the distributional hypothesis and argue that the conceptualization of MLMs emerges from the learned dependencies between the latent variables representing the objects and attributes. We made the following contributions: 1) We show that MLMs are implicitly doing Formal Concept Analysis (FCA), a mathematical theory of lattices for deriving a concept lattice from the observations of object and attributes. This is interpreted as: MLMs implicitly model objects or attributes as masked tokens, and model the formal context by the conditional probability between the objects and attributes under certain patterns. As result, concept lattice can approximately recovered from the conditional probability distributions of tokens given some patterns through FCA; 2) we propose novel and efficient formal context construction method from MLMs using probing and FCA; and 3) we provide theoretical analyses and empirical results our new datasets that support our findings. Notably, the reconstructed concept lattice contains "latent" concepts that are not defined by human and the evaluation does not require any human-defined concepts and ontologies. Our code and datasets will be made available upon acceptance.
PRELIMINARIES
Definition 1 (masked language model). Let w := [w), wr] denote an input sequence of tokens, each taking a value from a vocabulary V, and wit == [W1,-- ... Wt-1, [MASK], Wt+1, wr] denote the masked sequence after replacing the t-th token w, with special [MASK] token. A masked language model predicts we using its bidirectional context w.y, Let D denote the data generating distribution. MLM learns a probabilistic model po that minimizes the pseudo log-likelihood loss
PLL = Ew.DDer(l..... log Po (wa]un w.)...))
MLMs can be interpreted non-linear version of Gaussian graphical models (Wang & Cho, 2019). The idea is to view tokens as random variables that form a fully-connected undirected graph, i.e., by assuming that each variable is dependent all the other variables, and learn the dependencies across tokens. The dependencies cross tokens can be identified by conditional mutual information (CMI) (Anandkumar et al., 201 1), defined as the expected pointwise mutual information (PMI) conditioned
2

---

Published as a conference paper at ICLR 2025
on the rest of the tokens
CM (wy; wy (wr))))))) = Bu,,,, (og,,,,ooorr (wx/kny (5, w2)) log Bwy/w,P (ws/ury (i. (),w))) where w(i, wy) denotes the sentence substituting wy with new token wy.
(2)
Formal Concept Analysis (Ganter et al., 2003) is principled framework for deriving a concept lattice from collection of objects and their attributes. FCA follows the mathematical formalization of concepts where each concept is defined as a collection of objects that share some common attributes (Stock, 2010). FCA aims to reconstruct concept lattice from such observations, which are the objects and their corresponding attributes, represented by formal context, defined as:
Definition 2 (formal context), A formal context is a triple (G, M, I) where G describes a finite set of objects, M describes a finite set of attributes, and I C G x M denotes binary relation (called incidence) between objects and attributes with each element (g. m) E indicating whether an object E G possesses particular attribute m E M.
A formal context can be represented by matrix with rows and columns being the objects and attributes, respectively, and each element describing whether the object possesses the corresponding attribute. A formal concept can be defined attributes M1 S M, formally defined as:
set of objects G1 G sharing common set of
Definition 3 (formal concept). Given G S G and M1 S M, let G¹, == {m E Ml(g, m) E E G y denote all attributes shared by the objects in G1, and dually M2 == {gg E (g, m) E I Vm E Mj } denote all objects sharing the attributes in B. The pair (G1, MM) is formal concept iff G = M1 and Mi = G1. G1 and M1 are called the extent and intent of (G), respectively.
In nutshell, the pair (G1, Mi) is a formal concept iff the set of all attributes shared by the objects in G2 is identical with Mj and dually, the set of objects that share the attributes in Mj is identical with G1. The inclusion between objects/attributes induces partial order relation of formal concepts.
Definition 4 (partial order relation). A partial order relation Sc between concepts is defined by
(G1, M1) Sc (G2, M2) + G1 S. G2 and M2 C M1.
In this sense, top concept is T = -G, and bottom concept is 1 = (M', That is, for any concept (G1, M1), we have (M', (M) Sc (G1, Mi) Sc (G, FCA can be extended to deal with three-dimensional data where the abject-atribbut relations depend on certain patterns.
Definition 5 (triadic formal context (Jäschke al., 2006)). A triadic formal context is quadruple (G, M, B, Y), where G is a finite set of objects, M is a finte set of attributes, B is a finite set of conditions (patterss) and S G x M B is ternary relation between objects, attributes, and conditions.
An element (g, m) Y is interpreted as: The object g G possesses the attribute E M under
the condition 6 B. Triadic concept is defined as:
Definition 6. A triadic concept of the triadic formal context (G, B, B.Y) ) is triple (A1, A2, A2, where A C G. A2 S M, and Ag S B, and the following property holds:
(A1, As) = (1g G vm E A2, vb E Ag, (g. m, b) E Y}.
(4)
(m E M Ag E A1, 4b E As, (g. m. b) EY),
(5)
{be B Vg E AI, Vm E A2, m, b) b)))
(6)
where Ai is the set of objects shared by all attributes in A2 under all conditions in Ag. An is the set of attributes shared by all objects in A1 under all conditions in Ag, Ag is the set of conditions under which all objects in A share all attributes in Ag
Concept Lattice The set of all concepts and the partial order relation given in a formal context lead to concept lattice that describes inclusion relationship between their sets of objects and attributes.
3 THE LATTICE STRUCTURE OF MASKED LANGUAGE MODELS
We argue that MLMs implicitly model concepts by learning a formal context through conditional distributions between objects and attributes in a target domain. First, introduce how concept lattices are constructed from MLMs using FCA in Section 1. Next, we discuss the inductive biases of MLM pretraining they pertain to the learning of formal contexts in Section 3.2.
3

---

Published as a conference paper at ICLR 2025
(a) Observable world (training data)
(d) Abstract world (concept lattice)
eagle
Top
[MASK]
fly
dove
Perception
flyable
huntable
Swimmable
wolf
animal that
swim
walf
frog
fish
fly-huntable
swim-huntable
frog
can
hunt
fish
[MASK]
FCA
eagle
bottom
snake
BERT
classification
Fly
swim
hunt
frish
Swimmable
eagle
0.93
0.046
0.84
1.0
frog snake
dove
0.79
0.26
swim
Flyable
wolf
0.5
0.27
0.34
1
snake
dove
0.3
0.81
0.82
wolf
frog
0.0
0.3
0.86
0.25
Embedding
fish
0.0
0.5 fly
1.0
045
(b) Normalized conditional probability
(c) Conceptual embedding
Figure 2: An illustration of how MLMs learn conceptuall/nnollgicaal knowledge from natural language. (a) The observation is set of sentences that describe relationships between objects and attributes (nouns and verbs in this example). Each sentence is abstracted as filling of pattern with an object-atrribute pair; (b) the normalized conditional probability between objects and attributes is learned by the MLM (e.g., BERT) and it can be viewed as formal context in probabilistic space; (c) each row of the conditional probability can be viewed as the conceptual embedding of the object, and each dimension/columm of the embeddings corresponds to particular attribute. These dimensionally interpretable embeddings can be used for concept classification; (d) the abstract world (concept lattice) be recovered from the learned formal context in (c) through FCA, and the concept lattice can be viewed as a hierarchy for concept classification.
3.1 LATTICE CONSTRUCTION FROM MASKED LANGUAGE MODELS
The relationship between objects and attributes can be explored by probing using a cloze prompt, for example, "[MASK], is animal that can [MASK].. where [MASK], and [MASK],,. in serve as placeholders for objects and attributes, respectively. This type of template, where both object and attribute tokens are masked, is referred as a concept pattern, denoted by b. The instantiation of a pattern with a specific object-attribute pair (g, m) is termed the filling of b, denoted as bs.m, For partial fillings of the pattern, we use j0- denote filling with the object g alone, and b.m for the attribute m alone. Figure 2a illustrates concept pattern along with its potential fillings.
Probabilistic triadic formal context We estimate the conditional dependencies between objects and attributes using pa [g]b b b>) or po(m) (.):)). By cataloguing all possible fillings of objects and attributes within a domain, we construct a conditional probability matrix that serves as an approximation the formal context for lattice construction. The accuracy of this formal context largely depends on the pattern's 's efficacy in capturing the intended concept domain and its ability to mitigate confounding influences from other factors. To address these challenges, utilize multiple patterns and propose the creation of probabilistic triadic formal context, represented as atree-dimensional tensor.
Definition (construction of probabilistic triadic formal context). Given a pretrained MLM denoted as po. and considering set of objects G and set of attributes M that the model has been trained on (i.e., G.M C V where V the vocabulary), and a set of patterns B, the triadic formal context Y can be constructed by setting either Y Yg, po (g) (g (n) or Yg.m.b Po po(mi]b: (m))b3)) where G. m E M, and E B.
Fig. 2b provides a normalized representation of formal context incidence matrix using a single pat- tern. However, constructing the complete three- way tensor, which requires calculating the conditional probability among any combination of objects, attributes, and patterns, involves a time complexity of
BB)) To address efficiency, we propose method that requires only [ [ or [B[ M evaluations of the cloze prompt in the MLM, with representing the batch size and [-] the ceiling
4

---

Published as a conference paper at ICLR 2025
function. Although the number of possible natural language patterns is infinity in theory, considering the top most informative patterns is sufficient for construction (see Lemma 2). Therefore, it is reasonable to assume that G| > |M| > |B], rendering the time complexity nearly linear with respect to the number of objects or attributes. The efficient construction is formalized as follows:
Definition 8 (efficient construction). Let po(-)wat) E BIM denote the predicted conditional prob- ability vector given Whe and given a set of objects G, a set of attributes M, and a pattern we such construct W|M|XT (resp. W such that its ith row W, = bh (resp. W -Biis') ) denote the
partial filling of pattern with attribute my (resp. object gs). Let indg (resp. ind ) be the token indexes of objects G (resp. attributes M), the formal context of pattern can be reconstructed by Y.,. pa (M)))) (resp. Y.,.b po(-WW (GXT)))
The core idea of Def. 8 is to first extract the conditional probability over all possible tokens (i.e.. pe(-WWiM|xT) or pe(-IW/GIXT) in batch manner, and then index the relevant conditional probabilities using the attribute indices ind or object indices indg. This construction assumes that either objects or attributes are included in the vocabulary, while the counterpart may consist of multi-token text. Alternatively, instead of relying on a pre-defined set of objects and attributes within domain, one might generate the set of object-attribute pairs in the domain from the distribution. However, direct sampling from the joint distribution via Markov chain Monte Carlo (MCMC) sampling is challenging. By considering that MLMs can function as probabilistic generative models, we employ Gibbs sampling to sample a sequence of object-attribute pairs that can be used to approximate the joint distribution, which does not presuppose known set of objects and attributes.
Definition 9 (formal context generation via Gibbs sampling). Given a MLM po and concept pattern b, the process begins by sampling an initial object 9o directly from po and an attribute mo from p. At each subsequent step t (where t 2, ...). new object gt sampled from pe(-Ib- m) ) and a new attribute me from po Pa(-169t (.:)) The conditional probabilities for all objects and attributes are then computed based on the sequence (go. ma)." m.).
Lattice construction The reconstructed probabilistic triadic formal context, denoted as Y. serves as a smoothed version of the discrete triadic formal context. As shown in Fig. 2c, the smoothed formal context can be viewed as a conceptual embedding that encodes the object-attribute relation in a probabilistic space. We consider its 2-dimensional projection, which approximates the probabilistic incidence matrix i. This approximation is achieved by aggregating over multiple patterns, either through average pooling Ig.m Ess Youmi or max pooling in. max B Y Ygmm.b
[B]
As MLM outputs are softmax probabilities that may result in limited positive responses, and FCA requires a binary input, we apply a min-max normalization approach where, given threshold o, the binarization is performed as follows:
log(fg.m) - min log(
I g.m maxlog(( min log(i)
>
After binarization, a concept lattice is constructed by: (1) generating all formal concepts by identifying
combinations of objects and attributes that fulfill the closure property defined in Def. 3, and (2) structuring these formal concepts into a lattice based on the partial order relations defined in Def. 4. Fig. 2d illustrates the resulting lattice with 0.5 and the corresponding objects of each concept.
3.2 INDUCTIVE BIASES OF FORMAL CONTEXT LEARNING
We introduce the formal context learning under certain abstractions. Drawing inspiration from natural language concept analysis (Kamphuis & Sarbo, 1998), implying that conceptualization arises from observing the attributes present in objects, we formalize the world model as a ground-truth formal context defined as Fo := (Go, Mo, Io). As data captures the perception of the world, we formalize each data point as a "sentence" æ describing that an object has a particular attribute, which can be viewed a filling of concept pattern with object-attribute pair "sampled" from Fo.
In FCA, concepts inherently unnamed. Names, if required, must be determined in separate post- processing phase.
5

---

Published as a conference paper at ICLR 2025
Algorithm 1 A Simple Formal Context Learning Algorithm
Input: A dataset D, set of objects G and attributes M
Output: An estimated formal context incidence matrix I = [0, 1] [|
initialize = OlG|x|M!
for w E D do for w, E w do for wj E w do if wi E G, Wi E M then
I Mw,,
Ma,) +1
end if end for end for end for normalization: I normalize(I)
return /
Abstraction (data generation). Given a formal context Fo := (Go, Mo. Io) and a set of concept patterns B, each data point is generated by filling concepi puttern sampled iid. from B with an object-atribute pair (g, m) sampled i.i.d. from the formal context Fo. Thus, each data point is represented = ba.m, where b B. g E Go. m My, and (g.m) Io-
Under such an abstraction, formal context learning aims to reconstruct the world model (original formal context) from the data points generated by Abstraction 1. Given that real-world data generation is not as perfect as Abstraction I and might be influenced by other factors or noises. We hence acknowledge that an exact reconstruction of the original formal context improbable. Therefore, we focus on e-approximate formal context learning.
Definition 10 (e-approximate formal context learning). Given a set of N data points D = {w") N. generated according to Abstraction I, e-approximate formal context learning seeks to derive S.
probabilistic formal context incidence matrix i from D, such that the distance
Our hypothesis is that identifying the formal context incidence matrix from the sentences generated from Abstraction I is possible. Lemma establishes the feasibility of identifying the formal context from the generated dataset at the population level.
Lemma (feasibilityy. Lei D = } be a dataset consisting of data points generated by Abstraction I, then there exists an identification algorithm F mapping a dutaset D to formal context such that F(D) converges to the ground-truie formal context Ig as N > oo almost surely, i.e. Algorithyy kassan example algorithm. A full proof of Lemma is provided in the appendix, but the core idea is to demonstrate that / converges to In almost surely when the number of data points N + 00 by using the law of large numbers and concentration inequalities.
MLMs as formal context learner The MLM objective can be interpreted non-linear version of Gaussian graphical models for latent structure learning (Mohan et al., 2012), which resembles Lasso regression that aims to predict x; from Di; with each nonzero coefficient corresponding to dependency weight. Hence, we can formalize MLM objective as formal context learning by:
Definition 11 (masked formal context learning). Let D {w'} be a dataset consisting of data points generated by Abstraction 1, the goal of masked formal context learning is to minimize
PLLMFCL -EE (log pe (glb-"") + logpe(m) (e(m)
Eww,D, (log (o(sb-)))) (8)
The first two loss terms directly encourage the capture of the dependency between objects and attributes under certain the pattern b, i.e., Pe (") and po (m]b..)) The third loss term implies that the quality of the chosen pattern significantly influences the effectiveness of formal context learning. The intuition that pattern might convey much irrelevant or confounding information that confuses the learning of abtect-attribute dependency. We characterize this confounding information as latent variable information Z. Lemma 2 shows the significant role of the patterns and the influence of Z.
Lemma 2 (role of pattern). Let (MMIp (g: b)] denote the conditional MI without latent variables. and CMI, (g: m Z; b) denote the conditional MI with latent variables Z, we have [CMI,, (g: mlb) (MT,(g: m Z: b) < 2H (Z)b), where H (Z)b) is the conditional entropy.

---

Published as a conference paper at ICLR 2025
Table I: The candidate concept patterns used to construct formal contexts for different datasets.
Dataset
A set of concept patterns
MASK] m is the official language of [MASK]] 9
Region-language
The official language of [MASK], is [MASK]] m
[MASK] m serves the official language in [MASK]a
The [MASK], is an animal that can [MASK].r In. The [MASK], gg is type of animal that has the ability to [MASK]M
nnimal-behavine
An animal known the [MASK]] has the ability to [MASK]
The [MSK]], y is disease that has symptom of [MASK] m
Discase-symplom
People who infected the discase [MASK], g typically has symplom of [MASK] [MASK] m is kind of symptom of the [MASK]g
Lemma 2 implics that the difference between CMI,p (g: m|b)| and CM m ml b) is bounded by the conditional entropy (Z]6). This means that if the concept pattern b provides sufficient information that uniquely describes the object-atrribute relationship without introducing confounding factors (i.e., H (Z]b) ). the conditional MI directly captures the dependency between objects and attributes. This re- re-affirms the importance the quality of patterns and their critical role in accurately capturing the object-atrribte relations. full proof of Lemma is detailed in the appendix.
Takeaway
The inductive bias of MLMs in lattice structure learning originates from the learned condi- tional dependency between tokens. The precision of the lattice construction depends mainly on the quality of the chosen pattern in exclusively capturing the object-attribute relationship.
4 EVALUATION
In this section, evaluate whether the formal contexts constructed from MLMs align with estab- lished gold standards and assess their capability to reconstruct concept lattices. We have developed several gold-standard formal context datasets across various domains for empirical analysis (Sec. 4.1). Specifically, our investigation addresses two key research questions: 1) Can the conditional probabili- ties in MLMs effectively recover formal contexts (Sec. 4.2)? 2) Can the reconstructed formal contexts be utilized to construct concept lattices (Sec. 4.3)? We substantiate our findings with additional ablation studies and case analyses in Sec. 4.4.
4.1 FORMAL CONTEXT DATASETS
We construct three new datasets of formal contexts in different domains, serving as the gold standards for evaluation. Two of them are derived from commonsense knowledge, and the third one is from the biomedical domain. 1) Region-language details the official languages used in different administrative regions around the world. This dataset is extracted from Wiki44k (Ho et al., 2018), utilizing the "official language" relation, which is a densely connected part of Wikidata; 2) Animal-behavin captures the behaviors (c.g., live on land) of animals (c.g., tiger). This dataset is constructed through human curation. We compiled set the most popular animal names English, considering only those with single token. We identified 25 behaviors based animal attributes that aid in distinguishing them, such as habitat preferences, dietary habits, and methods of locomotion; 3) Disease-soe syrease-sympoom describes the symptoms associated with various diseases. We extracted diseases represented by single token and their symptoms from a dataset available on Kaggle². These datasets vary in terms of density and the nature of objects and attributes. particular, the Animal-behand -behavior dataset is notably denser compared to the other two that are relatively sparse. Additionally, these datasets accommodate cases where objects or attributes can consist of multi-token texts. We design three concept patterns for each dataset by varying the positions of objects and attributes, as given in Table The detailed statisties for these datasets are summarized in Table in the appendix.
7

---

Published as a conference paper at ICLR 2025
do comminal pain
los appelii
BERT-uncased
dhen pain
chili
coug
traddctt
high leee
kching
malti
ski cal
sweting
Animals that do not fly
#harmBETTancaseed
20
Animals that fly
A253
Arthrtts
AIDS
Asthma
Cald
Giabwies
Nalaria
Parlystis
-10
Incumorio
T.berrulesis
Al discases
popular diseases
Vertige
(a)
(b)
(c)
Figure 3: (a) Comparision of BERT, BioBERT, and PharmBERT on formal context learning disease-symptom, (b) The conditional distribution between diseases and symptoms learned by BioBERT; (c) The T-SNE visualization of the conceptual embeddings constructed from the learned formal context incidence matrix of animals, where the colors denote whether the animals fly or not.
Table 2: The performance of formal context reconstruction on Region lagguagn- and Animal-behavior by the generated conditional probability of three variants of bidirectional MLMs (BERT), where denotes trivial results whose numbers are < 0.1. The shaded rows are the baseline results. The best and second best results are bold and underline numbers, respectively.
Region language
Animul Behavior Hive5
MRR (1)
Hit@ 1 (1)
Hite (1)
Hic@ 10 (f)
MRR (1)
Hir@1
Hice (t)
BEKT-base
BertLattic (avg.)
RemFiibs
0.823
0.340
0.249
0.403
0.950
0:478
0.208
0,078
0.291
0.493
BeriLattice (max.)
BFRT-distil
BeriLattice lavg.)
Benfimb
0.838
0.094
0.002
0.746
0.945
0.109
0.975
0.214
0.207
0.067
0.301
0.537
BeriLattice (max.)
0.831
0.736
0.945
0.970
0.206
0.068
0,310
D.519
BaRT-large
Bertl lavg.)
Bentimb.
0.213
0.833
0.731
0.114
0.249
0.970
0.975
0.383
0.194 0.192
0.067
0.276
0.453
BertLattice (max.)
0.819
0.706
0.960
0.975
0.070
0.267
Implementation and computational resources We instantiate our approach (i.e. Def. 8) with BERT and dub it BertLattice. We also design naive baseline model, dubbed as BertEmb., which predicts object-attribute relations by measuring the distance of the last-layer hidden state embedding vectors of objects and attributes. All experiments were conducted machines equipped with Nvidia A100 GPU. Our method does not require training of MLMs. The extraction of the formal contexts is very fast (< 60 seconds) for all three datasets.
4.2 CAN CONDITIONAL PROBABILITY IN MLMS RECOVER FORMAL CONTEXT?
We evaluate the capability of MIMs on reconstructing formal contexts from the conditional probabil- ities by comparing the recovered formal contexts with gold standards. We formalize the problem as ranking problem and use ranking-based metrics commonly used in ranking problems, specifically mean reciprocal rank (MRR) and hit@k k: MRR N EN rank, hit@k = N E hit; We consider three variants of BERT models: BERT-distill, BERT-base, and BERT-large. For each model, we use the uncased versions and compare the Average pooling and Max pooling variants, denoted as BertLattice (avg.) and BertLattice (max.), respectively.
As Table 2 shows, all variants of BertLattice perform significantly better than BertEmb. on both Region-language and knimal-behaviae datasets. BERT-base performs best in terms of MRR and Hit@ 1 while BERT-distill and BERT-large outperform BERT-base on Animal-behavior and Region- language in terms of Hit@5 and Hit@ 10, respectively. The average results on Region-language surpass those of Animal- Animal-behavior. We hypothesize that this is due to the objects and attributes in Region-languag being more frequently observed in Wikipedia, where BERT is pretrained. This hypothesis is supported by the observation in Fig. 3(a) showing BERT's relatively lower performance on the domain-specific Disease-sym symptom dataset, while BioBERT and PharmBERT, pretrained on biomedical texts, significantly outperform BERT-base. Additionally, results on popular diseases are better than those other diseases, suggesting that observation frequency significantly influences MLMs' ability to learn correct object-attribute correspondences. Fig. 3(b) shows part of the Disease- symptom formal context extracted from BioBERT. BioBERT accurately identifies symptoms for
8

---

Published as a conference paper at ICLR 2025
0.8
F1 Score
0.4
F1 (min-max)
F1 (sigmaid) mAP max)
MAp (sigmaid)
larch
Lestineten
0.0
Garrer
0.3
0.6 threshold
0.9
Geran speating
(a)
(b)
Figure 4: (a) Comparison of different normalization approaches for lattice construction under different thresholds; (b) The reconstructed Pegiion-languge concept lattice. For visualization, only part of the concepts and the corresponding objects are labeled. We also highlight German and French speaking lattice paths; (c) The conceptual embeddings of regions/countries in the dimension of German-speaking and French-speaking.
Table 3:
The performance (F1 score and mAP) on concept classification, where denotes trivial results whose numbers are & 0.1. The shaded rows denote the baseline results. The best and second best results are bold and underline numbers, respectively.
Region-language FI (f)
Animal-behavior Fl (t)
Disease-sympton F1) mAP (t)
mÄP (1)
mAP (f)
BertEmb.
0.214
0.027
0.269
0.393
BERT-base
BertLattice (avg.)
0.630
0.528
0.643
0.388
BertLattice (max.)
0.703
0.667
0.620
0.413
BERT-distill
BertEmb.
0.207
0.026
0.292
0.382
BertLattice (avg.)
0.249
0.291
0.651
0.396
BertLattiie (max.)
0.703
0.667
0.617
0.417
BertEmb.
0.231
0.028
0.395
0.395
BERT-large
BertLattice (avg.)
0.673
0.635
0.641
0.389
BertLattice (max.)
0.709
0.701
0.636
0.407
BertEmb.
0.632
0.385
0.345
0.256
BioBERT
Bert (avg.)
0.598
0.391
0.601
0.377
BertLattice (max.)
0.631
0.382
0.611
0.389
common diseases, such as Asthma with symptoms Cough and chest pain, and Cold with symptom chills. Full visualizations of the constructed formal contexts are available in the appendix.
4.3 CAN THE RECONSTRUCTED FORMAL CONTEXTS IDENTIFY CONCEPTS CORRECTLY?
We now evaluate whether the formal contexts reconstructed from the conditional probabilities can be used to identify concepts. We frame this as a multilabel concept classification task, aiming to identify the correct concepts given object. Specifically, possess all the attributes of given concept, which is done by measuring the conditional probability under the given patterns. We use F1 score and mean Average Precision (mAP) metrics, which balance precision and recall and are commonly used multilabel classification (Xiong et al., 2022).
classify objects by determining whether they
Table 3 presents the results of concept classification. Generally, find that all variants of BertLattice outperform BertEmb. on all three datasets. The BERT-base, BERT-distill, and BERT-large models perform well on the Region-languan language and Animal-behavior datasets but not as well the Disease- symptom dataset. We hypothesize that this is because these BERT variants are not pretrained on texts involving disease-symi syeess- information. This hypothesis is supported by the observation that BioBERT variants achieve more reasonable results in the Disease-symyn symptom data set. An interesting exception is that BioBERT also performs well on the
dataset.
4.4 ABLATION STUDIES AND VISUALIZATION
Influence of normalization & threshold We investigate the impact of different normalization methods on the performance of lattice construction from the conditional probability. We compare two normalization techniques: min-max normalization, which normalizes the matrix level, and

---

Published as a conference paper at ICLR 2025
sigmoid normalization, which normalizes at the (lement-wise level. As shown in Fig. 4(a), sigmoid normalization achieves optimal results when the threshold a z 0.5, whereas min-max normalization performs best when = 0.6. Notably, different from the sigmoid normalization, further increasing the threshold of the min-max normalization does not significantly degrade performance, highlighting the robustness and advantages of min-max normalization.
Visualization Fig. 4(b) visualizes the concept lattice constructed from the Region- Region-language language datasets. The model discovers some latent concepts that humans have not predefined. For example, it identifies the concept of "German and French speaking regions," which includes Switzerland, Luxembourg, Belgium, etc. These are parent classes of the European Union because it includes regions where both German and French, among other languages, are spoken. Importantly, this Region-language concept lattice is not predefined by human ontology. Fig. 4(c) and Fig. 3()) further illustrate the conceptual embeddings derived from the conditional distributions of MLMs. The first figure shows the T-SNE embeddings and demonstrates that the flyable animals and non- non-flyable animals are clearly distinguished. Fig. 3(c) shows that the "German and French speaking regions" concept corresponds to set of countries that speak both German and French in the embedding space.
5 RELATED WORK
Conceptual knowledge in language models Pretrained MLMs have demonstrated capabilities in capturing conceptual knowledge (Wu et al., 2023; Lin & Ng, 2022). A variety of methods have been proposed to probe MLMs for such conceptual knowledge. Most of them use binary probing classifiers (Aspillaga et al., 2021: Michael et al., 2020) or hierarchical clustering (Sajjad al., 2022; Hawasly et al., 2024) to identify concepts and validate these against established human-defined ontologies like WordNet (Miller, 1995). Despite these methods offering empirical insights into the conceptual structures encoded by MLMs, they predominantly explore what MLMs learn without addressing how the masked pretraining objectives facilitate the encoding these conceptual structures. Moreover, is argued that MLMs can develop novel concepts that do not align strictly with existing human- -defned ontologies (Dalvi et al., 2022). This implies that traditional methods may not capture the full extent of conceptual understanding that MLMs are capable of. From theoretical perspective, recent studies suggest that language models represent concepts through distinct directions in the latent spaces of their hidden activations (Chanin et al., 2024). This observation aligns with the linear representation hypothesis, implying that attributes or features are represented as directional vectors within a model's hidden activations (Elhage et al., 2022; Park al., 2024). Our work is highly different from previous research, as we consider a formal definition of concepts without any reliance on human-defined concepts and ontologies, and we leverage mathematical framework (FCA) to analyze concepts.
CONCLUSION
This paper investigates how conceptualization emerges the pretraining of MLMs through the lens of FCA, a mathematical framework for deriving concept lattices from observations of object-attribute relationships. We demonstrate that MIMs implicitly learn the conditional dependencies between objects and attributes under certain patterns, which can be interpreted as probabilistic formal context that facilitates the reconstruction of the underlying concept lattice. We then propose a novel framework for lattice construction and discuss the origin of the inductive bias for lattice structure learning. Notably, our analysis of the conceptualization capabilities of MLMs does not rely on predefined human concepts and ontologies, allowing for the discovery of latent concepts inherent in natural language. Our empirical findings on three new datasets support our hypothesis.
Linitations and future work Currently, data. An immediate future work is to extend FCA to multi-relational data by using multi-relational concept analysis (Wajnberg et al., 2021). )). For instance, "Pizza lattice" could be constructed based on both the toppings and sauces of pizzas. Our lattice analysis could also be adapted to autoregressive language models, like GPT series (Ye et al., 2023), which predict the next token given all preceding ones. However, this adaptation requires the masked token for either objects or attributes to be placed the end of the concept pattern, which may not fit well in some domains.
FCA framework of MLMSs is limited to single-relational
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
Our study does not involve any training or hyperparameter tuning, but instead relies on pre-trained masked language models. We use the PyTorch Transformer library for all model implementations. Our code and datasets are included as supplemental materials and will be made available upon acceptance. Proofs of the lemmas presented this paper are also provided in the appendix in the supplemental materials.
ACKNOWLEDGEMENT.
This work was partially funded by the Deutsche Forschungsgemminschatt (DFG) in the project "COFFEE" - STA 55-2
REFERENCES
Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. A review on language models as knowledge bases. CoRR, abs/2204.06031, 2022.
Animashree Anandkumar, Vincent F. Tan, Furong Huang, and Alan S. Willsky, High-dimensional
structure estimation in ising models: Tractable graph families. CoRR, abs// 107. 201
Carlos Aspillaga, Marcelo Mendoza, and Alvaro Soto. Inspecting the concept knowledge graph encoded by modern language models. In ACL/IJCNPP (Findings). volume ACL/UCNNP 2021 of Findings of ACL, pp. 2984-398991 3000. Association for Computational Linguistics, 2021.
David Chanin, Anthony Hunter, and Oana-Maria Camburu. Identifying linear relational concepts in large language models. NAACL, 0tos/2311.0896. 2024.
Philipp Cimiano, Andreas Hotho, and Steffen Staab. Learning concept hierarchies from text corpora using formal concept analysis. J. Artif. Intell. Res., 24:305-339 339, 2005.
Fahim Dalvi, Abdul Rafae Khan, Firoj Alam, Nadir Durrani, Jia Xu, and Hassan Sajjad. Discovering latent concepts learned in BERT. ICLR. OpenReview.net, 2022.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (I), pp. 4171-4188 Association for Computational Linguistics, 2019.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCan- dlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition. CoRR, abs/2209. 10652, 2022.
Bernhard Ganter, Gerd Stumme, and R Wille. Formal concept analysis: Methods, and applications in computer science. TU: Dresden, Germany, 2003.
Bernhard Ganter, Gerd Stumme, and Rudolf Wille. Formal concept analysis: foundations and applications, volume 3626. springer, 2005.
Majd Hawasly, Fahim Dalvi, and Nadir Durrani. Scaling up discovery of latent concepts in deep NLP models. In EACL (1), pp. 793-806. Association for Computational Linguistics, 2024.
Vinh Thinh Ho, Daria Stepanova. Mohamed H. Bad-Elrab, Evgeny Kharlamov, and Gerhard Weikum. Rule learning from knowledge graphs guided by embedding models. In ISWC (1), volume 11136 of Lecture Notes in Computer Science, pp. 72-90. Springer, 2018.
Robert Jäschke, Andreas Hotho, Christoph Schmitz, Bernhard Ganter, and Gerd Stumme. TRIAS an algorithm for mining iceherg tri-lattices. In ICDM, pp. 907-911. IEEE Computer Society, 2006.
htug::/yyyoccaaaaaaaaaaaaaaeeeee hrme Transformer.hmml

---

Published as a conference paper at ICLR 2025
Vera Kamphuis and Janos J. Sarbo. Natural language concept analysis. In CoNLL, pp. 205-214 ACL, 1998.
Ruixi Lin and Hwee Tou Ng. Does BERT know that the IS-A relation is transitive? In ACL (2), pp. 94-99, Association for Computational Linguistics, 2022.
Julian Michael, Jan A. Botha, and Ian Tenney. Asking without telling: Exploring latent ontologies in contextual representations. In EMNLP (1), pp. 6792-6812. Association for Computational Linguistics, 2020.
George A. Miller. Wordnet: lexical database for english. Commun. ACM, 38(11):39911888111 41, 1995.
Karthik Mohan, Michael Yoon Chung, Seungyeop Han, Daniela M. Witten, Su-In Lee, and Maryam Fazel. Structured learning of gaussian graphical models. In NIPS, pp. 629-637, 2012.
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. JCML, 2024.
Hao Peng, Xiaozhi Wang, Shengding Hu, Hailong Jin, Lei Hou, Juanzi Li, Zhiyuan Liu, and Qun Liu. COPEN: probing conceptual knowledge in pre-trained language models. In EMNLP, pp. 5015-5035 Association for Computational Linguisties, 2022.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. Language models as knowledge bases? In EMNLP/IJCNLP (1), pp. 2463-2473, Association for Computational Linguisties, 2019.
Hassan Sajjad, Nadir Durrani, Fahim Dalvi, Firoj Alam, Abdul Rafae Khan, and Jía Xu. Analyzing encoded concepts in transformer language models. In NAACL-HLT, pp. 3082-3101. Association for Computational Linguistics, 2022.
Wolfgang G. Stock. Concepts and semantic relations in information science. J. Assoc. Inf. Sci. Technol., 111111999,,,,9,, 2010.
Betty Van Aken, Benjamin Winter, Alexander Löser, and Felix A Gers. How does bert answer
questions? a layer-wise analysis of transformer representations. In Proceedings of the 28th ACM international conference on information and knowledge management, pp. 1823-1832, 2019.
Mickaël Wajnberg, Petko Valtchev, Mario Lezoche, Alexandre Blondin-Massé, and Hervé Panetto. Fea went (multi-) relational, but does it make any difference? In 9th workshop" What Can FCA Do for Artificial Intelligence? colacated with 30th International Joint Conference on Artificial Intelligence, IJCAI-21, volume 2972, pp. 27-38, 2021.
Alex Wang and Kyunghyun Cho. BERT has mouth, and it must speak: BERT as markov random 2019.
field language model. CoRR,
Weiqi Wu, Chengyue Jiang, Yong Jiang, Pengjun Xie, and Kewei Tu. Do plms know and understand ontological knowledge? In ACL (1), pp. 3080-3101. Association for Computational Linguisties, 2023.
Bo Xiong, Michael Cochez, Mojtaba Nayyeri, and Steffen Staab. Hyperbolic embedding inference for structured multi-label prediction. In NeurIPS, 2022.
Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang Shen, al. comprehensive capability analysis of gpt-3 and gpt-3.5 series models. arXiv preprint arXiv:2303, 10420, 2023.
Tianyi Zhang and Tatsunori Hashimoto. On the inductive bias of masked language modeling: From statistical to syntactic dependencies. In NAACI.HIT, pp. 5131-5146. Association for Computational Linguistics, 2021.
12

---

Published as a conference paper at ICLR 2025
Table Statistics the used datasets, where density denotes the percentages of positive responses.
#objects
#attributes
density
object token
attribute token
Region-language
165
45
2.59
single or multi-token
single-token multi-token single or multi-token
Animal-bchavinn
354
25
40.3
single-token single-token
Disease-sy Disease-symptom
122
33
6.76
APPENDIX
SUPPLEMENTAL RESULTS
Computational resources All experiments were conducted on machines equipped with 4 Nvidia A100 GPU. Our method do not require training of MLMs. The extraction of the formal contexts for our datasets very fast (S 60 seconds).
PROOF OF LEMMAS
Lemma (feasibility). Let D (wo' Jim] be a dataset consisting of data points generated by Abstraction 1, then there exists an identification algorithum :: + I such that F(D) converges to the ground-truth formal context In as N + Do almost surely, i.e., F(D) I Io-
Algorithm 1 A Formal Context Learning Algorithm
Input: A dataset D, set of objects G and attributes M
Output: An estimated formal context incidence matrix / [0, 1] M|x|M]
initialize I= x
for E D do
for wi E w do for wy E w if G, Wy G,w, E M then (W,,) (G,, +
end if end for end for end for normalization: normalize(!) return /
Proof. The algorithm constructed derive formal context from Our target is to demonstrate that 1 converges to Lo almost surely when the number of data points N + 30. Let f denote the sample space, In represent the learned formal context from DN, and Xx [[[N.. Io) denote the random variable indexed by N. We need to establish that X, a.s. O.
Let
define FN
(w Ω NN(w) of for
where repre-
[1 g, E æ and
sents
element of the sample space.
Let Ngm.e
0 otherwise"
13

---

Published as a conference paper at ICLR 2025
Afrikasns
Albarian
Arabic
Bengail
Bulparian
Burnese
Catalan
Chiness
creation
Czech
Danish
Dutch
English
Estonian
Finnish
French
Geargien
German
Greek
Hindl
Icolandic
Itallon
Japanese
Khmner
Korean
Latvian
Malay
Nepail
Norwegian
Polich
Portugueso
Ramaniaa
Russian
Serbisn
Slovak
Slovene
Spanish
Swenish
tamil
Thai
Turkish
Ukrainian
urdu
Vietnamese
Albania
Andorra
Austria
Bangiadiss
Betsyana
Brune
Burkina Fasp
Cambadia
Canada
Cape
Central Mrrican Republl
China
Creati
Czeth
Republic
Dibous
Dominican
Republic
Egypt
Uhland
Gambia
Ghana
Guinea
Honduras
Halb
Iceland
Indonesa
rad
agan
Kazathstan
Knbba
Kyrgyz stan
nobanon
Licchtenstein
Molau
MexiED
New Zea Nepa
Nicer
North Korea
Ne
Philiggine
Portuga
Republic
Republic the Conap
Sen Marino
Soufh Arita
Sweden
i
Switzerland
Timor Leste
Trinidad
Turkey
kkraine
United States of America
Vanuatu
Figure 5: The normalized conditional probability of regions and their official language. The probabil- ity is generated by the cloze prompt "[object] is the official language [attribute]".
14

---

Published as a conference paper at ICLR 2025
consider
(EN))
=
P P P(ENGG.mEN
N EE= Yom.t -1og. >c).
we have
(a)
P(EN) S
g,m,t 10gmm
N
MEG
IA3
P
N
ggm,, - 10g.m
8 11
N
GEG,MEM
1=1
2NNe2
egexp
q²
where inequalities (a) and (b) use the union bound, and (c) applies the Hoeffding's inequality.
By applying the Borel-Cantelli lemma, we have P (lim supN EN) = 0. Hence, P p XN 0) 1. This means that / converges to fo almost surely when N -
Lemma (role of pattern). Ler [CMI (g: mlb) denote the conditional MI without latent variables and CMI, (g: b) denote the conditional MI with latent variables Z, we have |CMI,, (g: m|b) is the conditional entropy.
CMI,(g; m Z: b) z (Z)b) where H (Z
Proof. Proposition 3 from Zhang & Hashimoto (2021) shows that the dependency between two tokens can be captured by conditional MI. Our lemma can be proved in similar way by viewing that the objects and attributes are all tokens in the vocabulary. That is g, m E V.
Using the definition of conditional MI, we start with:
CMIp(g m/ b) CM) CMI Z[b) + CMI, Z/m, b)
Expanding this, we get:
CMI, m ZZ = CMI, mlb) H (Z(b)
+ HZ m. b) (2)g, m.
Now, let's consider the difference:
- m (Z;b)
CMI, m|b) CMI, (g: m Z: b)
HZ b) + H(Zbb) ( (mm, b) + (Zg,,,,
Next, apply the inequality properties of entropy:
CMI,(g; CMI, (g; m Z; < (Z + H g, m)
Since entropy is always negnavvae, we further have:
HZ g. m, b) < (Z
Combining these, we get:
ICMI (g)mlb)
CMI, [2;b) < (Z HHZZg, $ 2H (Z
Therefore, we conclude that:
CMI, - Z (Z(b)
CMI,,
This completes the proof.
15


Reviews:
[{"review_id": "gHrpeKNbky", "reviewer": "Reviewer_RPj7", "summary": "The work focuses on generating concept lattices from masked language models such as BERT, RoBERTA, etc. The idea is to utilize the information contained in these models for generating a partial ordering of the concepts instead of learning the concepts from textual information.", "questions": "--> Are authors directly generating a concept lattice or generating a formal context? \n--> What is the computational complexity of generating the context and then the lattice?\n--> What is the broader applicability on where these lattices can be used for solving which tasks? \n--> Why not prompt Large Language Models which contain much more information?", "limitations": "", "rating": 1, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "--> Concept lattices are directly constructed from masked language models.", "weaknesses": "--> It should be made clear in the paper if the authors are directly constructing the concept lattice or the formal context first and then generating the lattice. This makes a difference since generating concept lattices starting from a formal context is computationally expensive. \n--> The contributions/research questions of the paper should be explicitly mentioned in the introduction.\n--> Why not using Large Language Models for constructing lattice and comparing it to masked language models since the masked language models are not fine-tuned they are rather used by masking the tokens.\n--> The broader applicability of these lattices is a bit unclear, where would these lattices be used? If the objective is to classify the terms into the classes then this can be done without passing through the concept lattice generation phase. i understand that there is a partial ordering between the concepts but there is a concept of hierarchical classification.\n--> Introduction should have a clear example/scenario for motivating the objectives of this work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aEotSfON8G", "reviewer": "Reviewer_yYtU", "summary": "The authors propose a framework that uses Formal Concept Analysis (FCA), to explore the conceptual understanding of an MLM (Masked Language Model) and their hierarchical relationships. \nFCA is a well-known mathematical framework researched in the early 2000's or so. FCA aims to uncover concept lattices from attribute-value data. \nMLM are trained neural networks that aim to predict a masked token in a sentence given its context. They are known to be capable of contextual understanding that can be used as pre-training in various NLP tasks. \nThe big question the authors pose is: how does conceptualization emerge from pre-training an MLM?\nThe authors hypothesize that MLMs learn the conditional dependencies between objects and attributes under certain patterns. Then they propose a framework to probe MLMs through the use of FCA to construct a lattice of latent concepts without relying on human-defined ontologies. Their empirical evaluation, using three datasets they developed, showed that conditional probabilities in MLM can help recover formal contexts and construct concept lattices.", "questions": "1-  How can the lattice of concepts be leveraged in NLP tasks?\n2-  Does using only one relation mean that FCA does not allow capturing the full extent of the conceptual understanding the MLM is capable of?\n3- How are the uncovered concepts labeled?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "+ Bridging the gap between FCA and MLM is a good idea that shows that old and new AI can work in tandem, especially when it comes to the interpretability of black-box models and their usefulness in subsequent tasks. Using FCA could be a nice step in this direction. \n\n+ Paper well-written in general, appropriate citations.\n\n+ The formalization seems appropriate. \n\n+ The ability to uncover latent concepts without human ontologies gives a chance to non predefined concepts to emerge and hence has the potential to increase our knowledge in the domain.", "weaknesses": "- There is a lack of general motivation on why it is important to uncover the lattice of the concepts embedded in an MLM and how it will be used. If the goal is XAI, the authors should highlight it.\n\n-  The scalability of the framework to multi-relation is unclear and using one single relation seems quite limiting. While it is mentioned by the authors as a limitation, a discussion about how limiting the lattice to one relation can miss patterns would be good. The authors could discuss the challenges that might arise when tackling multi-relations. This might include the complexity of the lattice itself. \n\n- The experimental section is limited in the number and choice of the datasets. \n\n- Figures are hard to read.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wzcqqIIFZD", "reviewer": "Reviewer_t7YG", "summary": "In the paper “From Tokens to Lattices: Emergent Lattice Structures in Language Models” submitted to ICLR 2025, the authors answer the question whether Formal Concept Analysis  concept lattices can be extracted in a meaningful way from an MLM, such as BERT. They answer this question positively, which is demonstrated by a dedicated experiment.", "questions": "Every comment in the weakness section, except the list of minor corrections, is essentially a question.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "I find the paper interesting to read. It studies a sound problem about an ability of LLMs to perform Symbolic AI tasks.", "weaknesses": "Definition 5 is vogue, it is more like an intuition rather than definition  \n\nIn L (line) 263, it is said that data generation (i.e., Abstraction 1) is an inherently noisy process. However, I do not see anything noisy in Abstraction 1, everything is nice and clean. What is meant by noisy here?\n\nIn L 253, Animal-behaviour dataset is presented. But where does the data come from? Is it just the authors’ beliefs and nothing more?\n\nIn L 374, it is said “We formalize the problem as a recommendation system problem”. But this is not clear at all without further details, and so the essence of the first experiment is not clear for me.\n\nIn the introduction and the conclusion, it is emphasised that the suggested method does not depend on predefined human concepts and ontologies. However, it is misleading: it clearly depends on humans, who have to provide patterns B.\n\n\nMinor: \nL 53: extra whitespace after “possible”\nL 99: D should be \\mathcal D and p_\\theta should be defined\nL 125: Do not start a sentence with a math symbol, especially if the previous one ends with such a symbol\nL 211: Start the definition with “Given a pattern b”\nL 237: b \\in B should be in the subscript\nL 243: g, m should be in the subscript\nL 268: it should be defined what is the distance in this context\nL 275, 278: bf w should be just w\nL 285: hypothesis\nL 290: standard notation F: D \\to I says that F is a function with domain D and range I. This is clearly not what you want to say here. Also, D should be \\mathcal D.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Fe5ppms1pO", "reviewer": "Reviewer_D1N8", "summary": "This paper explores how masked language models (MLMs) like BERT inherently learn conceptual relationships by organising knowledge into \"concept lattices\" through a framework called Formal Concept Analysis (FCA). The authors demonstrate that through this framework MLMs can reconstruct concept relations directly from language patterns in an more effective and explainable way.", "questions": "- Have you considered the issue of potential ill-formed entity or relation names fitting into the template, which might introduce bias into evaluation?\n\n- A minor point concerns the definition of MLM. In practice, not every token in a sequence is masked, so it would be more accurate to define MLMs with respect to a set of sampled masked tokens.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This work introduces a well-structured FCA-based probing framework that translates MLM logits to formal contexts, allowing for insightful analysis.\n- The constructed concept lattices effectively recover and represent conceptual knowledge that MLMs encode.\n- The framework also ensures efficient construction of concept lattices.", "weaknesses": "- The baseline models only consider naive BERT embeddings, omitting a range of established probing techniques known for their effectiveness, such as prompt-based probing in natural language inference (NLI) formats. Incorporating these methods would provide a stronger basis for claiming that this FCA-based framework is more effective in extracting knowledge from MLMs.\n\n- The evaluation is limited to \"single-token\" named entities, which may restrict dataset coverage. While the `[MASK]` token only predicts one token at a time, several methods exist to approximate multi-token predictions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2Jz5z64Uaa", "reviewer": "Reviewer_nF1q", "summary": "This paper explores how masked language models (MLMs) like BERT implicitly capture conceptual knowledge and organize it into lattice structures. The authors employ Formal Concept Analysis (FCA) to illustrate that MLMs' training objectives inherently form a \"formal context\" that can be mapped into a lattice framework. The paper diverges from traditional approaches by not relying on human-defined concepts, instead proposing that MLMs capture \"latent\" or unspoken concepts beyond human ontologies. The author examines this approach in three datasets: region-language, animal-behavior and disease-symptom, and concludes that MLMs are efficient in modeling the concept hierarchies in probabilistic terms.", "questions": "1. Can this framework be applied to autoregressive language models (e.g., GPT series) instead of masked language models?\n\n2. How are concept patterns selected for this framework? Are they designed empirically by humans, and to what extent could different patterns impact the results?\n\n3. How might this framework perform on larger, more complex datasets with more ambiguous or polysemous relationships?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The innovative application of FCA enables the reconstruction of concept lattices without relying on human-defined concepts and ontologies, allowing for the discovery of latent concepts that traditional methods might overlook.\n\n2. The paper includes a variety of visualizations that effectively illustrate key concepts, enhancing clarity and engagement.\n\n3. The proposed method is computationally efficient, making it practical for implementation without requiring additional training or significant resources.\n\n4. The paper offers a thorough empirical analysis across diverse domains, which supports the validity of its claims that MLMs can generate meaningful concept lattices.", "weaknesses": "1. The study relies on carefully curated datasets that may not capture the complexity or variability of real-world language use. Expanding the dataset to include a broader range of contexts and ambiguities would enhance the framework’s robustness and applicability.\n\n2. While concept lattices offer a structured hierarchy, they are currently limited to modeling simple object-attribute relationships. Incorporating more nuanced or multi-relational knowledge types would broaden the framework’s applicability.\n\n3. The paper’s presentation could be significantly improved. Simplifying the exposition and emphasizing intuitive explanations would make the work more accessible and engaging.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)