Summaries as Centroids for Interpretable and Scalable Text Clustering
Authors: Jairo Diaz-Rodriguez
Venue: ICLR 2026 Poster (2026)
Keywords: Text clustering
unsupervised learning
natural language processing
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We replace k-means centroids with readable summaries for interpretable clustering that scales to streams (mini-batch), is LLM-optional with capped cost, and matches/beats baselines with far fewer calls.

Abstract: We introduce k-NLPmeans and k-LLMmeans, text-clustering variants of k-means that periodically replace numeric centroids with textual summaries. The key idea—summary-as-centroid—retains k-means assignments in embedding space while producing human-readable, auditable cluster prototypes. The method is LLM-optional: k-NLPmeans uses lightweight, deterministic summarizers, enabling offline, low-cost, and stable operation; k-LLMmeans is a drop-in upgrade that uses an LLM for summaries under a fixed per-iteration budget whose cost does not grow with dataset size. We also present a mini-batch extension for real-time clustering of streaming text. Across diverse datasets, embedding models, and summarization strategies, our approach consistently outperforms classical baselines and approaches the accuracy of recent LLM-based clustering without extensive LLM calls. Finally, we provide a case study on sequential text streams and release a StackExchange-derived benchmark for evaluating streaming text clustering.

Published as a conference paper at ICLR 2026
SUMMARIES AS CENTROIDS FOR INTERPRETABLE AND SCALABLE TEXT CLUSTERING
Jairo Diaz-Rodriguez Department of Mathematics and Statistics York University
Toronto, Ontario M3J 1P3 jdiazzoodeyrrku.oo
ABSTRACT
We introduce k-NI.meeans and k-L.M text-clustering variants of k-means that periodically replace numeric centroids with textual summaries. The key idea, cannnaar--asccetrool retains k-means assignments in embedding space while producing human-readable, auditable cluster prototypes. The method is LLM- optional: k-NLPmeans uses lightweight, deterministic summarizers, enabling of- fline, low-cost, and stable operation; k-LLMmeans is a drop-in upgrade that uses LLM for summaries under fixed per-iteration budget whose cost does not grow with dataset size. We also present mini-bated extension for real-time clus- tering of streaming text. Across diverse datasets, embedding models, and summa- rization strategies, our approach consistently outperforms classical baselines and approaches the accuracy of recent L.L.-based clustering without extensive L.LM calls. Finally, we provide a case study on sequential text streams and release a StackExchange-derived benchmark for evaluating streaming text clustering.
INTRODUCTION
Text clustering is a core problem in natural language processing (NLP). with applications in doc-
ument organization, topic exploration, and information retrieval (Schütze al. 2008 Steinbach
2000
A standard pipeline embeds documents into vectors (Devlin 2018 Sanh 2019 Mikolov
2013 Pennington et al. 2014 Brown al. 2020 Din al. 2023) and then groups them with
clustering algorithm (Petukhova et al. 2025). Among these algorithms, k-means (MacQueen, 1967) remains ubiquitous, iteratively updating each centroid as the mean of its assigned points. While effective, this purely numerical averaging can blur contextual nuance present in the original texts (Reimers Gurevych] 2019). Prior work has explored alternative centroid definitions and related objectives (Jain& & Dubes 1988 Bradley et al. 1996; Kaufman & Rousseeuw 22008), yet these ap- proaches remain anchored in vector space, which limits interpretability and can induce semantic drift between centroids and their underlying documents. This motivates centroids that explicitly textual, aligning prototypes with human-intenn interppratre summaries.
Our proposal. We introduce simple modification to k-means: periodically replace numerical centroid updates with summarization steps. Rather than averaging embeddings at every iteration, we compute, at spaced iterations, a textual prototype that summarizes each cluster and then re-embed it with the same encoder to obtain the centroid used for assignments. These updates remain inside the standard k-means loop but capture richer contextual meaning. Alternating numerical and summary- semantically coherent. Figure Illlsttras our proposal and shows how even a single summarization step can redirect k-means toward qualitatively improved solution.
updates yields clusters that more interpretable and often more
Our goal is not to strictly dominate every clustering algorithm, but to provide a simple novel ap- proach that (i) improves clustering quality standard k-means-style methods, (ii) yields human- readable centroids, and (iii) scales to large and streaming datasets with limited LLM usage.
Summarization step. We consider two families of summarizers: (1) Classical NLP such as centroid-based summarization, grsph- methods such TextRank, and LSA-style techniques

---

Published as a conference paper at ICLR 2026
Initialization
l-means Iterations
Summarizzation step
Ie I-means iterations
sccounti Addiionelly heeiican
pet an acsount chilet
*
How con uae Goople Apple
*
How cae larketteee and what She prosasa insaived
with single summarization step. First panel shows the text embeddings with stars marking the initial centroids; second shows the partition reached after k-means iterations (a local minimum); third performs the summarization step, each previous cluster is summarized into a textual prototype and re-embedded; final panel runs one more k-means iteration using these summaries as centroids, yielding qualitatively improved partition. This example is illustrative; in practice, clusters can be multi-topic, and summaries may describe multiple prominent subtopics for single cluster.
Figure I: Illustration of k-
(Radev et al., 2004 Mihalcea & Tarau 2004: Deerwester et al. 1990), which are fast and deter- ministic (yielding k-NI.Pmeans); and (2) LLL--baed summaries that can capture richer context and filter secondary or noisy content (Zhang et al. 2020a Raffel et al. 2020 Jia & Diaz-Rodriguee 2025) (yielding k-LLMmeans). In both variants, summaries are recomputed only scheduled it- erations, so the algorithms share the same summary-as cettttood mechanism and differ solely in the summarizer. While methods from (1) and (2) are standard in summarization, our periodic use of summary-derived centroids within the k-means loop is, to our knowledge, novel and central to the method's interpretability.
Interpretability and scalability. Each summarization step produces concise, human-readable centroid that exposes evolving cluster semantics, simplifying debugging, validation, and labeling. The summarizer is modular: it can be LLM-free (k- NL.means) or LLM-assisted under fixed per- iteration budget that does not grow with dataset size (k-LL.Mmeans). For large-scale and streaming settings, adapt mini-batch k-means (Sculley) 2010) by inserting summarization steps into the mini-batch update rule, enabling efficient online clustering while retaining interpretability.
Relation existing LLM-based clustering. Recent work shows that LLMs can deliver strong unsupervised clustering (Zhang et al., 2023 Feng et al. 2024 De Raedt et al. 2023 Viswanathan al. 2024 Shi & Sakai 2023] Tarekegn et al. 2024, Nakshatri
al. 2023). However, many pipelines face two practical issues: (i) scalability, as they may rely on semi-supervision, iterative labeling, or number of LLM calls that grows with dataset size; and (ii) opaque optimization, since they often combine prompts, greedy merges, and similarity thresholds without an explicit objective, making convergence behavior hard to analyze. Our k- L-LLamssns addresses both: by summarizing per cluster at spaced iterations, we cap LLM usage independently of dataset size, and by keeping assignments and numeric updates in embedding space, we preserve the standard k-means objective between summary steps. In both k-NLPmeans and k- k-LLMMeens, if the summaries are weak, the procedure gracefully degrades to vanilla k-means with a suboptimal initialization, retaining its stan- dard convergnce behavior. Note that our method is LLM-optional: I-NLPmeans replaces the summarizer with classical NLP techniques, incurring no LLM usage.
Contributions. In summary, we (i) propose k-NLPmeans and k-LLMmeans, summary-as- centroid variant of k-means that periodically replaces numeric centroids with textual prototypes re-embedded by the same encoder, an ILM-oppional design that, by construction, preserves the standard k-means objective between summary steps; interpretability follows from human- human-readable readable centroids and transparent intermediate outputs; (ii) extend the approach to mini-batch k-means for efficient, online clustering of streams; (iii) present comprehensive empirical study across datasets, embeddings, and summarizers, showing consistent gains over classical baselines and competitive- ness with recent LLM-based clustering under dataset-size srizzeeiddeeedeent LLM budget; and (iv) provide a case study on sequential text streams and release a StackExchange-ders benchmark for evaluating streaming text clustering.
2

---

Published as a conference paper at ICLR 2026
2 PRELIMINARIES: K-MEANS FOR TEXT CLUSTERING
Given a corpus of n text documents dimensional embedding vector x; E k such that:
... d :..) Each document d; is represented as a d-
Embedding(d:).
The goal of k-means clustering is to partition these n document embeddings into & clusters, mini- mizing the intra-cluster variance. Formally, we define the clustering objective as:
min Σ Cr,C2
Σ Xx, j=l
Myll
where C3 denotes the set of embeddings assigned to cluster j. [cy] {i|x; E C₃} denotes the set of embedding indices assigned cluster j and μ1 the cluster centroid, computed as the mean of the assigned embeddings:
X
C2
EE(C))
Lloyd's algorithm (Looyd 1982), the standard heuristic for k-means, alternates between assigning each document embedding x; to nearest centroid and recomputing each centroid as the mean of its assigned points. Repeating these two steps for I iterations monotonically reduces the within-cluster sum of squared distances, steering the procedure toward a (locally) optimal set of centroids. How- due to its sensitivity to initialization and the non-convex nature of its objective function, k- means does not guarantee convergence to the global optimum and can instead become trapped in local optima (MacQueen) 1967; Looyd 1982). Various strategies, such k-means++ initialization and multiple restarts, have been proposed to mitigate these issues and improve the likelihood of achieving better clustering results (Arthur & Vassilvitskii 2006).
K-MEANS WITH SUMMARIZATION STEPS
We enhance k-means for text clustering by periodically replacing the numerical centroid update with summarization steps that yield textual-summary-based centroids. The procedure (Algorithm) [ Appendix D) identical k-means algorithm except that every / iterations the mean update in equation
2 is replaced by the embedding of cluster textual summary. At all other iterations, the standard update used (see Figure T for an illustration with one summarization step). The summarizer can be instantiated with classical, deterministic methods (K-NLPmeans) or with LLM- based summaries (k-LLMmeans), and not restricted the specific techniques discuss, any textual summarization operator can be dropped in.
3.1 K- K-ELPM
Our first variant uses classical extractive summarization to compute a textual prototype in place of the standard centroid update. Formally, replace equation [ with
μy Embedding ( (NLP(()) (S,))
where S, is the collection (multiset) of sentences obtained by tokenizing all documents assigned to cluster j. Unless otherwise noted, sentence-sentence similarities are computed cosine similarity between sentence embeddings produced by the same encoder used for documents. The operator <(9) NLP returns short summary of sentences; typical instantiations include:
Centroid-hased summarization (Radev et al. 2004):
compute the centroid of the sentence embeddings for Sy; rank sentences by cosine similarity to this centroid; concatenate the top
q sentences (by rank) form (4)) (S,).
Graph-based (TextRank) (Mihaleea & Tarau 2004): build graph whose nodes are tences in S, with edge weights given by parrwise cosine similarity of their embeddings; run PaneRank-style algorithm to score sentences; select and concatenate the top q-
3

---

Published as a conference paper at ICLR 2026
LSA-style SVD in embedding space (Deerwester et al. 1990 stack the sentence embed- dings for Sj. apply singular value decomposition, score sentences by contribution to the leading components, and concatenate the top q.
The resulting summary text <(0) (N)) is then embedded in the same vector space as the documents
to yield the new centroid j4j-
3.2 K-LLMMEANS.
If instead generate the summary with an LLM, have k-LLMmeans. Formally, we replace equation by
M Emmeedinn(((1mM(())))
where pj Prompt (d,(dz, (ent~~1,,,,0 (C,)), ), and my min(m, (C,)). Here, 2; [Cy] denotes
sampled index of the embeddings assigned to cluster C (without repetitions) and m is a parameter that represents the maximum number of sampled indices used compute the cluster centroid μ4j. In simple terms, we update a cluster's centroid by using the embedding of the response generated by an LLM when queried with prompt containing a summarization instruction / and a representative sample of documents from the cluster. Rather than providing all documents within the cluster as input, the LLM processes representative sample as a context prompt. While incorporating the entire cluster is theoretically possible, it poses practical challenges due to prompt length limitations. Therefore, we propose selecting the sample cluster documents using a k-means++ sampling of the cluster embeddings. Our experiments demonstrate that this sampling process facilitates more effective synthesis of the cluster's content, leading to improved summaries and, consequently, more refined centroid updates. The instruction I varies depending the clustering task, but standard summarization prompts are generally sufficient. Figure Iillustrates how k-LLMmeans with a single summarization step enhances the standard k-means algorithm.
3.3
ADVANTAGES OF OUR APPROACHES
Over k-means. Periodic summarization steps act semantic prototype update that can redi- rect the search trajectory and reduce sensitivity to initialization. Instead of relying solely on Eu- clidean means, a textual summary captures contextual cues present in the underlying documents; embedding this summary yields centroids that better reflect cluster semantics. In practice, summary- based centroids produce more interpretable and often more semantically coherent partitions, when k-means++ seeding is suboptimal. Apart from the summary steps, the procedure follows stan- dard k-means: assignments and numeric updates unchanged and the usual objective is preserved between summary iterations. Section 5.1 shows that our methods often outperform vanilla k-means across diverse settings.
Over advanced L LLMeb clustering methods. Our approach offers three key advantages over more complex LLM-based clustering methods: (1) Optimization landscape. Grounded in the stan- dard k-means objective, we inherit algorithmic convergence without relying on the fragile heuristics common in newer LLM--riven methods. poor summary simply leads the procedure back toward regular k- k-means local optimum. whereas competing approaches hinge on stable, format-specits LLM outputs. (2) Scalability Unlike most state-of-the-as methods, whose LLM usage complexity grows with sample size (Feng et al. 2024 De Raedt et al. (023) or require fine-tuning (Zhang et al., 2023). k-NLPmeans requires no LLM calls and k- M-LLMmeans only performs & LLM calls per sum- marization step. with even few summarization steps yielding substantial performance gains. (See Section D. (3) Interpretabiliry Replacing numeric centroids with textual summaries turns each prototype into concise, human-readable synopsis; practitioners
track how cluster semantics evolve over time without post-hoc labeling. This transparency extends naturally to our mini-batch variant, enabling real-time monitoring streaming scenarios (see Figure 2 and Section 5.1
MINI-BATCH K-NLPMEANS AND X-LLMMEANS
Mini-batch k-means (Sculley 2010 is efficient strategy for large-scale text clustering that pro- cesses small, randomly sampled bani-batches instead of the full dataset. This approach substantially reduces memory usage and computational cost, making it well suited for continuously generated text
4

---

Published as a conference paper at ICLR 2026
streams, such as those from social media, news, customer feedback; where data must be clustered incrementally without full dataset access. Mini-batch k-means exhibits convergence properties com- parable to standard k-means while offering superior scalability.
Although numerous streaming clustering methods that do not rely on LLMs have been studied (Sil
et al., 2013 Aggarwal 2018, Ribeiro al. 2017 Aggarwal al. 2003 Ackermann et al. 2012
Ordonez 2003), ). only few incorporate LLMs (Tarekegn et al. 2024 Nakshatri et al.) 2023). More-
over, existing offline LLM-based clustering approaches face scalability issues, highlighting the need for scalable summary-based clustering in an online setting. To address this, we introduce mini-batch k-NLPmeans and k- L-LMMmans, which directly extend mini-batch k-means by inserting summariza- tion steps into the mini- batch update.
Algorithm 2 details how our approaches sequentially receive b batches of documents DI.... Do where each batch contains set of documents (these batches can either be random samples from large corpus or represent sequential data). It processes each batch sequentially with k-NL.means/k- LLMmeans and updates centroids incrementally using weighted rule like mini-batch k-means. Our algorithm preserves the desirable properties of mini-batch k-means, with low memory and none or low LLM usage. Section shows that it also outperforms in simulations.
STATIC EXPERIMENTS
We use four benchmark datasets that span diverse domains and classification granularities: Bank77 (Casanueva et al. 2020), CLINC (Larson et al.] (2019), GoEmo (Demszky et al.] (020) and MAS- SIVE (domain and intent) (FitzGerald et al. (202).. See Appendix|A.I]foor detailed descriptions of the datasets. We evaluate our algorithms on each of the four datasets using the known number of clusters and performing 120 centroid-uddate iterations. We calculate two variants of our algorithms differing in the number of summarization steps. The single variant uses single summarization step (1 60), while the multiple variant performs five summarization steps = 20). To demon- strate the robustness of our approach, we compute embeddings with models: DistilBERT (Sanh, Gurevych] (209)), and text-embedding-3 evaluate using TextRank, Centroid and LSA summariza- tion methods mentioned in Section 3.1 with g = 5. For the LLM component of K-LLMmenns, we use 19733.5-urbooOpenA1
2019) e5-large( et al.] 2022), S-BERT(Reimers
OpennA 2024). For k-NLPmeans
et al.. 2024). et al.J (024). For the instruction task 7, we employ a simple summarization prompt depending on the task. For example, for Bank77 we use the prompt: "The following is a cluster of online banking questions. Write a single question that represents the cluster concisely." We explore the effect of prompt size by summarizing using all cluster documents as input, and a few-shot (FS) variant where each prompt includes only m = 10 randomly selected documents instead of the full cluster. For each setting we run five different seeds.
2023). , GPT HPTT-o(Hurstt et al., 2024),
Claude-3. 1aaude-3.7(Atthropic (2025) and DeepSeek-V3ie
As traditional clustering baselines, we consider k-means, k-medoids (Kaulman & Rousseeuw 2008). spectral clustering (Ng et al.] 2001), agglomerative clustering (Johnson) 1967). and Gaus- sian Mixture Models (GMM) (Dempster et al., 1977). We use ground-truth number of clusters and ten different seeds. We also include BERTopic (Grootendorsi 2022)) as a strong topic-modeling baseline. In addition, we report results obtained by applying our proposed methods to the docu- ment embeddings produced by BERTopic. For advanced LLM- based methods, we compare with results obtained by Feng al. (2024) ClusterLLM (Zhang et al. (2023), IDAS (De Raedt et al. 2023), and LLMEdgeRetine (Feng et al., 2024). All k-means based methods are initialized with k-means++.
We evaluate performance using clustering accuracy (ACC), which measures the best one-to-one alignment between predicted clusters and gold labels, and Normalized Mutual Information (NMD, which quantifies the mutual information between them normalized to [0. 1] (See Appendix C for details on the metrics). We now provide summary of the results.
5.1 RESULTS WITH STATIC DATA
Comparing summarization variants and traditional clustering methods. Table I reports mean accuracy (ACC) and normalized mutual information (NMI) for multiple k-NLPmeans and k-LLL variants (LLM: GPT-4o), using text-embeddingg---maall embeddings on four datasets.
5

---

Published as a conference paper at ICLR 2026
Table 1: Average ACC and NMI for A-NLLs and k-LaL variants using GPT-4o, com-
pared against traditional baselines, BERTopic, and our k-NLPmeans LSA-multiple and K-LLMmeans
FS FS-multiple variants applied BERTopic embeddings, using text-embedding- 3-mallll embeddings
on benchmark datasets.
BANK77
CLINC
GoEmo
Massive (D)
Massive (I)
Dataset/Method
ACC
NMI
ACC
NMI
ACC
NMI
ACC
NMI
ACC
NMI
k-NL.means
TextRank-singe
66.4
78.7
21.5
20.9
72.5
TexiRank-mulipp
66.8
83.9
79.6
92.8
21.6
21.0
60.5
69.3
53.7
72.8
Centroid-
67.2
84.0
78.5
92.2
22.1
21.2
60.9
68.3
53.5
73.0
centrodd-multipe
67.5
84.0
79.0
92.4
21.4
21.1
60.8
69.1
54.3
73.5
LSA single
66.7
83.7
78.8
92.5
21.9
20.5
61.2
68.7
54.1
73.0
LSA-mulinn multiple
67.1
84.0
80.2
92.9
22.3
20.2
63.3
70.0
55.3
73.4
K-LLMmeans
single
67.1
83.6
78.1
92.5
24.0
22.3
61.1
69.6
54.0
73.0
multiple
66.7
83.6
79.1
24.1
22.1
60.6
69.5
55.8
73.5
Fi-single
67.5
83,8
79.2
92.8
23.0
21.9
60.8
69.4
55.3
73.6
FS-multiple
67.9
84.1
80.2
93.1
24.2
22.3
63.2
70.6
56.7
73.9
BERTopic
71.2
83,0
79.0
91.6
18.2
16.6
36.1
61.6
54.8
70.6
BERTopic+KNLPmenn
76.0
87.3
82.1
93.0
17.2
74.3
BERTopic+kLIMMM
75.7
87.4
82.1
93.0
17.7
17.2
63.9
69.8
56.4
74.2
k-means
66.2
83.0
77.3
92.0
20.7
20.5
59.4
67.9
52.9
72.4
k-medoids
41.7
69.5
49.3
77.7
15.7
15.9
37.6
38.8
35.9
52.4
GMM
67.7
78.9
92.7
21.5
20.6
73.2
Agglomerative
69.9
83.7
81.0
92.5
15.8
14.1
62.8
67.1
56.6
70.5
Spectral
68.2
83.3
76.3
90.9
17.6
15.2
61.5
67.0
56.6
71.4
Table Average ACC, NMI, and dist for k-means, k-NLPmeans LSA-multiple and K-LLMmeans FS- TS-multiple, evaluated on three datasets using four different embedding models.
Dataset
CLINC
GoEmo
Massive (D)
embedding/Method
NMI
dist
ACC
NMI
ACC
NMI
dist
DistilBERT
k-means
53.9
77.2
0.34
17.8
18.3
0.364
44.4
45.1
0.309
k-NLPmeans
52.9
77.3
0.353
18.5
18.2
0.365
44.7
45.3
0.311
k-LLMmeans
55.3
78.7
0.343
18.2
18.8
0.351
46.2
46.4
0.295
text-ent
Someedding---smeee
k-means
77.3
92.0
0.2
20.7
20.5
0.287
59.4
67.9
0.246
-.N.nmeans
80.0
92.9
0.173
22.3
20.2
0.29
63.3
70.0
0.227
k-LaM
80.2
93.1
0.179
24.2
22.3
0.278
63.2
70.6
0.227
e5-large
k-means
73.8
90.8
0.131
22.8
22.8
0.176
58.4
63.7
0.138
k-NL.mmaans
76.5
92.0
0.119
22.6
22.6
0.179
60.4
65.7
0.137
k-LLMmeans
77.2
92.5
0.119
24.2
24.3
0.168
62.3
65.9
0.133
S-BERT
k-means
76.9
91.0
0.215
13.7
13.3
0.355
58.2
64.6
0.271
k-NLPmeans
79.0
91.9
0.201
14.1
12.8
0.363
58.5
64.6
0.272
K-LLMmeans
79.7
92.5
0.198
14.7
13.9
0.346
59.7
65.6
0.254
Across all datasets, our methods outperform traditional baselines NMI and generally achieve higher ACC. Within k-NLPmeans, differences are modest, with LSA yielding the best overall scores among extractive summarizers. k-LLMmeans achieves the strongest results, attributable to higher- quality abstractive summaries; interestingly few-shot variants show better performance. Multiple summarization steps tend to offer additional gains over a single step. Overall, both k-NI.Pmeans and k- k-LLMmeans improve upon traditional algorithms, with k-LLMmeans delivering the best sults, and remaining efficient, as few-shot summarization appears sufficient without needing exten- sive prompt length.
We also compare with BERTopic and apply our methods to its document embeddings. While BERTopic is strong its own, adding our variants consistently improves its performance. Across all datasets, the best results come from either our stand-alone methods or their BERTopic-bart versions.

---

Published as a conference paper at ICLR 2026
Table 3: Number of LLM calls (prompts), average ACC, and average NMT for k-NI.Pmeans (LSAA
multiple) and k-LLMmeans (FSanssFFFmmmmaa (FS-multiple) using various LLMs with e5-large embeddings, compared
against BERTopic, our variants applied BERTopic embeddings, and other state-of-the-ar LLM-
based clustering methods on three benchmark datasets.
CLINC
GoEmo
Massive (D)
Dataset/Method
prompts
ACC
NMI
prompts
ACC
NMI
prompts
ACC
NMI
K- K-NLPmeans
76.5
92.0
0
22.6
22.6
0
60.4
65.7
k K-LLMmeans
GPT-3.5-turbo
750
76.0
23.9
24.1
90
63.3
66.2
GPT-40
750
77.2
92.5
24.2
24.3
90
62.3
65.9
1lama-3.3
750
77.2
92.3
135
23.8
23.1
90
62.0
66.3
DeepSeek-V3 V3
750
69.7
90.8
135
22.8
22.7
90
62.6
66.3
Claude-3.7
750
76.9
92.5
135
24.2
23.7
90
61.8
66.0
ToRiopic
77.8
90.8
0
20.3
20.8
38.5
61.7
81.5
93.0
0
23.2
22.6
60.8
66.1
BEKLopic+kLLMMean
750
81.4
93.0
135
24.0
23.0
90
58.6
65.6
ClusterLLM
1618
83.8
94.0
1618
26.8
23.9
1618
60.9
68.8
IDAS
4650
81.4
92.4
3011
30.6
25.6
2992
53.5
63.9
LLMEdgeRefine
1350
86.8
94.9
895
34.8
29.7
892
63.1
68.7
Table 4: Average ACC, and average NMI for four sequential mini-batch variants, k-means, mini- batch k-means, sequential mini-batch k-means on the yearly StackExchange data.
2020
2021
2022
2023
Year/Method
(69147
posts)
(54322
posts)
(43521
posts)
(38953
posts)
ACC
NMI
ACC
NMI
ACC
NMI
ACC
NMI
mini-hatch k-NLPmcans
68.0
79.5
67.9
78.5
69.0
78.9
71.6
78.8
mini-batch k-LLMmcans
multiple
72.5
80.9
73.6
80.5
74.4
80.5
73.4
80.3
FS multiple
75.4
73.5
80.2
72.8
72.7
80.1
k-means
73.4
80.6
67.7
79.0
68.6
79.0
72.0
79.6
mini-batch k-means
67.0
78.2
67.7
77.4
67.5
77.6
67.2
77.0
seq. mini-batch k-means
67.0
76.6
66.7
75.2
65.6
75.6
65.8
74.8
Comparing our approaches with k-means using different embeddings. Table 2 compares k- NLPmeans (.SA-mulriple) and K-LLMMeanns (Fi-mulriplle with k-means with different embeddings on three benchmark datasets, reporting average ACC, average NMI, and average Euclidean distance between the learned and ground-truth centroids (dist). The dist metric directly gauges how closely each algorithm recovers the true centroids, an especially meaningful criterion for centroid-based methods. Across all tested embeddings, our approaches achieve higher average ACC and NMI than k- m-maass producing smaller average dist values. Interestingly k-LLMmeans stands the best, demonstrating that its LLM --gidddddd centroid updates converge to solutions that are both more accu- rate and more faithfully aligned with the underlying cluster structure.
Comparing our approaches with different LLMs and state-of-the-art LLM-based clustering methods. Table the number of LLM calls (prompts) and mean ACC/NMI for k-NL.means (LSA-multiple) and K-LLMmeans (FS-multiple) on three benchmarks with e5-large embeddings. For K-LLMmeans we vary the LLM generator, including GPT-3.5, and observe stable performance across LLMs, indicating robustness to the specific model. By design, k-NLPmeans uses zero LLM calls. For context, we also include results from Feng et al. (2024): ClusterLLM (fine-tuned) and IDAS use GPT-3.5 with the same e5-lerge embeddings, while ELMEdgeRefine uses the stronger In- structor embeddings (Jin et al. 2023). Our GPT-3.5 configuration K-LLMmeans achieves compa- rable, sometimes slightly lower ACC/NMI, but with much fewer LLM calls (independent of dataset size) and no fine-tuning. k-NLPmeans trails k-LLMmeans slightly but remains competitive with- out any LLM usage. We also evaluate BERTopic and apply our variants to its embeddings. In this setting, our methods achieve results extremely close to the best LLM-based models while requiring only small number of LLM calls. Overall, framework offers favorable quality- cost trade-off: k-NL N for arr-inference-cost deployments, and k- k-mLM for higher accuracy under small, fixed summarization budget.

---

Published as a conference paper at ICLR 2026
Additional results and experiments. Results with standard deviations reported in Tables 5 6. and 7 of Appendix B.1 We also assess sensitivity to the selection of the number of clusters
IE in Appendix B.2.1 Table 9 shows robustness when the number of clusters mildly differs from the ground truth clusters, consistently better than vanilla k-means. We also evaluate sensitivity to the instruction prompt 7 in k-L..Mmeans and to parameter q in k-NLPmeans in Appendix [B.2.2] As shown in Table 10, performance remains generally stable across prompt choices and values. Table [I in Appendix [.2.3 also compares several sampling strategies for the few-shot (FS) version of k- M-mLLss showing that our k-means++ strategy has consistent performance.
Summary of static experimental results Across our experiments, our methods (i) improve over classical and topic-moeeling baselines (Table []: (ii) outperforms k-means across embeddings (Ta- b and (iii) approaches strong L.L.-based clustering methods much lower L.L. cost (Table
SEQUENTIAL (MINI-BATCH) EXPERIMENTS
We use yearly posts from 2020 2023 from 35 StackExchange sites (StackExchange) 2024). Each post is accompanied by the site label (domain) and timestamp, making this dataset well-suited for evaluating online sequential clustering methods. We release this dataset alongside our submission (see Appendix for for details). For each yearly subset, we split the data into b = [ 10000 equal-sized batches D1, ... D, in chronological order, where is the number of documents for that year. We run the mitc- k-LLMmeans algorithm in its four variants described in Section (for practical reasons we m 50 for the full cluster variant). We compare with three baselines: mini-batch k- k-means with b chronological batches,
means with standard random sampling, sequential and standard k-means on the full dataset. We use ground-trurt clusters, text-embedding-3 embeddings, GPT--4o for the summarization step, and five different seeds.
for
6.1 RESULTS WITH STREAMING DATA
Table reports average ACC and NMI on the yearly StackExchange corpus, comparing mini-batch k-NI. .meens (LSA-multiple) and mini-batch k-LL.Mmeans variants against standard k-means, mini-batch k-means, and sequential mini-batch k-means. Mini-batch k-LLMmeans variants achieve the highest ACC and NMI, consistently outperforming basclines, and even surpassing full-dataset k- means. Despite operating in a streaming regime, our approach clusters the entire corpus of 205, 05,943 posts using no more than 3,850 LLM calls, demonstrating favorable accuracy-pers per-LLLMtooken trade-off. The few-shot summarization design keeps prompts short, sidestepping context-window limits while preserving cluster quality. Mini-batch K-NLPmeans also improves over all mini-batch baselines, though trails the k-LLMmeans variants, likely because its extractive summaries are more sensitive to the noisiness and heterogeneity of community posts than LLM-generated cen- troids. Overall, these results highlight that our interpretable, sequential streams with strong accuracy and limited L.M usage.
formulation scales to long
Results with standard deviations are reported in Table 8 (Appendix B.
7
CASE STUDY
To demonstrate the interpretability of our method capturing the evolution of clusters within se- quential data, we present case study using posts from the Al site in the 2021 Stack Exchange dataset (StackExchange 2024). We apply our batc- K-LLMmeans algorithm with three equal- length batches and a total of ten clusters. We the instruction "The following is cluster of questions from the Al community. Write a single question that represents the cluster".
Interpretation. Our interpretable clustering enables temporal analysis of topic change, since each centroid is a human-readable summary rather than a latent vector. Figure 2 shows this for three major clusters: Image Model Optimization, Al Evolution and Challenges, and Advanced NLP Tech- niques. Across 2021, these clusters drift from fundamentals toward integration-at- how tools, models, and deployment pressures matured. Image Model Optimization begins with bustness anxicties (adversaries, occlusion, scale) around mainstream detectors/ssemmenter,, pivots mid-year to "how do I implement this well?" (transfer learning, augmentation, preprocessing), and
mirroring
8

---

Published as a conference paper at ICLR 2026
Claster dosctistion
First batch
Second batch
Ihird batch
Image Model Optimization
Image sbeiffcaries and object detection model, such as Chit,, YOLD, Muck R-CNN, against
How can we improse the rebsotneas and accuracy of
How can effectively imrp elmmnt and optimiee variosa deep learning technigues such traader loar sltt,
How can effectiedly utilics Al techniquss, such consolatioral sourt netw (CUUs)) and enject detection model,, todks involving image datis,
object decuction, imagi propraceste scesing, for tooks
foc
varying scales, while aftectively uttitting addttionaal data types and addinesing issues like overtitiin, sugmentation, and bounding box prediton?
regmentatian, and anamaly detrction, while
handiing imbalanced dettuett, improving model performance through techniques like trarsfer teamingg data sugmentation, and specialized loss functions?
aidressing challenges like input size incorsstencs,
data sugmertation, and model uncertainty?
evolution and challenges
ligenoo
slgorithess models like GANe, incorporate benedtt from the concupt of flaws iimiuaions, and what
spprosches, systems, effectively uttiteed combined schieve AGL condierring factors like computational effidiene, and the
netwerit,, and computational theortes, whatttre
the practical implications and challennees of
kmp lemenntingg All aneas thee garning. formal logie and general ligeelignnce
phinsophical popipii practical in implicatons there imperfections hase achiesing human level
innage stion various Al techniques, Including knoowedgg ingineeting, neuro-emn marroeymbelic methods, andreishicceeee learning while addressing
intellieesse solving complex problems?
Bay diserse application??
Advanced NLP Techniques
How can offectiuely utillne NLP and
modeo, soch as WordWVes, BERT, and Translomers,
How can we effectiusly euskute and understard the
contestted and semandic capabliitics modes like
How can effectianly utiibe pre rn-iriniedd longuage
moddd and NUP tuchniquus to handle tusks soch
for varbous tade the word enbedding sendimenn
BERT and GPT-3, conddering thei use of smbeddingss
text trandation, entity recagntion, and teut
anelysis toxt dessiication, and handling grammatical
sell atterion mecharisms, trarofer learning
classifcaaioon while addressing challiengos
these models and rale preprocentingin improving their performance?
beyond traditinna onen Iike ELEU for tastss text gineration translaton?
vocab stary, the need for accurate algrement
between beet and audio muitifiguaal conterts?
Figure 2: Sequential evolution of the LLM--enerated centroids for three primary clusters during the three batches of the sequential mini-batch K-LLMmeans process applied to 2021 posts from the AI Stack Exchange site (StackExchange 2024). Main aspects are manually highlighted.
ends with production hardening (class imbalance, specialized losses, edge efficiency) as strong pre- trained models and libraries made setup routine and pushed bottlenecks to data and deployment (Bochkovskiy et al. 2020; Kolesnikov et al., 202T). A/ Evolution and Challenges moves from his- torical "what is AT becoming?" to curiosity about large generative/multindaal systems, probably sparked by public releases like CLIP/DALL-E and their failure modes, and finishes with system- level synthesis (neuro-symbolic integration, RL control, compute efficiency, safety) as enterprise adoption and policy attention grow (Radford et al., 2021 Ramesh et al., 2021 Nayak 2021 Ope- nAl 2021 European Commission 2021). Advanced NLP Techniques tracks the transformer boom:
early questions focus on which models and preprocessing to use; mid-year attention shifts to measur- ing semantics beyond BLEU (e.g., BERTScore, BLEURT) long-context and domain-shin issues surface; late-year concerns are product-driven, sequence limits, domain vocab, multilingual/cross modal alignment; reflecting rapid uptake of pretrained encoders/decoders in industry (Brown al.) 2020 ZZhang et al., 2020b Sellam et al. 2020; Xue et al., 202T). Practically, these trajectories can power better post categorization, searchability, answer routing, and trend detection. More broadly, they showcase how our mini-bater M-mnns exposes readable centroids enabling end-to-ent in- terpretability for sequential text streams, letting practitioners track topic evolution, attribute causes, and make transparent, timely decisions in dynamic corpora.
8 RELATED WORK
Traditional clustering. Hierarchical methods (Johnson 1967:
Blashfield & Aldenderfer 1978)
build tree-structured representations of nested document relationships. Density-based approaches like DBSCAN (Ester et al. 1996) and graph-based methods detect clusters of arbitrary shapes, while spectral clustering (gg et al. (0001) leverages cigen-decomposition to uncover complex structures. Model-based techniques; including Gaussian mixture models (Dempster et al.) 1977) and recent neural network frameworks (Zhou et al. 2019 Huang et al. [014: Yang et al.) 2016]] Zhang al.
2021
Xie et al. 2016); provide probabilistic clustering formulations. Additionally, topic modeling methods, from probabilistic latent semantic analysis (Hofmann 2001) to latent Dirichlet allocation
BBlei et al. 2003), capture word co-occurrence patterns and latent topics. BERTopic (Grootendorsi) rereiiingrrruetoon-edeeeee
pipeline for interpretable topics, while DeepCluster (Caron et al. 2018) uses clustering to iteratively train encoders in self-supervised manner (or- thogonal to our fixed-embedding setting). Jia & Diaz- Rodriguud (2026) also incorporate modern embeddings into text segmentation within classical change point detection methods.
2022) offers an
LLMe- clustering. Viswanathan et al. (024) employ LLMs to augment document rep- resentations, generate pseudo pairwise constraints, and post-correct low-confidence assignments for query-efficient, few-shot semii-supervised clustering. Zhang et al.] (2023) propose ClusterLLM, which uses instruction-tuned L.MSs via interactive triplet and pairwise feedback cost-effectively

---

Published as a conference paper at ICLR 2026
refine clustering granularity. Complementary approaches (Tipimeni al. 2024 Petukhova al. 2023) show that context-derived representations capture subtle semantic nuances beyond traditional embeddings. Waang et al.] (023) introduce goal-driven, explainable method that employs natural language descriptions to clarify cluster boundaries, while (De Raedt et al., (023) present IDAS for intent discovery using abstractive summarization, Feng (2024) propose I.E.EdeRRefnne it- erative mechanism that forms super-points to mitigate outliers and reassign ambiguous edge points,
resulting in clusters with higher coherence and robustness.
DISCUSSION
We presented K-NLPmeans and K-LLMmeans, which replace the numeric centroid in k-means with textual summaries, yielding interpretable prototypes while preserving scalability. The LLM-freee k- NLPmeans achieves competitive accuracy with minimal additional complexity; k-LLMmeans aug- ments clusters with LLM summaries under fixed, peration-boundee LLM budget, offering favor- able accuracy-efficiency trade-off without fine-tuning. mini-batch extension enables streaming operation. Overall, sunmary-as-centron is simple yet powerful and novel modification that uni- fies interpretability and efficiency and broadens the applicability of k-means to modern text streams. Beyond text, summary-as--eniiee ceseetrridd could be generalized by replacing numeric centroids with embedded, domain-specific prototypes (e.g., image collages, synthetic utterances, DNA consensus motifs, representative subgraphs), preserving interpretability without altering the clustering.
Limitations. Our k-LLMmeans relies on LLM-generain summaries; thus, biases or errors in the underlying model can propagate cluster prototypes and downstream analyses. Although sim- ple instruction prompts are effective (Appendix B.2). prompt design can still influence outcomes. The few-shot variant assumes that a small set of representative samples captures cluster structure, practical in many settings but potentially limiting for heterogeneous clusters. Regarding potential pretraining exposure, note that the LLM our pipeline used solely to summarize texts already assigned to each cluster; it does not inject external labels or unseen content, so any exposure does not confer an unfair advantage. As indicated by suboptimal results on the StackExchange dataset (Table 4. k-NLPmeans may benefit from applying text-cleaning preprocessing prior use. Finally, we illustrate qualitative failure cases in Appendix
Cost. Our k-NLP Pmeans LLM-ffre; runtime is dominated by standard k-means and lightweight summarization step. In contrast, k-LLMmeans incurs one LLM call per cluster per summarization step. These calls highly parallelizable, wall-clock time is roughly the latency of single LLM call multiplied by the number of summarization steps. In practice running any static dataset with five summarization rounds using GPT-4o and cext-embeddingg--smee
cost $1 and completed in = 1 minute on single laptop without parallelization; for the Stack Overflow dataset, the end-to- end run cost $2.5 and took minutes under the same conditions. By comparison, competing LLMM-heay approaches would require $18-$25 and >> 40 minutes on identical hardware and API settings. Thus, k-LLMmeans remains interpretable and cost-efficient among LLM assistedd methods, while k-NLn provides strong, interpretable, zero-LLM alternative when budget or latency is constrained.
REPRODUCIBILITY STATEMENT
We release data and code to reproduce all results in this paper at httpp::////// fairoadiazr/o wtw.a//eeee.ammmmmmnoooor.t (see the README for instructions).
httts:///ithub. ggth/b..t.uu...g/ com/
ACKNOWLEDGMENTS
This work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) under grant AGECR-2022-04533 The author thanks Mumin Jia, the anonymous viewers, and the session chair for their valuable feedback and insightful suggestions, which greatly improved the quality of this work.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Marcel R Ackermann, Marcus Märtens, Christoph Raupach, Kamil Swierkot, Christiane Lam- mersen, and Christian Sohler. Streamkm++ clustering algorithm for data streams. Journal of Experimental Algorithmics (JEA), 20221,2012
Charu Aggarwal. A survey of stream clustering algorithms. In Data Clustering, pp. 231-258. Chapman and Hall/CRC, 2018.
Charu Aggarwal, Yu Philip, Jiawei Han, and Jianyong Wang. A framework for clustering evolving data streams. In Proceedings 2003 VLDB conference, pp. 81-92. Elsevier, 2003.
Anthropic. Claude 3.7 sonnet: Hybrid reasoning ai model.
com/
news/claude-33 s-onnet, 2025. Accessed: 2025-05-09.
David Arthur and Sergei Vassilvitskii. k-means++: The advantages of careful seeding. Technical report, Stanford, 2006.
Roger K Blashfield and Mark $ Aldenderfer. The literature on cluster analysis. Multivariate behav- ioral research, 13(3):771-295, 1978.
David M Blei, Andrew Y Ng. and Michael Jordan. Latent dirichlet allocation. Journal of machine Learning research, 3(Jan) 993n))99331022, 2003.
Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yual Mark Liao. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004. 100.34004.10944, 2020.
Paul Bradley, Olvi Mangasarian, and W Street. Clustering via concave minimization. Advances in neural information processing systems, 9, 1996.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for un- supervised learning of visual features. In Proceedings of the European conference computer vision (ECCV), pp. 132-149, 2018
Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić. Efficient in- tent detection with dual sentence encoders. In Tsung- Hsien Wen, Asli Celikyilmaz, Zhou Yu, Alexandros Papangelis, Mihail Eric, Anuj Kumar, Inigo Casanueva, and Rushin Shah (eds.), Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, pp- 38-45, Online, July 2020. Association for Computational Linguisties. doi: 10. 10.18653/w120200. nlp4convai- 1.5. BRLhttps:
/a//cclaaaa http:///aaaaattooooooo org/2020 w1p4convai-
Maarten De Raedt, Fréderic Godin, Thomas Demeester, and Chris Develder. Idas: Intent discovery with abstractive summarization. arXiv preprint arXiv:2305.199 19783, 2023,
Scott Deerwester, Susan T Dumais, George W Furnas. Thomas K Landauer, and Richard Harshman. Indexing by latent semantic analysis. Journal of the American society for information science, 41 (6):391 10799 1990.
Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series (methodological), 39(1): 1-22, 1977.
Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Su- jith Ravi. GoEmotions: A dataset of fine-grained emotions. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault ), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4040-4054, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v 1/2020.acl-mmai.. mainmma.n..77 URL https: htaas///hoaaaatoooog...ooo hor.tooooooggoogoo//ooooo
2020. acl -al-mmain 1n..-mainn372

---

Published as a conference paper at ICLR 2026
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1r1iv:1810.8 0.000, 2018.
Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second Inter- national Conference on Knowledge Discovery and Data Mining, KDD'96, pp. 226-231. AAAI Press, 1996.
European Commission. Proposal for regulation laying down harmonised rules on artificial intelligence (artificial intelligence act). COM(2021) 206 final, April 2021. URL https:// eur-lex. www.op...aa.....m...............
EEee/EN////A EN/IXT wttee//rriiccCEe SAA.ELE2X2222222000000000
Brussels, 21.4.2021.
Zijin Feng, Luyang Lin, Lingzhi Wang, Hong Cheng, and Kam-Fai Wong. LImedgerefine: En- hancing text clustering with llmebased boundary point refinement. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 18455-18462, 2024.
Jack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie, Kay Rottmann, Ana Sanchez, Aaron Nash, Liam Urbach, Vishesh Kakarala, Richa Singh, Swetha Ranganath, Laurie Crist, Misha Britan, Wouter Leeuwis, Gokhan Tur, and Prem Natarajan. MASSIVE: A 1M-example multilingual natural language understanding dataset with 51 typologically-divrsee languages. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4277- 4302, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 18653/v1/ 2023 wow.acll-ooo..000 235. URL a http:://///aaaaatoooo.o/..o org. 2ttp:/22222..1111oog..2223333
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, al. The Ilama 3 herd of models. arXiv preprint arXiv:440.221783, 2024.
Maarten Grootendorst. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXiv:2203.05794, 2022.
Thomas Hofmann. Unsupervised learning by probabilistic latent semantic analysis. Machine learn- ing, 42:177-196, 2001.
Peihao Huang, Yan Huang, Wei Wang, and Liang Wang. Deep embedding network for clustering. In 2014 22nd International conference on pattern recognition, pp. 1532-1537. IEEE, 2014.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21277 2024.
Anil K Jain and Richard C Dubes. Algorithms for clustering data. Prentice-Hall, Inc., 1988.
Mumin Jia and Jairo Diaz-Rodriguez. Dynamics of spontaneous topic changes in next token predic- tion with self-attention. Advances in neural information processing systems, 39, 2025.
Mumin Jia and Jairo Diaz-Rodriguez. Unsupervised text segmentation via kernel change-point detection on sentence embeddings. arXiv preprint arXiv:2611.18783 2026.
Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. Instructor: Instructing unsupervised conversational dense retrieval with large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 6649-6673 2023.
Stephen C Johnson. Hierarchical clustering schemes. Psychometrika, 32(3):241-254 1967.
Leonard Kaufman and Peter J. Rousseeuw. Partitioning Around Medoids (Program PAM), pp. 68- doi: 10. 161000097880477018681111
125. John Wiley & Sons, Inc., 2008 ISBN 9780470316801.
URL /ttp::///ox..o....00/00000/////1 01000100//07777733166000111
Alexander Kolesnikov, Alexey Dosovitskiy, Dirk Weissenborn, Georg Heigold, Jakob Uszkoreit, Lucas Beyer, Matthias Minderer, Mostafa Dehghani, Neil Houlsby. Sylvain Gelly, Thomas Un- terthiner, and Xiaohua Zhai, An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference in Learning Representations ICLR, 2021.
12

---

Published as a conference paper at ICLR 2026
Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. An evaluation dataset for intent classification and out-of-scope scope prediction. In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empir- ical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 1311-1316, Hong Kong, China, Novem- ber 2019. Association for Computational Linguistics. doi: 10.18653/1/D19--1131. URL
Bttts://aclanthol.oo wtw.eehooooo..o///111111310
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. 2024.
Stuart Lloyd. Least squares quantization in pem. IEEE transactions on information theory, 28(2): 129-137, 1982.
J MacQueen. Some methods for classification and analysis of multivariate observations. In Pro- ceedings of 5-th Berkeley Symposium Mathematical Statistics and Probxcbliyy/Universiiy of California Press, 1967.
Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. In Proceedings of the 2004 conference on empirical methods in natural language processing, pp. 404411, 2004. 2004.
Tomas Mikolov. Efficient estimation of word representations in vector space. arXiv preprint 2013.
Nishanth Nakshatri, Siyi Liu, Sihao Chen, Dan Roth, Dan Goldwasser, and Daniel Hopkins. Us- ing IIm for improving key event discovery: Temporal-guided news stream clustering with event summaries. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp-
4162-4173, 202..
Pandu Nayak. Mum: A new ai milestone for understanding information, May 2021. URL https:
Andrew Ng, Michael Jordan, and Yair Weiss. On spectral clustering: Analysis and algorithm.
Advances in neural information processing systems, 14, 2001.
OpenAl. Clip:
Connecting text and images; and dall-e: Creating images from text, January 2021. URL httpps://ppeai............./ com/ cttpo//innx//cip///// See also: https://opmmiiccominneellllall
OpenAl. Gpt-3.5 turbo. https://pl at form.
wtt.ooooooooo...oooooooooo aepernai.ccom/occs//mddd gpt 3- 5ttubbo 2023. Accessed: 2025-05-09.
OpenAl. text-embeeding---small.. https::/platformm openai.ccmm 2024. Accessed: 2025-02-01.
Carlos Ordonez. Clustering binary data streams with k-means. In Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery, pp. 12-19, 2003.
Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 4 conference on empirical methods in natural language processing (EMNLP), pp. 1532-1543, 2014.
Alina Petukhova, João Matos- Carvalho, and Nuno Fachada. Text clustering with large language model embeddings. International Journal of Cognitive Computing in Engineering, 6:100-108, 2025. ISSN 2666-3074, doi: https://doi.org/// 1010./.00000010000006//0// 10le 2024. sci Reieeced11 rect com/ sci wnwc/
http:://ooooogggooooooooieeeeee0 .004. URL htt.hhtts::////
Dragomir R Radev, Hongyan Jing. Malgorzata Stys, and Daniel Tam. Centroid- summarization of multiple documents. Information Processing & Management, 40(6):919-938, 2004.
13

---

Published as a conference paper at ICLR 2026
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748-8763. PmLR, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140): 2020.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alee Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning, pp. 8821-8831, Pmlr, 2021.
Nils Reimers and Iryna Gurevych. Sentence-BERT: Sentence embeddings using Siamese BERT- networks. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th ternational Joint Conference Natural Language Processing (EMNLP-IJCNLP,, pp. 3982- 3992, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: N0.18653/11/D19-144
anthol www.nooohooo.oooooooooooo o0t.0///0990//11111111110
Swen Ribeiro, Olivier Ferret, and Xavier Tamnier. Unsupervised event clustering and aggregation from newswire and web articles. In Proceedings of the 2017 EMNLP Workshop: Natural Lan- guage Processing meets Journalism, pp. 62-67, 2017.
Sanh. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.
Hinrich Schütze, Christopher D Manning, and Prabhakar Raghavan. Introduction to information retrieval, volume 39. Cambridge University Press Cambridge, 2008.
David Sculley. Web-scale k-means clustering. In Proceedings of the 19th international conference on World wide web, pp. 1177-1178, 2010.
Thibault Sellam, Dipanjan Das. and Ankur Parikh. BLEURT:
Learning robust metrics for text generation. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7881-7892, Online, July 2020. Association for Computational Linguistics. doi: 10. 10.18653/v1/2020 mainm. 704.
http:///aaaaatttooooooooooo wrwp:/rgg///00/ acl0-acll-- -aanmmiinn n.iin.0047
Haoxiang Shi and Tetsuya Sakai. Self-supervised and few-shot contrastive learning frameworks for text clustering. IEEE Access, 2023.
Jonathan A Silva, Elaine R Faria, Rodrigo C Barros, Eduardo R Hruschka, André CPLF de Carvalho, and João Gama. Data stream clustering: survey. ACM Computing Surveys (CSUR), 46(1):1-31, 2013.
StackExchange. Stack exchange data dump, 2024. URLAttts::/archivee.oos staaceeccangel Creative Commons Attribution- Shattioon-SSaa-Allkeeke 4.0 License. Accessed: 2025-02- 08.
Michael Steinbach. A comparison of document clustering techniques. Technical report, Technical Report# 00_034/University of Minnesota, 2000.
Adane Nega Tarekegn, Fazle Rabbi, and Bjørnar Tessem. Large language model enhanced clustering for news event detection. arXiv preprint arXiv:2406. 10552, 2024.
Sindhu Tipimeni. Ravinarayana Adkathimar, Nurendra Choudhary, Gaurush Hiranandani, Rana Ali Amjad, Vassilis N Ioannidis, Changhe Yuan, and Chandan K Reddy. Context-aware clustering using large language models. arXiv preprint 1rXiv:2405.009888 2024.
Vijay Viswanathan, Kiril Gashteovski, Carolin Lawrence, Tongshuang Wu, and Graham Neubig. Large language models enable few-shot clustering. Transactions of the Association for Computa- tional Linguistics, 12:321-333, 2024.
14

---

Published as a conference paper at ICLR 2026
Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Ma- jumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.035533 2022.
Zihan Wang, Jingho Shang, and Ruiqi Zhong. Goal-driven explainable clustering via language descriptions. arXiv preprint arXiv:2305. 13749, 2023.
Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In International conference on machine learning, pp. 478-487. PMLR, 2016.
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami AI-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mT5: A massively multilingual pre-trained text-to-text transformer. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tun Tur, Iz Beltagy. Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 483-498, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v 1/2021 maacl-main.41. URL ht /ttps::/aclanthol..oo org/ 2021. www.aaaaaammnnnmmm
Jianwei Yang, Devi Parikh, and Dhruv Batra. Joint unsupervised learning of deep representations and image clusters. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pp. 5147-5156, 2016.
Dejiao Zhang, Feng Nan, Xiaokai Wei, Shangwen Li, Henghui Zhu, Kathleen McKeown, Ramesh Nallapati, Andrew Arnold, and Bing Xiang. Supporting clustering with contrastive learning. arXiv preprint arXiv:2103.12955 2021.
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In International conference on machine learning. pp. 11328-11339, PMLR, 2020a.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi Bertscore: Evalu- ating text generation with bert. In International Conference of Learning Representations ICLR, 2020b.
Yuwei Zhang, Zihan Wang. and Jingbo Shang. ClusterLLM: Large language models as a guide for text clustering. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.). Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. (3903-13920 Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v 1/208/63///0222.mmmllloo. 858. URL
/202220 2203 emn emnlp-mainnn 858/|
Jie Zhou, Xingyi Cheng, and Jinchao Zhang. An end-to-end neural network framework for text clustering. arXiv preprint arXiv: 190300440 2019.
A DATASETS
A. NON-STREAMING DATASETS
We evaluate our clustering approach on four benchmark datasets. Bank77 (Casanueva al.) 2020):
Consists of 3,080 customer queries related to banking services, categorized into 77 distinct intents. CLINC [arsson et al. 2019): A diverse set of 4,500 queries spanning 150 intent classes across multiple domains, designed for open-domain intent classification. GoEmo (Demszky et al.) 2020): Contains 2,984 social media posts annotated with 27 fine-grained emotion categories. We removed the neutral expressions to address data imbalance and retained only entries with single, unique emotion. MASSIVE (FitzGerald et al. 2023): Comprises 2,974 English-language virtual assistant utterances grouped into 18 domains and 59 intent categories.
These datasets provide a robust evaluation setting for text clustering across different domains and classification granularities.
15

---

Published as a conference paper at ICLR 2026
A.2 NEW COMPILED DATASET FOR TESTING TEXT-STREAMEL CLUSTERING ALGORITTMS
We extract and unify a challenging data stream comprising unique archive posts collected from 84 Stack Exchange sites (StackExchange 2024). Each post accompanied by the site label (domain) and timestamp, making this dataset well suited for evaluating online or sequential clustering meth- ods. Our raw dataset spans 84 domains, each containing at least 20 posts per year from 2018 to 2023 (with post lengths ranging from 20 to 1 (000 characters), totaling 499,359 posts. For our experiments, we focus on posts from 2020 to 2023 and further filter out labels that do not exceed 500 posts in 2023. The resulting subset comprises 35 distinct groups and 147 posts. Both the raw and clean data are provided with this paper. Stack Exchange content is licensed under the Creative Commons Attribution-Sharte Slire 4.0 International (CC BY-SA 4.0) license.
B
SUPPLEMENTARY RESULTS
B.1
MAIN RESULTS WITH STANDARD DEVIATIONS
In Tables 5
6
and report the same results as in Tables
2 and including standard
deviations.
B.2
ADDITIONAL EXPERIMENTS
B.2.1 SENSITIVITY TO NUMBER OF CLUSTERS
We also examined the sensitivity to the parameter & (the number of clusters) by assessing the effect when the number of clusters k set below the ground truth (k-20% 10%,,, the ground truth (A), and above it (k+10%. k++)0% The corresponding results reported in Table g As shown there, both K-NLPmeans and k-LLMmeans consistently outperform kmeans regardless of the estimation error in k, and the overall impact ACC and NMI remains minor.
B.2.2 SENSITIVITY TO PROMPT 7 AND PARAMETER !
We evaluated the sensitivity of M-LLMms to the instruction prompt / on BANK77 by testing five prompt variants using k- L-LLMmeans (FS-single) with rext-embedding-3--e 3-mal and GPT-4o. We also examined the effect of the parameter g on the same dataset and embeddings. As shown in Table IO] performance remains remarkably stable across prompt choices and settings, indicating that the methods are insensitive these hyperparameters.
B.2.3 SENSITIVITY TO NUMBER SAMPLING STRATEGY FOR THE FEW-SHOT (FS) VARIANT OF K K-LLMMEANS
We also examine the effect of alternative sampling procedures for the few shot variant. We propose in the manuscript k-means++ but here we also explore uniform random selection (random), choosing closest the cluster centroid (centroid) and choosing the farthest ones (edge). The corresponding results are reported in Table As shown there, our proposed k-means++ seems to have a consistent performance.
C EVALUATION METRICS
For evaluation, we report clustering accuracy (ACC) and normalized mutual information (NMI). NMI is computed with normali implements the standard mutual-information
from sklearn. metrics, which normalization. Let Y denote the ground-truth labels and y the predicted cluster labels. We first form the contingency matrix W e NDxD with entries We Kn 7n. i, Vn. = -.},, where D is the maximum label index plus one and N the number data points. The empirical mutual information between Y and is
Wú log
N Wij
MI
Σ 45
EywWo
(E Wxz)
16

---

Published as a conference paper at ICLR 2026
Table 5: Average ACC and NMI for k-Na.mmeans and X-LaMs variants using GPT-4o, com-
pared against traditional baselines, BERTopic, and our K-NLPmeans LSA LSA-maltiple and k-LLMmeans FS FS-multiple variants applied BERTopic embeddings, using text-embedding-3- on benchmark datasets. Standard deviations of ACC and NMI in parenthesis
embeddings
BANK77
CLINC
GoEmo
Massive (D)
Massive (I)
Dataset/Method
ACC
NMI
ACC
NMI
ACC
NMI
ACC
NMI
ACC
NMI
K-NLPmeans
MunRank-single
66.4
83.6
78.7
92.4
21.5
20.9
60.6
68.9
53.0
72.5
(0.94)
(0.47)
(1.56)
(0.41)
(0.63)
(0.29)
(3.47)
(1.27)
(1.8)
(0.71)
mextRank-mulipple
66.8
83.9
79.6
92.8
21.6
21.0
60.5
69.3
53.7
72.8
(0.94)
(0.45)
(1.51)
(0.45)
(1.29)
(0.65)
(3.66)
(1.22)
(1.65)
(0.58)
Centroid-single
67.2
84.0
78.5
92.2
22.1
21.2
60.9
68.3
53.5
73.0
(0.81)
(0.39)
(1.68)
(0.37)
(0.6)
(0.41)
(3.02)
(0.91)
(2.07)
(1.13)
Centroid-multiple
67.5
84.0
79.0
92.4
21.4
21.1
60.8
69.1
73.5
(0.57)
(0.3)
(1.95)
(0.46)
(0.87)
(0.12)
(2.72)
(1.09)
(1.72)
(0.84)
LSA-single
66.7
83.7
78.8
92.5
21.9
20.5
61.2
68.7
54.1
73.0
(0.99)
(0.47)
(2.07)
(0.63)
(1.11)
(0.87)
(2.66)
(1.03)
(1.86)
(0.91)
LSA-mulriple
67.1
84.0
80.2
92.9
22.3
20.2
63.3
70.0
55.3
73.4
(0.98)
(0.42)
(0.98)
(0.41)
(0.85)
(0.56)
(3.06)
(1.02)
(1.32)
(0.72)
K-LLMmcans
single
67.1
83.6
78.1
92.5
24.0
22.3
61.1
69.6
54.0
73.0
(1.22)
(0.46)
(L.67)
(0.47)
(1.35)
(0.57)
(3.39)
(1.6)
(1.51)
(0.58)
multiple
66.7
83.6
79.1
92.8
24.1
22.1
60.6
69.5
55.8
(0.78)
(0.2)
(1.68)
(0.48)
(1.33)
(0.65)
(2.82)
(1.66)
(1.4)
(0.76)
FS-single
67.5
83.8
79.2
92.8
23.0
21.9
60.8
69.4
55.3
73.6
(1.07)
(0.36)
(L.73)
(0.43)
(0.94)
(0.64)
(3.21)
(L.09)
(L.76)
(0.77)
FS-multiple
67.9
84.1
80.2
24.2
22.3
63.2
70.6
(1.55)
(0.41)
(1.52)
(0.32)
(1.16)
(0.67)
(2.84)
(1.38)
(1.32)
(0.7)
BERTopic
71.2
83.0
79.0
91.6
18.2
16.6
36.1
61.6
54.8
70.6
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
BERTopic*KNLPmeans
76.0
87.3
17.2
65.0
70.2
(0.76)
(0.11)
(0.84)
(0.09)
(0.7)
(0.3)
(2.39)
(0.66)
(1.86)
(0.27)
BEKMopss
75.7
87.4
82.1
93.0
17.7
17.2
63.9
69.8
56.4
74.2
(1.26)
(0.17)
(0.83)
(0.2)
(0.43)
(0.3)
(L.35)
(0.63)
(1.19)
(0.12)
k-means
66.2
83.0
77.3
92.0
20.7
20.5
$9.4
67.9
72.4
(1.82)
(0.56)
(1.83)
(0.35)
(1.0)
(0.67)
(4.03)
(1.67)
(1.07)
(0.54)
k-medoids
41.7
69.5
49.3
77.7
15.7
15.9
37.6
38.8
35,9
52.4
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
GMM
67.7
83.0
78.9
92.7
21.5
20.6
56.2
68.6
53.9
73.2
(1.78)
(0.54)
(1.18)
(0.22)
(0.82)
(0.47)
(2.88)
(1.01)
(1.91)
(0.80)
Agglomerative
69.9
83.7
81.0
92.5
15.8
14.1
62.8
67.1
56.6
70.5
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
Spectral
68.2
83.3
76.3
90.9
17.6
15.2
61.5
67.0
56.6
71.4
(0.56)
(0.38)
(0.401)
(0.14)
(0.32)
(0.32)
(0.02)
(0.02)
(0.84)
(0.23)
and the entropies are
Σ, Wij
Wi
Σ. Way
Σ, W
E, Wo
H(Y) = Σ Following the default in sk NMI is then defined
log
H(Ý)=
= Σ
log
N
N
N
MI(Y, NMI(Y,) {(H(Y)+ + H
ACC is defined as the fraction of correctly assigned points after the best one-to-one relabeling of clusters. Concretely, given the same contingency matrix W, solve maximum-weigh bipartite matching using the Hungarian algorithm on W and compute
ACC
Σ WM
(JJ)EM
where M is the set of matched label pairs returned by the Hungarian algorithm.
17

---

Published as a conference paper at ICLR 2026
Table Average ACC, NMI, and dist for k-means, k-NLPmeans LSA -mlttipe and k-LLMmeans FS 7S-multiple, evaluated on three datasets using four different embedding models. Standard deviations of ACC, NMI and dist in parenthesis.
Dataset
CLINC
GoEmo
Massive (D)
embeddinn/MMtthd
ACC
NMI
dist
ACC
NMI
dist
ACC
NMI
dist
DistilBERT
k-means
53.9
77.2
0.34
17.8
18.3
0.364
44.4
45.1
0.309
(0.96)
(0.49)
(0.007)
(0,86)
(0.67)
(0.005)
(1.53)
(0,99)
(0.021)
k-NL.Pmeans
52.9
77.3
0.353
18.5
18.2
0.365
44.7
45.3
0.311
(0.25)
(0.22)
(0.003)
(0.58)
(0.38)
(0.004)
(2.44)
(1.22)
(0.02)
k-Enn
55.3
78.7
0.343
18.2
18.8
0.351
46.2
46.4
(1.1)
(0.53)
(0.009)
(0.52)
(0.73)
(0.004)
(1.35)
(1.12)
(0.015)
wext-enbedding---snieee
k-means
77.3
92.0
0.2
20.7
20.5
0.287
59.4
67.9
0.246
(1.83)
(0.35)
(0.014)
(1.0)
(0.67)
(0.005)
(4.03)
(1.67)
(0.015)
k-NL.mmeans
80.2
92.9
0.173
22.3
20.2
0.29
63.3
70.0
0.227
(0.98)
(0.41)
(0.006)
(0.85)
(0.56)
(0.007)
(3.06)
(1.02)
(0.018)
k-LLMmeans
80.2
93.1
0.179
24.2
22.3
0.278
63.2
70.6
0.227
(1.52)
(0.32)
(0.015)
(L.16)
(0.67)
(0.011)
(2.84)
(1.38)
(0.022)
e5-large
k-means
73.8
90.8
0.131
22.8
22.8
0.176
58.4
63.7
0.138
(2.34)
(0.74)
(0.01)
(0.92)
(0.73)
(0.003)
(3,89)
(1.75)
(0.02)
k-NLPmeans
76.5
92.0
0.119
22.6
22.6
0.179
60.4
65.7
0.137
(1.76)
(0.58)
(0.009)
(0.7)
(0.32)
(0.003)
(3.2)
(1.79)
(0.008)
k-LLMmeans
77.2
92.5
0.119
24,2
24.3
0.168
62.3
65.9
0.133
(1.6)
(0.4)
(0.007)
(0.76)
(0.7)
(0.003)
(1.36)
(0.78)
(0.007)
S-BERT
k-means
76.9
91.0
0.215
13.7
13.3
0.355
58.2
64.6
0.271
(1.63)
(0.5)
(0.013)
(0.62)
(0.57)
(0.005)
(2.03)
(1.09)
(0.023)
k-NLPmeans
79.0
91.9
0.201
14.1
12.8
0.363
58.5
64.6
0.272
(1.65)
(0.31)
(0.017)
(0.3)
(0,29)
(0.003)
(2.24)
(1.29)
(0.022)
X-LLMmeans
79.7
92.5
0.198
13,9
0.346
59.7
65.6
0.254
(0.73)
(0.25)
(0.008)
(0.55)
(0.5)
(0.004)
(2.54)
(0.79)
(0.019)
D ALGORITHMS
This section includes Algorithm (K-NLPmeans K-LLMmeans) and Algorithm (mini batch ver- sion).
E QUALITATIVE ANALYSIS AND FAILURE MODES
In the main paper we focused successful examples to illustrate how summary-as-centrrid make clusters more interpretable. For completeness, we provide qualitative failure cases that high- light typical limitations of our approach. The examples below are representative, not exhaustive, and are lightly anonymized paraphrased for clarity.
E. PROMPT-ECHO AND META SUMMARIES
The first failure mode mention (for k- M-Lenss is "prompt echoing", where the LLM partially repeats the instruction instead of summarizing the actual content.
Example 1 (Prompt-ccho/ / meta summary).
Here www.PP/mmmm.....aa// is initialized with the final centroids of the previous batch
18

---

Published as a conference paper at ICLR 2026
Table 7: Number of LLM calls (prompts), average ACC, and average NMT for k-NI. (LSAA
multiple) and k-LLMmeans (FS-multiple) using various LLMs with e5-large embeddings,, compared
against BERTopic, our variants applied BERTopic embeddings, and other state-of-the-art LLM-
based clustering methods on three benchmark datasets. Standard deviations of ACC and NMI in
parenthesis (not available for baseline
methods).
CLINC
GoEmo
Massive (D)
Dataset/Method
prompts
ACC
NMI
prompts
ACC
NMI
prompts
ACC
NMI
K-NLPmeans
76.5
22.6
22.6
60.4
(0.02)
(0.01)
(0.01)
(0.0)
(0.03)
(0.02)
k-LLMmeans
GPT-3.5
750
76.0
92.1
135
23.9
24.1
90
63.3
66.2
(1.16)
(0.28)
(0,65)
(0.39)
(1.47)
(0.62)
GPT-4o
750
77.2
92.5
135
24.2
90
62.3
65.9
(1.6)
(0.4)
(0.76)
(0.7)
(1,36)
(0.78)
1lama-3.3
750
77.2
92.3
135
23.8
23.1
90
62.0
66.3
(1.6)
(0.44)
(0.66)
(0.62)
(2.97)
(1.56)
DeepSeek-Vä
750
69.7
135
22.8
90
62.6
(1.02)
(0.29)
(1.68)
(0.88)
(2.24)
(0.71)
Claude-3.7
750
76,9
92.5
135
24.2
23.7
90
61.8
66.0
(1.83)
(0.52)
(1.0)
(0.65)
(4.83)
(1.99)
BERTopic
77.8
20.3
38.5
61.7
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
(0.0)
BEKTopic*KNLPmeans
81.5
93.0
23.2
22.6
60.8
66.1
(0.78)
(0.18)
(0.62)
(0.13)
(1.42)
(0.58)
BEKTopic+kLLMMMenss
750
81.4
93.0
135
24.0
23.0
90
58.6
65.6
(0.55)
(0.11)
(0.36)
(0.21)
(2,96)
(0.98)
ClusterLLM
1618
83.8
94.0
1618
26.8
23,9
1618
60.9
68.8
IDAS
4650
81.4
92.4
3011
30.6
25.6
2992
53.5
63.9
L.MFFggeRefine
1350
86.8
94.9
895
34.8
29.7
892
63.05
68.67
Table 8: Average ACC, and average NMI for four sequential mini-batch variants, k-means, mini- batch k-means, sequential mini-batch k-means on the yearly StackExchange data. Standard devia- tions of ACC and NMI in parenthesis.
2020
2021
2022
2023
Year/Method
(69147 posts)
(54322
posts)
(43521
posts)
(38953
posts)
ACC
NMI
ACC
NMI
ACC
NMI
ACC
NMI
mini-batch k-NL.Pmeans
68.0
79.5
67.9
78.9
71.6
78.8
(3.09)
(0.61)
(2.5)
(0.27)
(4.15)
(0.71)
(2.74)
(0.43)
mini-batch k- M-meansss
miltiple
72.5
80.9
73.6
80.5
74.4
80.5
73.4
80.3
(3.02)
(0.65)
(2.18)
(0.46)
(2.48)
(0.87)
(2.24)
(0.81)
FS + multiple
75.4
81.6
73.5
80.2
72.8
80.1
72.7
80.1
(2.02)
(0.48)
(2.39)
(0.7)
(2,28)
(0.91)
(1.38)
(0.45)
k-means
73.4
80.6
67.7
79.0
68.6
79.0
72.0
79.6
(3.79)
(0.75)
(2.8)
(0.25)
(4.03)
(0.69)
(3.79)
(0.56)
mini-batch k-means
67.0
78.2
67.7
77.4
67.5
77.6
67.2
77.0
(2.09)
(0.9)
(2.12)
(0.84)
(4.02)
(0.98)
(2.76)
(0.97)
seq. mini-batch k-means
67.0
76.6
66.7
75.2
65.6
75.6
65.8
74.8
(2.39)
(0.57)
(4.61)
(1.54)
(1.23)
(0.78)
(2.55)
(0.9)
Cluster snippets (paraphrased)
Generated summary
"My subscription renewed without my
"A group of user support messages asking about the topics mentioned above. focusing on account
sent."
"Please cancel my premium plan and refund the
hilling issues."
last payment."
"I was charged after thought had cancelled."
19

---

Published as a conference paper at ICLR 2026
Table 9: Average ACC and NMI when the number of clusters & is set below the ground truth
10-20% k k-10%,, at the ground truth (k), and above it (k+10%, k+20%) for k-means, k- NLPmeans (Lng-si and k-LLMmeans (FS-single, GPT-4o) using text-embedding-33 -maall,,,,,,
evaluated on benchmark datasets. Standard deviations of ACC and NMI in parenthesis.
BANK77
CLINC
GoEmo
Massive (D)
Massive
Dataset/Method
ACC
NMI
ACC
NMI
ACC
NMI
ACC
NMI
ACC
NMI
k-means
k-20%
61.1
81.6
70.4
91.1
21.4
19.7
61.4
66.8
53.0
71.7
(0.84)
(0.21)
(1.17)
(0.43)
(1.25)
(0.46)
(2.62)
(2.04)
(0.95)
(0.57)
k-10%
63.5
82.3
73.4
91.6
20.6
19.9
60.7
67.2
53,2
72.3
(2.25)
(0.77)
(1.33)
(0.54)
(1.03)
(0.47)
(3.38)
(1.7)
(1.77)
(0.63)
&
65.1
83.0
77.5
92.0
20.7
20.5
60.6
67.9
53.2
72.5
(1.36)
(0.62)
(1.55)
(0.41)
(0.21)
(0.73)
(2.93)
(1.22)
(0.94)
(0.45)
k+10%
64.8
83.1
77.8
91.8
20.0
20.3
59.1
67.7
53.1
72.7
(0.96)
(0.49)
(1.13)
(0.15)
(1.02)
(0.79)
(3,95)
(1.59)
(1.62)
(0.78)
k+400%
64.6
83.1
77.2
91.8
19.4
20.7
59.8
69.8
52.1
72.4
(1.35)
(0.54)
(1.23)
(0.25)
(0.68)
(0.69)
(0.91)
(0.38)
(0.83)
(0.64)
k-NLPmeans
k-20
62.1
82,2
71.3
91.4
21.9
19.4
62.1
67.4
53.7
72.4
(0.42)
(0.37)
(1.23)
(0.43)
(1.08)
(0.63)
(2.03)
(1.54)
(2.02)
(0.58)
k-10%
65.5
83.1
74.3
91.9
21.2
19.5
61.3
67.7
53.4
72.8
(1.61)
(0.7)
(1.25)
(0.47)
(1.22)
(0.97)
(2,94)
(1.71)
(0.92)
(0.63)
k
66.7
83.7
78.4
92.4
21.7
20.5
61.1
68.6
54.1
73.0
(0.99)
(0.47)
(2.06)
(0.66)
(1.19)
(0.97)
(2.96)
(1.11)
(2.08)
(1.01)
k+10%%
66.9
83.9
78.6
92.3
20.1
20.4
60.8
69.2
53.9
73.3
(1.03)
(0.4)
(1.22)
(0.23)
(0.38)
(0.28)
(3.48)
(1.02)
(1.76)
(0.86)
I++20
66.7
83.9
78.0
92.3
70.2
52.3
73.1
(1.44)
(0.31)
(1.3)
(0.15)
(0.98)
(0.49)
(1.43)
(0.76)
(0.76)
(0.69)
k-LLMmeans
k-20%%
62.4
82.5
71.4
91.5
24.2
20.8
60.9
67.2
55.3
72.9
(0.89)
(0.38)
(1.24)
(0.57)
(1.92)
(1.14)
(2.56)
(L.5I)
(1.18)
(0.81)
k-10%
65.3
83.2
74.8
92.0
23.2
21.0
60.4
68.2
54.3
73.0
(1.19)
(0.42)
(1.61)
(0.47)
(0.11)
(0.49)
(4.04)
(1.94)
(1.99)
(0.34)
k
66.8
83.8
79.1
92.7
22.9
21.6
60.8
69.2
55.3
73.7
(0.94)
(0.27)
(0.75)
(0,29)
(0.32)
(0.59)
(2.79)
(1.14)
(2.08)
(0.97)
k+100%
67.4
84.2
80.7
92.9
23.1
22.2
60.1
69.8
55.1
73.9
(0.85)
(0.31)
(0.09)
(0.42)
(0.54)
(0.52)
(3.78)
(1.2)
(2,14)
(0.96)
k+400%
66.6
84.1
79.2
92.9
22.2
22.6
58.7
70.0
53.5
73.6
(1.23)
(0.36)
(1.06)
(0.27)
(0.86)
(0.41)
(0.9)
(0.5)
(2.07)
(1.39)
Meta phrases like "messages" and "topies mentioned above" make the summary less useful concise description. Slightly tightening the prompt (e.g., "Do not mention that you are summarizing, and avoid meta phrases like 'questions' or 'messages"") reduces this behavior.
E.2 SPURIOUS DETAIL AND HALLUCINATED CONSTRAINTS
We also observe occasional cases where the summary introduces details that are not consistently supported by the cluster.
Example 2 (Hallucinated detail).
Cluster snippets (paraphrased)
Generated summary
"The app keeps crashing when I open the camera."
"Complaints about the mobile app crashing Android after latest security update."
"It freezes on the loading screen after the last up- date."
"The app closes automatically when try to log in."
The summary correctly captures crashes but adds a mislcading detail ("on Android") that is not con- sistently supported. Mitigations include (i) asking the LLM to avoid unsupported specifies ("avoid
20

---

Published as a conference paper at ICLR 2026
Table 10: Average ACC, and average NMI for multiple instruction prompts for k-I.L.means FS- multiple; and multiple values of q for k-NLPmeans LSA-multiple on BANK77. Standard deviations parenthesis.
ACC
NMI
k-LLMneeans (Prompt I)
The following is cluster online banking queries. Write a text that summarizes the following cluster:
67.5
83.8
(1.07)
(0.36)
The following is cluster of texts.
Write text that summarizes the following cluster:
66.3
83.6
(0.2)
(0.3)
Write query that summarizes the following online banking queries:
67.4
83.8
(0.7)
(0.3)
Write text that summarizes the following texts:
66.9
83.7
(0.6)
(0.2)
Summarize the following list of online banking queries:
67.0
83.7
(0.4)
(0.1)
Summarize the following list of texts:
66.4
83.8
(0.6)
(0.1)
k- k-NLPmeans (q)
66.7
84.0
(1.8)
(0.8)
66.4
83.8
(0.5)
(0.3)
10
66.6
83.8
(0.7)
(0.4)
15
66.4
83.7
(0.6)
(0.3)
Table 11: Average ACC and NMI for multiple sampling strategies for k-LLMmeans FS-single ing GPT-4o with text-embedding- --mmalll embeddings, evaluated on benchmark datasets. Standard deviations of ACC and NMI in parenthesis.
BANK77
CLINC
GoEmo
Massive (D)
Massive (I)
Dataset/Method
ACC
NMI
АCC
NMI
ACC
NMI
ACC
NMI
ACC
NMI
K-LLMmeans
kmeans++
67.3
84.0
79.5
92.8
23.1
21.8
60.3
69.1
55.0
73.5
(1.35)
(0.41)
(1.18)
(0.34)
(0.63)
(0.67)
(2.66)
(1.03)
(1.92)
(0.95)
random
66.2
83.6
80.6
93.0
23.4
21.9
62.2
68.0
53.5
72.8
(0.46)
(0.31)
(1.27)
(0.3)
(0.86)
(0.51)
(2.95)
(1.16)
(1.18)
(0.64)
centroid
67.2
84.0
79.3
92.5
22.5
61.6
69.5
53.9
72.8
(0.54)
(0.31)
(1.5)
(0.34)
(1.26)
(0.53)
(3,33)
(1.7)
(1.11)
(0.28)
edge
66.4
83.3
78.1
92.5
22.6
21.7
63.1
70.2
53.6
72.7
(0.58)
(0.27)
(1.62)
(0.44)
(1.33)
(0.47)
(2.08)
(0.64)
(0.99)
(0.15)
making up details such platforms versions"), and (ii) using extractive summarization in high- stakes settings.
E.3 COFRRCOMPRESSED MULTI-TOPIC SUMMARIES
Another failure mode appears when cluster genuinely mixes several distinct topies of similar fre- quency. In this case, the summarizer sometimes tries to "cover everything at once", producing an over-compressed summary that mentions multiple things but is too broad be useful.
Example 3 (Over-compressed multi-topic summary).
21

---

Published as a conference paper at ICLR 2026
Algorithm 1: K-NL. /k---//eaam
input: D {d1..... dn K I, m, I,T
for i f- to do
X; Embedding( d:);
end
for t t- to do
it then
// Initialize using k-means++ (MI..... [b f- (fmeeans+(((d....... .... k);
end
else
i mod = 0 then
Summarizaaion step every / iterations
for j - to do
Mj - Embedding (i cluster summary);
end
end
else
// k-means step for j + 1 to do
Mj C Eie(o,) X
end end
for
j f 1 to do
C, = =}:
end
for
i + 1 to n do + arg (I... d(s;;
// Assign 24 to cluster Cj.
Cj- + Cj- U {2,};
end end
return [MI.... 1 fsi..... 35}
Cluster snippets (paraphrased)
Generated summary
"How do change password?"
"Questions about managing and modifying user accounts, including changing settings and making updates."
"Where I update my email address?"
"How do delete my account permanently?"
"Can I change my username without losing data?"
Here, the summary technically mentions "managing and modifying user accounts" but compresses several distinet operations (password reset, email change, account deletion, username change) into a single vague description. This makes the prototype less informative for downstream users who might want distinguish, for example, deletion vs. simple edits. In practice, ing slightly more detailed summaries (e.g., "changing passwords, emails, usernames, and deleting accounts") or generating short bullet-style summaries can alleviate this issue, but also highlights the inherent tension between brevity and specificity when clusters are truly multi-topic.
found that allow-
E.4 OVERLY GENERIC SUMMARIES
This failure mode is opposite the previous one. It happens when summaries are too generic and do not convey the specific intent or topic of the cluster. This often occurs when the cluster is relatively heterogeneous or when the summarizer is forced to be very short.
Example 4 (Overly generic summary).
22

---

Published as a conference paper at ICLR 2026
Algorithm 2: Mini-batch k-NI.meeans MmmLLMs
input: [D1,... Do),k, 1, m,,,T // batches of documents
for j to do
C = 03:
end
(PI } +- (,,...,0}; 0};
for i + to do
// Compute K-NLPmeans / k X-LLMmeans with documents in batch (pi...., ........ C. + k k-NLPPeans // Update centroids proportional to cluster and batch sizes for j 1 to do 1071
(D; T, m, k-LMMMeans (D, k m,,,
Mj My (1 71) + mp;;
end
end
return
( |k (S (.b
Sb}
Cluster snippets (paraphrased)
Generated summary
"I lost access to my card, you freeze it?" "My card was stolen and I need replacement." "Please block my card, someone used without permission."
"Questions about using the banking service."
The cluster is clearly about card freezing / blocking, but the summary collapses this into a vague description. The summary is still related to the domain but less helpful for human interpretation than it could be. Allowing slightly longer summaries and explicitly prompting for "key actions or problems" tends reduce this failure mode.
E.5 MINORITY-TOPIC OVERSG
A fifth failure mode appears when cluster contains a dominant subtopic alongside smaller but important minority subtopic. Because our approach uses a single prototype per cluster, the summary can under-represent the minority.
Example 5 (Minority-topic overshadowing).
Cluster snippets (paraphrased)
Generated summary
"How do book a train ticket for tomorrow? ?"
"Questions about booking train tickets."
"Can I book round trip in one payment?"
"I need to cancel my ticket and get refund."
Here, the summary focuses on the majority "booking" pattern and omits cancellation/refund, which might be crucial for downstream labeling. This limitation inherent to any single-prototype method (numeric or textual). Diversity-aware sampling within the cluster (for k-LLMmeans) and permitting a brief list subtopics (e.g., "booking and cancelling train tickets") can partially mitigate this issue.
E.6 MILDLY MISLEADING EMPHASIS
sixth failure mode arises when the summary is broadly correct but emphasizes a less central aspect of the cluster, which can be slightly misleading for a human reader.
Example 6 (Mildly misleading emphasis).
23

---

Published as a conference paper at ICLR 2026
Cluster snippets (paraphrased)
Generated summary
"How can change the delivery address for my or- der!"
"Questions about tracking online orders."
"I moved, can you update the shipping address?"
"Can edit the address before the package sent?"
The summary remains in the e-commerce domain but shifts the focus from updating addresses to tracking. This type of error is usually easy to spot and correct by human annotator, but shows that summaries are not guaranteed perfectly match the most salient aspect of cluster.
E.7 CROSS-LINGUAL
CODE-MIXED DRIFT
Finally, on multilingual or code-mixed data (e.g., MASSIVE or StackExchange posts mixing En- glish and another language), we occasionally see summaries drifting toward the dominant language or dropping non-English nuances.
Example 7 (Cross-lingual drift).
Cluster snippets (paraphrased)
Generated summary
"(Puedo cambiar idioma de la app a español?"
"Questions about using the app."
"How do switch the interface to French?"
"Comment changer la langue par défaut de
l'application
Here, the summary fails to capture the central concept of changing the app language and collapses to a very generic description. Using multilingual embeddings and explicitly mentioning in the prompt that the summary should "describe the main problem or request, even if the questions are in different languages" improves such cases but does not completely eliminate them.
E.8 DISCUSSION ON FAILURE MODES.
These qualitative cases illustrate that our textual prototypes are infallible: they can be too generic, under-represent minority subtopics, echo the prompt, hallucinate details, or struggle with multilingual nuance. Nonetheless, our quantitative results show that summary based centroids con- sistently outperform numeric centroids across datasets and embeddings, and provide an interpretable handle on cluster semantics. the worst case, poor summary behaves like a suboptimal centroid update and the procedure effectively falls back toward a vanilla k-means solution, so performance does not catastrophically degrade. We view developing more robust prompts, stronger multilin- gual summarization, and multi- molti-prototype extensions (e.g., per-cluster sub-summaries) as promising directions for future work.
24


Reviews:
[{"review_id": "wdI7UDzyo6", "reviewer": "Reviewer_ywzp", "summary": "The work proposes to modify standard k-means clustering such that instead of using the mean of the embeddings of each element in the cluster to compute the centroid, we generate a textual summary of cluster population and then embed the summary text to get the centroid. The summarization can be done for a subsample of the current cluster population using either LLMs or traditional summarization methods. This method of computing centroids is applied in certain iterations of the kmeans algorithm besides the usual arithmetic centroid calculation in other steps. This approach demonstrates better clustering performance as measured by accuracy and NMI scores with respect to ground truth labels over multiple datasets, and also when using diverse embedding models for embedding the texts. The authors apply this approach also to a streaming based setting where data comes in batches over time. For this, they create a benchmark based on StackExchange posts divided over multiple years. They demonstrate performance gains from using the summary as a centroid approach on this benchmark and also release this benchmark.", "questions": "Have you done analysis of what happens when and if the embedding of the summary lies outside the convex hull of cluster's datapoints? Intuitively it seems that the centroid must stay inside the hull and this is guaranteed to be true when the centroid is the arithmetic mean. However, since this is not guaranteed that the embedding of the summary would behave this way, I wonder if the stability of convergence would be affected negatively by this.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposal of using summaries as centroids in k-means clustering looks like a very innovative and easy to implement approach.\n- The authors included diverse ways of computing the summary centroid besides simply querying an LLM such as TextRank, SVD etc which can provide computationally cheaper alternatives.\n- The approach shows good gains in performance over standard k-means clustering demonstrated consistently across many datasets and while using many different embedding models (Table1-4).\n- The authors experiment with clustering in both full dataset and batched (streaming) modes, showing benefits in both settings.", "weaknesses": "Not much of a weakness but a suggestion : I would suggest elaborating on the NMI metric to give the readers a brief explanation of how it is calculated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VRplON20lm", "reviewer": "Reviewer_KSHB", "summary": "In this manuscript, the authors aim to address the problems of poor interpretability of numeric centroids and high scalability costs in traditional text clustering methods. Specifically, the proposed k-NLPmeans uses lightweight and deterministic classical NLP summarizers to periodically replace numeric centroids with textual summaries. The proposed k-LLMmeans leverages LLMs for summaries under a fixed per-iteration budget. Experimental results across diverse datasets and embedding models show that the proposed method outperform baselines.", "questions": "See the Weaknesses.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "In this manuscript, the authors aim to address the problems of poor interpretability of numeric centroids and high scalability costs in traditional text clustering methods. Specifically, the proposed k-NLPmeans uses lightweight and deterministic classical NLP summarizers to periodically replace numeric centroids with textual summaries. The proposed k-LLMmeans leverages LLMs for summaries under a fixed per-iteration budget. Experimental results across diverse datasets and embedding models show that the proposed method outperform baselines.", "weaknesses": "There are some concerns for the manuscript as follows:\n\n1.How to set k in the experiments? The influence of k in the k-means to the experimental results is not discussed.\n2.In the example of Figure 1, it is based on the results of k-means. However, in the proposed method, the authors proposed new summarization to compute a textual prototype in place of the standard centroid update. Thus, how the proposed method guarantee that the instances in the same cluster can be used to generate promising summaries? In other words, if the texts in the same cluster express very different topics, how can it be ensured that the generated summary can cover all the topics?\n3.The experimental results on Table 1 are not satisfied. The baseline including agglomerative of 1967 year can still achieve impressive performance in some case.\n4.In the experimental results reported on Table 3, the proposed method does not seem to show advantages either. So, how can the effectiveness of the proposed method be proved?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TD9s2Je3sJ", "reviewer": "Reviewer_hKNu", "summary": "The paper proposes k-NLPmeans and k-LLMmeans, two variants of k-means for text clustering in which textual summaries are used as centroids. At scheduled iterations, the numeric centroid of each cluster is replaced by a textual summary of its documents, which is then re-embedded to continue k-means updates. The authors show that this “summary-as-centroid” update improves interpretability, improves semantic faithfulness of cluster prototypes, and often boosts accuracy. They also provide a mini-batch extension for streaming text. Experiments across several benchmarks and embeddings show improvements over classical clustering baselines and competitive accuracy relative to LLM-based clustering, while using a fixed LLM budget.", "questions": "1. **On summarizer choice:** Could the authors provide guidance or heuristics on when extractive vs. abstractive summaries are preferable? In particular, do you expect different summarizers to be more effective under heterogeneous vs. homogeneous clusters? Is there any metric that could be used to adaptively select the summarizer type?\n2. **On interpretability baselines:** Since interpretability is a key motivation for the work, why are topic-model-style methods not included as baselines? These approaches also produce human-readable cluster prototypes, so they seem more directly comparable than centroid-only methods.\n3. **On failure modes:** All interpretability examples provided in the paper illustrate successful summarization. Could the authors provide examples or qualitative analysis of failure cases (e.g., when summaries are too generic or misleading) to better understand the limitations of the technique?\n4. **On sampling strategy in few-shot summarization:** The few-shot variant relies on sampling only a subset of cluster documents to feed the LLM summarizer. Have you evaluated whether different sampling strategies (e.g., diversity-based, entropy-based, or centroid vs. edge-document sampling) materially affect summarization quality and, in turn, clustering performance?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Simple but novel idea:** The notion of introducing interpretable textual centroids inside the k-means loop is elegant, practical, and original. It creates a direct, auditable link between cluster means and human-interpretable summaries.\n- **Interpretability without post-hoc processing:** Unlike topic models or LLM-based pipelines that only label clusters afterward, the prototype is the cluster, which is useful for debugging, transparency, and downstream analyst workflows.\n- **Low-resource applicability:** The LLM-free version (k-NLPmeans) is already stronger than vanilla k-means and viable in offline or low-budget environments.\n- **Streaming extension:** The mini-batch variant is an important practical contribution.\n- **Reproducibility:** Anonymous code is supplied, and the method is easy to reimplement.", "weaknesses": "- **Summarization hints:** Performance depends on the summarizer, especially in heterogeneous clusters. The paper tests several summarizers but does not provide guidance on when one strategy is preferable (e.g., extractive vs. abstractive by dataset characteristics).\n- **Missing comparison on interpretability:** Interpretability is a key selling point, but comparisons are mostly against centroid-based clustering. Topic-model-style baselines (e.g., BERTopic,) would give a fairer interpretability comparison.\n- **No analysis of failure cases:** Interpretability examples are all positive. The paper does not provide examples of misleading or low-quality summaries, which are critical for understanding robustness and limitations.\n- **Limited cluster sampling analysis:** The few-shot summarization step relies on sampling a subset of documents per cluster, but the effects of different sampling schemes are not studied. This leaves uncertainty regarding robustness under noisy clusters.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fDtzRr0q9A", "reviewer": "Reviewer_CgDo", "summary": "This paper proposes a method that analyzes clustering centroids from a semantic perspective, aiming to enhance clustering performance by integrating large language model–based semantic understanding. The idea is interesting and relevant, particularly in its attempt to move beyond traditional embedding-based approaches. However, several aspects remain unclear, and the overall level of innovation is somewhat limited.", "questions": "1. How does the proposed semantic clustering method fundamentally differ from using pretrained embedding models like BERT or SimCSE for vector-based clustering?\n﻿\n2. Can the authors provide ablation studies or comparative results to isolate the contribution of the summarization-as-centroid mechanism from other components?\n﻿\n3. How robust is the proposed approach when applied to diverse domains, noisy data, or varying semantic granularities? does it generalize beyond the tested datasets?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper addresses an important problem in semantic text clustering by exploring how cluster centroids can be derived and interpreted from a semantic perspective rather than purely from embedding similarity. This direction is conceptually appealing, as it attempts to bridge the gap between surface-level vector representations and high-level semantic understanding. The integration of large language model (LLM) semantics into the clustering process provides a meaningful enhancement over conventional embedding-based methods, offering improved interpretability and potentially richer cluster representations. The idea of treating summaries as centroids introduces an intuitive way to represent cluster meaning in natural language, which aligns well with current trends toward explainable and human-understandable machine learning. Empirically, the method demonstrates promising improvements on selected benchmarks, suggesting that semantic augmentation can indeed refine clustering boundaries.", "weaknesses": "A major weakness of the paper lies in the lack of clarity regarding its methodological distinction from existing embedding-based clustering approaches. Although the authors claim to perform clustering from a “semantic” perspective rather than through vector representations, it is not clearly explained how this process fundamentally differs from using pretrained models such as BERT or SimCSE to obtain semantic embeddings. Without a formal theoretical or algorithmic justification, the reader is left uncertain whether the proposed method truly captures semantics in a new way or simply reuses existing embedding mechanisms under a different formulation. In addition, while the paper presents interesting results, the novelty appears incremental, mainly in the use of summaries as cluster centroids without demonstrating a substantial methodological breakthrough.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)