Explaining Datasets in Words: Statistical Models with Natural Language Parameters
Authors: Ruiqi Zhong
Heng Wang
Dan Klein
Jacob Steinhardt
Venue: NeurIPS 2024 poster (2024)
Keywords: language model; explainability; exploratory analysis; data science; explainable modeling
Primary area: natural_language_processing
TL;DR: We explain datasets by modeling with natural language parameters (e.g. clustering where each cluster is associated with an explanation). We propose an algorithm based on continuous relaxation and iterative refinement to learn these models.

Abstract: To make sense of massive data, we often first fit simplified models and then interpret the parameters; for example, we cluster the text embeddings and then interpret the mean parameters of each cluster.
However, these parameters are often high-dimensional and hard to interpret.
To make model parameters directly interpretable, we introduce a family of statistical models---including clustering, time series, and classification models---parameterized by *natural language predicates*. 
For example, a cluster of text about COVID could be parameterized by the predicate ``*discusses COVID*''.
To learn these statistical models effectively, we develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradient descent and discretizes them by prompting language models (LMs).
Finally, we apply our framework to a wide range of problems: taxonomizing user chat dialogues, characterizing how they evolve across time, finding categories where one language model is better than the other, clustering math problems based on subareas, and explaining visual features in memorable images.
Our framework is highly versatile, applicable to both textual and visual domains, can be easily steered to focus on specific properties (e.g. subareas), and explains sophisticated concepts that classical methods (e.g. n-gram analysis) struggle to produce.

Explaining Datasets in Words:
Statistical Models with Natural Language Parameters
Ruiqi Zhong"
Heng Wang
Dan Klein
Jacob Steinhardt
Abstract
To make sense of massive data, we often first fit simplified models and then inter- pret the parameters; for example, we cluster the text embeddings and then interpret the mean parameters of each cluster. However, these parameters often high- dimensional and hard to interpret. To make model parameters directly interpretable, introduce a family of statistical models--iccludig clustering, time series, and classification models pae--pteraaaeeeee by natural language predicates. For example, cluster of text about COVID could be parameterized by the predicate "discusses COVID", To learn these statistical models effectively, we develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradi- ent descent and discretizes them by prompting language models (LMs). Finally, we apply framework to wide range of problems: taxonomizing user chat dialogues, characterizing how they evolve across time, finding categories where one language model better than the other, clustering math problems based on subareas, and explaining visual features in memorable images. Our framework is highly versatile, applicable to both textual and visual domains, can be easily steered to focus on specific properties (e.g. subareas), and explains sophisticated concepts that classical methods (e.g. n-gram analysis) struggle produce
Introduction
To analyze massive datasets, often fit simplified statistical models and interpret the learned parameters. For example, to categorize set user queries, we might cluster their embeddings, look samples from each cluster, and hopefully each cluster corresponds an explainable category, e.g. "asks about COVID symptoms" or "discusses the U.S. Election". Unfortunately, each cluster might contain an uninterpretable group of queries, thus failing to explain the categories.
Such a failure is not an isolated incident: many models explain datasets by learning high dimensional parameters, but these parameters might require significant human effort interpret. For example, BERTopic [18] learns uninterpretable cluster centers over high-dimensional neural embeddings. LDA 7l Dynamic Topic Modeling [6] (time series), and Naive Bayes (classification) learn weights
a large set of words/phrases, which do not directly explain abstract concepts [9,52] (3) We want model parameters that
interpretable, since explaining datasets is important in machine
learning [60), business [4, political discussion 47 and science [17 34].
To make model parameters directly interpretable, we introduce family of models where some of their parameters are represented as natural language predicates, which are inherently interpretable. predicate to extract a 0/1 feature by checking whether it is true
Our core insight is that on a sample. For instance, given the predicate & = "discusses the U.S. Election", its denotation [o]] is binary function that evaluates to on texts x discussing the U.S. Election and 0 otherwise:
[ : "discusses the U.S. Election (z "Is Georgia a swinging state this year?") 1.
ruiqi-zhone berkeley corresponding author. All authors affiliated with UC Berkeley.
Our code and dataset are at
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Cluster assignment
Leamed features
Learned features
Class labels
Sample at time
Cluster descriptions
Learned weights
Sample text
Text samples
Weight evalving across time
aasks about the U.S. Election discusses pandemics
>; [[x asked by Ted]
[10, 10]
W
pix (%)
gep
huge.
Ted asks about the Election not about the pandemics
Georgia swinging state?
Mix (5)
Can It go jogging outdoor during lockdown??
(Year) 2018 2019
2020
2021
2022
got
fatality
Figure Our framework can use natural language predicates parameterize wide range of statistical models. Left. A clustering model that categorizes user queries. Middle. A time series model that characterizes how discussion changes across time. Right. A classification model that summarizes user traits, Once we define the model, we learn & and w based on (and y).
Using these 0/1 feature values, we define a wide variety of models, including clustering, classification, and time series modeling, all parameterized by natural langauage predicates (Figure
Learning these predicates & requires optimizing them to maximize the log-likelihood of the data. This challenging because o discrete and thus do admit gradient-based optimization. We propose general method effectively optimize di create continuous relaxation á of and optimize with gradient descent; then we prompt an LLM explain the behavior of o, thus converting back to discrete predicates (Section 4). We repeat this process to iteratively improve performance.
To evaluate our optimization algorithm, create statistical modeling problems where the optimal predicate parameters are known, so we can use them as the ground truth. We evaluated on three different statistical models (clustering, multilabel classification, and time series modeling, as illus- trated in Figure and used five different datasets (NYT articles, AG-News, DBPedia, Bills, and We found that both continuous relaxation and iterative refinement improve
Wiki 40 performance; additionally, our model-agnostic algorithm matches the performance (2% increase in F1 score) of the previous algorithm specialized for explainable text clustering (53)
58 32 23
Finally, we show that our framework is highly versatile by applying it to a wide range of tasks: taxonomizing user chat dialogues [59], characterizing how they evolve, finding categories where one language model better than the other, clustering math problems (1 based on their subareas, and explaining what visual features make image memorable [24]. Our framework applies to both text and visual domains, can be easily steered to explain specific abstract properties, and explains complicated concepts that classical methods (e.g. n-gram regression/topic model) struggle to produce. Combining I.LM's ability to generate explanations along with traditional statistical models' ability to process sophisticated data patterns, our framework holds the promise to help humans better understand the complex world.
2 Related Work
Statistical Modeling in Text. Statistical models based on n-gram features or neural embeddings are broadly used to analyze text datasets. For example, logistic regression or naïve Bayes models frequently used explain differences between
distributions [5]]: Gaussian mixture models topic models can mine major topics across large collection of documents [ and across time 6 However, since these models usually rely on high-dimensional parameters, they are difficult to interpret: for example, human studies from [9] show that the most probable words for a topic might not form semantically coherent category. interpret these models, prior works proposed to explain each topic or cluster by extracting candidate phrases either from the corpus or from Wikipedia [8 49 (77. Our work complements these approaches to explain models with natural language predicates, which are potentially more flexible.
on pre-trained embeddings can create text clusters 2

---

Prompting Language Model to Explain Dataset Patterns. Our algorithm heavily relies on the
ability of LLMs to explain distributional patterns in data when prompted with datasets [3
46
61
62 15 have prompted L.MSs to explain differences between two text distributions;
53
38
50
27
prompted L.MMs to generate topic descriptions over unstructured texts;
44
22 64
prompted L.MMs
to explain the function that maps from an input to an output; 45 prompted LLMs to explain
what inputs activate a direction in the neural embedding space. However, these works focused on individual applications or models in isolation; in contrast, our work creates a unifying framework to define and learn more complex models (e.g. time series) with natural language parameters.
Concept Bottleneck Models (CBM). CBMs aim to achieve explainability by learning a simple model over set of interpretable features (26). and recent works have proposed to extract these features using natural language phrases/predicates [3, 55 30 12 41. While most of these works focus on classification tasks, our work formalizes a broad family of models-including clustering and time series and proposes a model-agnostic algorithm to learn them. Additionally, these prior works focus on downstream task performance (e.g. classification accuracy), thus implicitly assuming that the model grounds the feature explanations in the same way as humans; in contrast, since our focus is on ex explanations, we focus on our algorithm's is ability to recover ground truth explainable features.
We discuss more related work on discrete prompt optimization, exploratory analysis, and learning with latent language in Appendix A
Mathematical Formulation
3.1 Predicate-Conditionee Distribution
In order to model text distributions with natural language parameters, we introduce a new family of distributions, predicate-conditionn distributions; these distributions will serve as building blocks for the models introduced later, just like normal distributions are building blocks for many classical models like Gaussian Mixture Kalman Filter. reddicate-cndittinnl distributions p are supported on the set X of all the text samples we observe from the dataset, and they are parameterized by (1) a list of K predicates 5 E ", and (2) real-valued weights w E RK on those predicates. Formally,
p(x o. w) x e (w)
We now explain how to (1) extract feature vector from using 5. (2) linearly combine Ø by re-weighting with w, and (3) use the reweighted values define p(æ ww, Ø).
Natural Language Parameters ē. Each predicate ó E 4 natural language string and its denotation [o] : X + {0, 1} maps samples to their value under the predicate. For examplc, if & = "is sports-related", then [o][[] love soccer.")= Since model typically requires multiple features to explain the data, we consider vectors 5 E k of K predicates, where now [o] maps X to {0, 1
[~]()) (I),1(x), [d=]),)[[))),...))))>))))))))<)) ... [ox](x)).
To instantiate I.I computationally, we prompt language model check whether is true the input æ, following the practice from prior works [2] See Figure for the prompt we used.
Reweighting with w. Consider the following example:
w [- z = Iis in English", "is sports-related"
(3)
Then wT( has value of 5 1 + 3 0 = - for an English, non-sports related sample x. More generally, w [
(Ir) is larger for non- -on-Ennlish sports-related samples.
Defining p(z T, w). According to Equation 1 p( 5, w) is a distribution over X, all the text samples we observe, but it puts more weights on x with higher values of w w=)))(<)). Using the example w and 5 above, p( 5, w) has higher probability on non-English sports-related texts.
Finally, we define Uxx) the uniform distribution over X for later use.
3

---

Denotation: HAROO
Discretization: Discrelizerg))
Check
PROPERTY. output
corpus samples,
highest score.
complese tillowing
"winele demomttatted nemarkatle proweex." (soore: -0.9)
player?' (score:
PROPERTY: sports-related
TEXT. socoar"
U(3) cosLe, ())
"Wondardd painting ... |score:
Ougput: yes
Please suggest predicates about the text samples that are more likely to achieve higher scores.
The input variables illustrated blue and the output of each prompt bold
Your sok
"has topic of
"has topic of sports"
Figure 2: Left. The prompt to compute (x)). Right. The prompt to Discretize 6 which
generates a set of candidate predicates based on samples from U and their scores cos Qk).
3.2 Example Models Parameterized by Natural Language Predicates
We introduce three models parameterized by predicates: clustering, time series, and multi-label classification. For each model, we explain its input, the learned parameters & and w. the log-likelihood loss L, and its relation classical models.
Clustering. This model aims to help humans explore a large corpus by creating clusters, each explained by predicate. Such model may help humans obtain quick overview for large set of machine learning imputs [60], policy discussions [47]. or business reviews [4. Given a set text X, model produces set of K clusters, each parameterized by learned predicate Φ& for example, if the predicate "discusses the U.S. Election", then the corresponding cluster is a uniform distribution over all samples in X that discuss the U.S. Election.
Similar to K-means clustering, each sample æ assigned to a unique cluster. We use a one-hot basis vector by E KRK to indicate the cluster assignment of and set wz br. where has large value (e.g. 10). We maximize the total log-likelihood:
w( = Σ
log(p(æ Ø. w²));; We T be, where J + co and by is a basis vector. x
However, some samples might not belong to any cluster and thus have probability: to prevent infinite loss, add another "background cluster" (a:) that is uniform over all samples in X; therefore, each sample x can back off to this cluster and incur a loss of at most log U (xx) ) = Io)(
Time Series Modeling. This model aims to explain latent variations in texts that change across time; for example, finding that an increasing number people "search about flu symptons" (d) can help us forecast a potential outbreak []6]. Formally, the input is sequence of T text samples X 17" Our model produces predicates du that capture the principle axes variation z across time. We model W1 ... Wr being drawn from a Brownian motion, i.e.,
p(a, 3, we) x exp(w, [ (x));
Wt w4-1 + (O
where À a real-valued pprem-prreeetrr that regularizes how fast w can change. The loss L is hence
I
T-1
L(Ø, w)
log(p(æ G. wx)) +
Σ I - w
(5)
Multiclass Classification with Learned Feature Predicates. This model aims explain the decision boundary between groups texts, e.g. explaining what features are more correlated with the fake news class [55 compared to other news, explaining what activates neuron [5]. Suppose there C' classes in total; the dataset a set of samples I; each associated with class y: E [C]. Our model is hence linear logistic regression model on the feature vectors extracted by i.e.e.
L W) = Σ log( e log
logits(z:) x.)) = W (x;);
(6)
where W BCCK is the weight matrix for logistic regression.

---

Method
We can now learn the parameters for each model above by minimizing the loss function C. Formally,
www.aingggaa...o..ooo....o.a..r PEOK ((o, w).
However, optimizing is challenging, since it is discrete and therefore cannot be directly optimized by gradient-based methods. To address this challenge, we develop a general optimization method, which we describe high level in Section 4.1 introduce its individual components in Section 4.2 and explain our full algorithm in Section 4.3
4.1 High-Level Overview
Our framework pieces together three core functions that require minimal model-specine design:
1. OptW, which optimizes w.
2. OptRelaxedPhi, which optimizes continuous relaxation 6x for each predicate dx-
3. Discretize, which maps from continuous predicate Φx to list of candidate predicates.
Using these three components, our overall method initializes set of predicates by first optimizing w and é using OptW and OptRelaxedPhi and then discretizing ő with Discretize. To further improve the loss, it then iteratively removes the least useful predicate, re-optimizes its continuous representation, and discretizes it back to a natural language predicate.
To provide more intuition for these three components, we explain what they should achieve in the context of clustering. OptW should optimize the 1-hot choice vectors wx by assigning each text sample to the cluster with maximum likelihood. OptRelaxedPhr should find continuous cluster representation शo similar to the sample embeddings assigned to this cluster, and Discretize generates candidate predicates that explain which samples' embeddings are similar to Φs. Next, introduce these three components formally for general models with predicate parameters.
4.2 Three Components of our framework
OptW optimizes while fixing the values of Φ. Formally, OptW(o) argmin L(,, w).
This function needs to be designed by the user for every new model, but it is generally straightforward: in the clustering model, it corresponds finding the cluster that assigns the highest probability for each sample; in classification, it corresponds to learning a logistic regression model; in the time series model. the loss is convex with respect to and hence can be optimized via gradient descent.
For later use, we define the fitness of list of predicates @ as the negative loss after is optimized:
Fitness (σ) == -( Op)W(o)).
(8)
Next, we discuss OptRelaxedPin The parameters are discrete strings, so the loss function is not differentiable with respect to J. To address this, we approximate [0](x) with the dot product of two continuous vectors, σx where e₂ Rd is a feature embedding of x normalized to unit length (e.g. the last-layer activations of some neural network), and &x Rd is unit-length, continuous relaxation of Φt. Intuitively, if the optimal á "is sports-related" and x a sports-related sample with [o]] (ar) = 1, then we hope that would correspond the latent direction encoding the sports topic and it has high similarity with the embedding e, of x. Under this relaxation, L becomes differentiable with respect to dy and can be optimized with gradient descent.
Formally, PotRelaxeddhi optimizes all continuous predicates ल1...k given a fixed value of w:
OptRelaxedPhi (w) argmin 2
21.8 L( w).
We sometimes also use it to optimize single continuous predicate ex given fixed and all discrete predicate variables other than du (denoted as d_
OptRellxedPhiiddx. w) argmin ((0 w).
(10)
5

---

Reference
Size
Learned
Size
Surface
F1
"artist"
0.07
"music"
0.12
0.50
0.37
"animal"
0.07
"a specific species of plant or animal"
0.14
0.50
0.65
"book"
0.08
"literary works"
0.07
0.50
0.64
"politics"
0.06
"a political figure"
0.06
0.50
0.96
"plant"
0.07
"a specific species of plant or animal"
0.14
0.50
0.68
"cmmany"
0.08
"business and industry" "schools"
0.07
0.50
0.83
"cchool"
0.06
0.07
1.00
0.97
"athlete"
0.07
"sports"
0.07
0.50
0.98
"building"
0.08
"historical buildings"
0.08
0.50
0.92
"film" "
0.06
"film"
0.07
1.00
0.91
... Table I: We compare the reference predicates and our learned predicates when clustering the DBPedia dataset. We abbreviate the predicates, c.g. "art" "has a topic of art". For each reference, we match it with the learned predicate that achieves the highest Fi-score at predicting the reference denotation. We also report the surface similarity (defined in Section 5.2) between the learned predicate and the reference. Our learning algorithm mostly recovers the underlying reference predicates, though it sometimes learns larger/correlated cluster that disagrees with the reference but still meaningful.
Finally, Discretize converts 6& into list of M discrete candidate predicates to update the variable Øk. Our goal is to find @ whose denotation is highly correlated with the dot product simulation Φy- Ez.
To discretize dy. we prompt a language model to generate several candidate predicates and then draw samples
(1: Cand sort them based their dot product .x. We then prompt a language model with these sorted samples and ask it to generate candidate predicates that can explain what types of samples are more likely to appear later in the sorted list (Figure [ bottom). To filter out unpromising predicates, we re-rank them based on the pearson-r correlations between [o] and Φ ex on U cannot be negative (e.g. clustering), and the absolute value of pearson-r correlation otherwise. We then keep the top-M predicates.
re-rank them. Concretely,
4.3 Piecing the Three Components Together
Our algorithm has two stages: we first initialize all the predicate variables and then iteratively refine each of them. During initialization, we
1. randomly initialize continuous predicates & to be the embedding of random samples from X
2. optimize L(Ø, w) by alternatively optimizing and all the continuous predicates ő with OptW and OptRelaxedPhn and
3. set Φb as the first candidate from Discretize((t)
During refinement, we repeat the following steps for S iterations:
1. find the least useful predicate dx: we define the usefulness of du how much the fitness would decrease if we zero it out, i.e. -Fitness( 0).
2. optimize de using OptRelaxedPhi and choose the fittest predicate from Discretize(1s)
We include a formal description of our algorithm Appendix Algorithm
Experiments
In this section, we benchmark our algorithm from Section we later apply it to open-ended applications in Section 6 We run algorithm datasets where we know the ground truth predicates and evaluate whether it can recover them. On five datasets and three statistical models, continuous relaxation and iterative refinement consistently improve performance. Our general method also matches previous specialized method for explainable clustering (53
".e uniformly draw samples from all samples we observe from the dataset

---

5.1 Datasets
We design a suite of datasets for each of the three statistical models mentioned from Section 3.2 Each dataset has set of reference predicates, and we evaluate our algorithm's 's ability to recover them.
Clustering. We consider five datasets, AGNews, DBPedia, NYT, Bills, and Wiki 40 58 32
23
The datasets have 4/14/9/21/15 topic classes each described in a predicate, and we sample 2048 examples from each for evaluation.
Multiclass Classification. We design a classification dataset with 5,000 articles and 20 classes; its goal is evaluate method's s ability to recover the latent interpretable features useful for classification. Therefore, we design each class to be a set of articles that satisfy three predicates about its topic, location, and language; for example, one of the classes can be described by the predicates "has topic of sports", "is in Japan", and "is wrilten in English". We create this dataset by adapting the New York Times Articles dataset [40), where each article is associated with topic and location predicate; we then translate them into Spanish, French, and Deutsch. We consider in total 4 + 4 + = 12 different predicates for each the topic/location//anguugg attributes and subsample 20 classes from all x 4 x = 64 combinations.
Time Series modeling. We synthesize time series problem by further adapting the translated NYT dataset above. We set the total time T = 2048 and sample I1 ... 2T according to the time series model in Section 3.2 to create the benchmark. We set e to be the 12 predicates mentioned above and the weight w for each predicate @k be a cosine function with a period of ''' simulate how each attribute evolves throughout time. In addition, we included three simpler datasets where there is only variation on one attribute (i.e. varies only on one of topic/location/annuaee)) We name these four time series modeling all, topic, locat, and lang, respectively. See Appendix B for a more detailed explanation.
5.2 Metrics
To evaluate algorithm, we match each learned predicate de with reference oi compute the F1-score and surface similarity for each pair, and then report the average across all pairs. To create the matching, we match 6% to the Φf with the highest overlap (number of samples where both are true); formally, define bi bi-partite matching problem to match each predicate in ø with one in ø*, define the weight of matching o and o% to be their overlap, and then find the maximum weight matching via the Hungarian algorithm. We now explain the Fi-score and surface similarity metric.
F1-score Similarity. We compute the Fl-score of using δ(z) predict o* (x) X, the set of samples we observe. This similar to the standard protocol for evaluating cluster quality (28).
Surface Form Similiarity. We can also directly evaluate the similarity between two predicates based on their string values, e.g. "is about sports" is similar in meaning "has a topic of sports", metric previously used by [62]. For pair of predicates, we ask gpt-4 to evaluate whether they are similar meaning, related, or irrelevant, with each option associated with a surface-similarity score of 1/0.5/0. We display the prompt in Figure and and example ratings in Table
5.3 Experiments Our Benchmark
We now use these metrics and datasets to evaluate the optimization algorithm proposed in Section and run ablations investigate whether continuous relaxation and iterative refinement effective. We will first introduce the overall experimental setup, and then discuss individual takeaways supported by experimental results each paragraph.
Experimental Setup. When running the algorithm, we generate candidate predicates Discretize with gpt-3.5 3.9t-3.-tuubb. [37]: to perform the denotation operation [6] (z), use flan- [ we create the embedding for each sample x with the Instructor-22 x1 model 48) and then normalize it with & norm. We set the number of candidates M returned by Discretize to be 5 and the number of optimization iteration S to be 10. To reduce noises due to randomness, we average the performance of five random seeds for each experiment.
Table 2 reportt the results of clustering and Table (epoortss other results. For each dataset, we perform several ablation experiments and present the takeaways from these results.
7

---

FI/Surface
AGNews
DBPedia
NYT
Bills
Wiki
Average
Prompting
0.43/0.60
0.31/0.44
0.21/0.40
0.16/0.47
0.22/0.34
0.27/0.45
No-Refine
0.72/0.57
0.57/0.52
0.54/0.58
0.34/0.49
0.47/0.51
0.53/0.54
No-Relax
0.86/0.60
0.59/0.53
0.58/0.53
0.31/0.51
0.46/0.50
0.56/0.54
Ours
0.86/0.62
0.68/0.54
0.70/0.63
0.45/0.52
0.51/0.53
0.64/0.57
GoalEx (Specialized)
0.86/0.62
0.75/0.64
0.68/0.63
0.33/0.50
0.49/0.48
0.62/0.57
Table 2:
Results clustering. Ours always outperforms No-Refing and No-Relax, indicating that both continuous relaxation and iterative refinement are helpful. Compared to GoalEx [53]. our method is slightly better on all datasets except DBPedia, which we analyze in Table I
FI/Surface
topic
lang
locat
all
time-avg
classification
Prompting
0.40/0.35
0.39/0.38
0.26/0.30
0.54/0.57
0.40/0.40
0.51/0.42
No-Refine
0.53/0.53
0.39/0.50
0.37/0.55
0.58/0.44
0.47/0.50
0.58/0.44
No-Relax 0.65/0.50
0.52/0.65
0.48/0.68
0.61/0.56
0.56/0.60
0.68/0.62
Shuffled 0.46/0.33
0.52/0.45
0.33/0.28
0.60/0.39
0.47/0.35
N/A
Ours
0.67/0.57
0.62/0.70
0.55/0.68
0.72/0.64
0.64/0.65
0.73/0.70
Table Our performance on time series (left) and classification (right). Both continuous relaxation and iterative refinement improve the performance (comparing Ours to No-Refine and No No)Relax).
Is our method better than naïvely prompting language model to generate predi- cates? How does our approach compare naïve baseline approach, which directly prompts the language model to generate predicates based dataset samples? For this bascline, we repeatedly prompt language model to generate more predicates until we obtain K predicates, compute their denotation, evaluate them using the metrics in Section 5.2 and report the performance in Table and and 5. the Prompting row. Across all entries, our approach significantly outperforms this baseline.
Takeaway O:
Takeaway 1: Relax + discretize is better than exploring randomly generated predicates. Our optimization algorithm explores the top-5 HLLM-generate predicates that have the highest correlations with @% Cz. Would choosing random predicate be equally effective? To investigate this question, we experimented with variant of our algorithm that randomly chooses five predicates without utilizing and 3 No-Relax underperforms our full algorithm (Ours) in all cases. Appendix Figure(6) we plot the loss after each iteration averaged across all tasks, and we find that Ours converges much faster than No-Relax.
the continuous representation dh. (No-Relax). In Table 2
Takeaway Iterative refinement improves the performance. We considered a variant of our algorithm that only discretizes the initial continuous representations and does not iteratively refine the predicates (No-Refine). Table and Ref ine underperforms the full algorithm in all cases.
Takeaway 3: Our model-agnostic method is competitive with previous methods specialized for explainable clustering. We compare method to GoalEx from [53), which designs specialized method for explainable clustering based on integer linear programming. Even though our method is model-agnostic, it matches outperforms GoalEx on four out of five datasets and improves F1 by 0.02 on average.
Takeaway 4:
Our method accounts for information beyond the set of text samples (e.g. temporal correlations in the time series). We investigate this claim using the time series datasets, where we shuffle the text order and hence destroy the time-dependent information model could use to extract informative predicates (Shuffled). Table 3 finds that Ours is better than Shuffled in all cases, indicating that our method does make use of temporal correlations.
Appendix D includes additional results: 1) compared to topic modeling and K-means, our method achieves similar or better performance while being explainable; 2) we ran ablations the effect of neural embeddings and show that informative embeddings are crucial to good performance; 3) Takeaways 1, 2, and 4, are significant with p 1% under paired t-tests.
8

---

Ours
Root
Topic Model
Root
format despription image gan ganeration prompt
Gusstr 2: requaioo information or arewars rigarzing a specific taple
Topic promot Dessriptiion delailee
Toplo year game
Cluster A, the user wants requistt graphic desigg prompts for Mijjoumey
Cluster 2.A ask for Nistorical intormation on a speciti topic
Cluster 2.B request technizal essu rescilttion
Topic L::: stle generate post short....
Topio create sky
Topic
Figure 3: Left. We generate taxonomy with sophisticated explanations by recursively applying our clustering model. Right. We cluster with topic models and present the top words for each topic. Although some topics are plausibly related certain applications, they are still ambiguous.
6 Open-Ended Applications
We apply our framework to broad range of applications to show that highly versatile. Our framework can monitor data streams (Section [.1]). apply to the visual domain (Section F.T). and be easily steered to explain specific abstract properties (Section F2).. Across all applications, our framework is able to explain complex concepts that classical methods struggle to produce.
6.1 Running Our Models Out of the Box: Monitoring Complex Data Streams of LLM Usages
We apply our models from Section 3.2 to monitor complex data streams of LLM usages. In particular, we recursively apply our clustering model to taxonomize user queries into application categories, apply our time series model to characterize trends in use cases across time, and apply our classification model to find categories where one LLM is better than the other. Due to space constraints, we present the key results in the main paper and the full results in Appendix G
Taxonomizing User Applications via Clustering. LLMs are general-purpose systems, and users might applyLLMs in ways unanticipated by the developers. the developers can better understand how LhsLLMs are used, they could collect training data correspondingly, ban unforeseen harmful applications, or develop opplication-speeiffi methods. However, the amount of user queries is too large for individual developers to process, so an automatically constructed taxonomy could be helpful.
We recursively apply clustering model to queries to the ChatGPT language model. We obtain the queries by extracting the first turns from the dialogues in WildChat [59], a corpus of 1M real-world user-ChatGPT dialogues. We use go (36] discretize and cl
I to compute denotations. We first generate K = clusters on subset of 2048 queries; then we generate K 4 subclusters for each cluster with 32 samples.
We present part of the taxonomy in Figure (left) and contrast it with the taxonomy constructed by directly applying LDA recursively (right). Although some LDA topics are plausibly related to certain applications, they are still ambiguous; for example, it is unclear what topic "ar prompt description detailed means. After manually inspecting the samples associated with this topic, found that they were related to the application of writing prompts for an image-generee generation model. contrast, framework can explain complicated concepts that are difficult to infer from individual words; for example, it generates "requesting graphic design prompts" for the above application, which is much clearer in its meaning when explained in natural language.
Characterizing Temporal Trends via Time Series Modeling. Understanding temporal trends in user queries can help forecast flu outbreaks [16), prevent self-reinforcing trends [19,, identify new application opportunities. We run our time series model on 1000 queries from WildChat with K 4 to identify temporal trends in user applications, and report part of the results in Figure 4 Based on the blue curve, we find that an increasing number of users "requests writing or content creation... creating stories based given prompts." This helps motivate systems like Coauthor 29 to assist with this use case.
Finding Categories where One Language Model is Better than the Other. One popular method to
evaluatel.L.Me is crowd-soureing: an evaluation platform (e.g. ChatBotArena IT) or a company (c.g. OpenAl) accepts prompts from users, shows users responses from two different LLM systems,

---

Trends on WildChat
Pe request information translation in a different language;
M
the query involves requesting another language translating
01 engage academic or scholarly discussion; L.c. the query involves seeking information or topics related to academic subjects or discussions
Frequency
0.3
Py request writing or content creation in a specific style or genre;
ic. the query involves asking for creating
0.21
stories other written content based given prompts
0s: request general information answers factual questions;
i.c. the query involves seeking answers to broad
0.1
or factual questions
0.0
0.8
Percentile of turns sorted by time
Figure 4: We analyze WildChat queries with our time series model. For each learned predicate, we plot how its frequency evolves and the 99% confidence interval of the average frequency (shaded).
and the users indicate which one they like better. The ranking among the LLM systems then determined by Elo-rating, i.e. how often they win against each other.
However, aggregate Elo-rating omits subtle differences between L.M systems. For example, LLLaa- 3-70B achieved similar rating as Claude-3-Opus, and the LLM community was excited that open-weight models were catching up. However, LLama-3-700 similarly capable across all categories, is significantly better/worse under some categories? Such information is important for downstream developers, since some capabilities are more commercially valuable than others: e.g. programmer usually does not care about LLM's capability to write jokes. We need a more fine-grained comparison.
We directly apply the classification model from our framework to solve this task. To understand the categories where LLama-3- 3-70B is better/worse than Claude-3-Opus, we gather user queries from the ChatBotArena maintainers (personal communication) set y = if the I.Lama---70B response to x preferred and y = otherwise. We set K 3.
Our model finds that LLama-3-700 3-70B is better when the query "asks an open-ended or thought- provoking question" but worse when "presents a technical question" or "contains code snippets". These findings are corroborated by manual analysis by the ChatBotArena maintainers, who also found that Llama-3 better at open-ended and creative tasks while worse technical problems We hope that our model can automatically generate similar analysis in the future when new LLM is released, thus saving researchers' efforts.
To summarize, our framework 1) enables us to define a time series model to explain temporal trends in natural language, and 2) outputs sophisticated explanations that LDA fails generate. However, is far from perfect: it is slow to compute denotations for all pairs of x and candidates Φ since involves many LLM API calls, and the predicates themselves are sometimes redundant. We describe these limitations and potential ways to improve them in Appendix G
Due to space constraints, present applications in explaining visual features to make images memorable to humans and clustering math problems based on subareas in Appendix E. and
Conclusion
In this work, we formalize broad family of models parameterized by natural language predicates. We design a learning algorithm based on continuous relaxation and iterative refinement, both of them effective based on our ablation studies. Finally, we apply our framework to wide range of applications, showing that it highly versatile, practically useful, applicable to both text and vision domains, and explains sophisticated concepts that classical methods struggle to produce. We hope future works can make our method more computationally efficient and apply it to more realistic applications, thus assisting humans to discover and understand complex patterns in the world.
httt:::/lmmyss.orr/bbggg/2222--2 /1mmmma/3//
10

---

Acknowledgement
Ruiqi Zhong designed the conceptual framework, the algorithm, and all the experiments; he also implemented all the experiments and wrote the entire paper. Heng Wang contributed to earlier explorations of the project. Dan Klein provided feedback on the paper draft. Jacob Steinhardt provided feedback throughout the project.
Ruiqi Zhong is supported by Simons Foundation fund, chartstring 71815-13090-44 members from the Berkeley NLP group, Jacob Steinhardt group, Lisa Dunlap, Zihan Wang, Tatsunori Hashimoto, and anonymous reviewers for their feedback on our project and paper draft.
We thank
References
[1] URL hol.hhtps:///ae htttp::/www.anthrppp....om
http://wwaaac.ccc//////////aaaaa son-eet
[2] Rose Aharoni and Yoav Goldberg. Unsupervised domain clusters in pretrained language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7747-7763, Online, July 2020. Association for Computational Linguistics. doi: 10.18653. w1/2020.aclmmain.62. URL httpp::/acclanthollee
org. 2020 aa100acc--a0 acl.mmin................ammmmm mma.n.n2n222 692
[3] Jacob Andreas, Dan Klein, and Sergey Levine. Learning with latent language. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Pupers), pages 2166-2179, New Orleans. Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/ N18- 1197. URL http:ttts//aaaaaatoo//////
orggooo- 1197
[4] Kasturi Bhattacharjee, Rashmi Gangadharaiah, Kathleen McKeown, and Dan Roth. What do users care about? detecting actionable insights from user feedback. In Anastassia Loukina, Rashmi Gangadharaiah, and Bonan Min, editors, Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies: Industry Track, pages 239-246, Hybrid: Seattle, Washington + Online, July 2022. Association for Computational Linguistics. doi: 10. 18663//1/20222mmaclllaa innuuusy-.d URL https://accaantholggs org/ 2022. nww.caaclll-.ocmmm incustrynduttyy.... 27
[5] Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models. URL https://oeeaiipblic blob. core. windows. nethneuron- explainer/paper/indet. (Dttl accessed: 14.05. 2023), 2023.
[6] David M Blei and John D Lafferty. Dynamic topic models. In Proceedings of the 23rd international conference Machine learning, pages 113-120, 2006.
[7] David M Blei, Andrew Y Ng, and Michael Jordan. Latent dirichlet allocation. Journal of machine Learning research, 3(Jan):993-1022. 2003.
[8] David Carmel, Haggai Roitman, and Naama Zwerdling, Enhancing cluster labeling using wikipedia. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 139-146, 2009.
[9] Jonathan Chang, Sean Gerrish, Chong Wang, Jordan Boyd-Graber, and David Blei. Reading tea leaves: How humans interpret topic models. Advances in neural information processing systems, 22, 2009.
[10] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et Evaluating large language models trained on code. arXiv preprint arXiv:2107.0333 2021.
[11] Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E Gonzalez, et al. Chatbot arena: An open platform for evaluating llms by human preference, arXiv preprint arXiv:2403.041:3 04132, 2024.
11

---

[12] Mia Chiquier, Utkarsh Mall, and Carl Vondrick. Evolving interpretable visual classifiers with large language models. arXiv preprint arXiv:2404.0994/ 2024.
[13] Hyung Won Chung. Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang. Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-fing finettnnd language models. arXiv preprint arXiv:2210. 1416, 2022.
[14] Mingkai Deng, Jianyu Wang. Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. RLPrompt: Optimizing discrete text prompts with rein- forcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3369-339,, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10. 1/20:/.1122222..mmmmmaal https://aclanthhologs org/
URL
002/2022. 2022. mmmlp-main.22
[15] Lisa Dunlap, Yuhui Zhang, Xiaohan Wang, Ruiqi Zhong, Trevor Darrell, Jacob Steinhardt, Joseph E Gonzalez, and Serena Yeung-Levy, Describing differences in image sets with natural language. arXiv preprint arXiv:2312.02974. 2023.
[16] Jeremy Ginsberg, Matthew H Mohebbi, Rajan S Patel, Lynnette Brammer, Mark S Smolinski, and Larry Brilliant. Detecting influenza epidemics using search engine query data. Nature, 457 (7232):1012-1011 1014, 2009.
[17] Thomas L Griffiths and Mark Steyvers. Finding scientific topics. Proceedings of the National academy of Sciences, 101 (supp___)):52
2004.
[18] Maarten Grootendorst. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXivv2203.05774, 2022.
[19] Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning, pages 1929- 1938. PMLR, 2018.
[20] Md Mahadi Hassan, Alex Knipper, and Shubhra Kanti Karmaker Santu. Chatgpt as your personal data scientist. arXiv preprint arXiv:2305. 13657, 2023.
[21] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint
2021.
[22] Or Honovich, Uri Shaham, Samuel R. Bowman, and Omer Levy. Instruction induction: From few examples natural language task descriptions. In Annual Meeting of the Association for Computational Linguistics, 2022. URL https://t://ppt semanticscholest org/ Corpus 248986755
[23] Alexander Miserlis Hoyle, Pranav Goel, Rupak Sarkar, and Philip Resnik. Are neural topic models broken? In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Findings of the Association for Computational Linguisties: EMNLP 2022, pages 5321-5344, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/
1//202..inndins-eo centiingseemll..300.. 390.. URL (RLLhttps:://
htttputt//ttt///aaaaaaatetoe hadtts://aclanthologgy .rg.//ggooor..oooooooooo 22gg/2222.222 findings sindings--enllppp
390
[24] Phillip Isola, Devi Parikh, Antonio Torralba, and Aude Oliva. Understanding the intrinsic memorability of images. Advances neural information processing systems, 24, 2011.
[25] Siddharth Karamcheti, Megha Srivastava, Percy Liang, and Dorsa Sadigh. Lila: Language- informed latent actions. In Conference on Robot Learning, pages 1379-1390 PMLR, 2022.
[26] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference on machine learning, pages 13338-3348.. PMLR, 2020.
[27] Michelle S Lam, Janice Teoh, James Landay, Jeffrey Heer, and Michael S Bernstein. Concept induction: Analyzing unstructured text with high-level concepts using lloom, arXiv preprint arXiv:2404112259 2024.
12

---

[28] Tilman Lange, Volker Roth, Mikio L Braun, and Joachim M Buhmann. Stability-basee validation of clustering solutions. Neural computation, 16(6):1299-13222) 1323, 2004.
[29] Mina Lee, Percy Liang, and Qian Yang. Coauthor: Designing human-ai collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI conference fantors
on human factors in computing systems, pages 1-19, 2022.
[30] Josh Magnus Ludan, Qing Lyu, Yue Yang. Liam Dugan, Mark Yatskar, and Chris Callison- Burch. seeepretableeby-desiiee text classification with iteratively generated concept bottleneck. arXiv preprint arXiv:2310. 19660, 2023.
[31] Pingchuan Ma, Rui Ding, Shuai Wang, Shi Han, and Dongmei Zhang. Demonstration of insight- pilot: An llm-empowered automated data exploration system. arXiv preprint arXiv:2304000477, 2023.
[32] Stephen Merity, Nitish Shirish Keskar, and Richard Socher. Regularizing and optimizing LSTM language models. In International Conference Learning Representations, 2018. URL htttp:://pperee
hett/forumiid-SSyPPPP
[33] Jesse Mu, Percy Liang, and Noah Goodman. Shaping visual representations with language for few-shot classification. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4823-4830, Online, July 2020. Association for Computational Linguistics. doi: 18.18653/v1I 1/ 2020.acl-main 1a0.aacl-main.366 URL wttht:tt///// aalaaaanaanntoooooooooo org/ http:/222..c01mmamama... www.aaccl......... 436
[34] Dong Nguyen, Maria Liakata, Simon DeDeo, Jacob Eisenstein, David Mimno, Rebekah Tromble, and Jane Winters. How we do things with words: Analyzing text social and cultural data. Frontiers in Artificial Intelligence, 3:62, 2020.
[35] Pontus Nordberg, Joakim Kävrestad, and Marcus Nohlberg. Automatic detection of fake news. In 6th International Workshop Socio Tecc--eechiica Perspective in IS Development, virtual conference in Grenoble, France, June 8-9, 2020, pages 168-179. CEUR-WS, 2020.
[36] OpenAL. Gpt-4. 2024-05-14.
2023. Accessed:
2023.
[37] OpenAL. Gpt-3.5 turbo. httee://paattoor...ppaa...cccc/p Accessed: 2024-05-14.
2024.
[38] Chau Minh Pham, Alexander Hoyle, Simeng Sun, and Mohit lyyer. Topicgpt: A prompt-based topic modeling framework. arXiv preprint arXiv:2311.0144)) 2023.
[39] Linlu Qiu, Liwei Jiang, Ximing Lu, Melanie Sclar, Valentina Pyatkin, Chandra Bhagavatula, Bailin Wang, Yoon Kim, Yejin Choi, Nouha Dziri, and Xiang Ren. Phenomenal yet puzzling: Testing inductive reasoning capabilities of language models with hypothesis refinement. In The Twelfth International Conference on Learning Representations, 2024. URL https:// openreview net oowoood orwm?id-bbt777aj222
[40] Evan Sandhaus. The new york times annotated corpus. Linguistic Data Consortium, Philadel- phia, 6(12).22675,, 2008.
[41] Simon Schrodi, Julian Schur, Max Argus, and Thomas Brox. Concept bottleneck models without predetined concepts. arXiv preprint arXiv:2407.03921, 2024.
[42] Pratyusha Sharma, Antonio Torralba, and Jacob Andreas. Skill induction and planning with latent language. In Smaranda Muresan. Preslav Nakov, and Aline Villavicencio, ed- itors, Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Pupers), pages 1713-1726, Dublin, Ireland, May 2022. Associ- ation for Computational Linguistics. doi: 10. 18653/v1/20222acl-looo 120. URL https
//aclanthology.org .rg. 202g./2222..c2 a0w.://o22.ccllloog acl- 120
[43] Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. Auto- prompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXi::2010.15980 2020.
13

---

[44] Chandan Singh, John X Morris, Jyoti Aneja, Alexander M Rush, and Jianfeng Gao. Explain- ing patterns in data with language models via interpretable autoprompting. arXiv preprint urXiv:2210001860 2022.
[45] Chandan Singh, Aliyah R Hsu, Richard Antonello, Shailee Jain, Alexander G Huth, Bin Yu, and Jianfeng Gao. Explaining black box text modules natural language with language models. arXiv preprini (tf:::33050086633 2023.
[46] Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. Re- thinking interpretability in the era of large language models. arXiv preprint arXiv:2402.01761, 2024.
[47] Christopher T Small, Ivan Vendrov, Esin Durmus, Hadjar Homaei, Elizabeth Barry, Julien Cornebise, Ted Suzman, Deep Ganguli, and Colin Megill. Opportunities and risks of Ilms for scalable deliberation with polis. arXiv preprint arXiv:2306. 1932, 2023.
[48] Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. One embedder, any task: instruction-finttueed text embeddings. 2022. URL /attps://arte wwwpvvaa/vvvv.aa///ai//aaaaaaava org/abs/ /2tp./221.... 221129922111111
[49] Pucktada Treeratpituk and Jamie Callan. Automatically labeling hierarchical clusters. In Proceedings of the 2006 international conference on Digital government research, pages 167- 176, 2006.
[50] Vijay Viswanathan, Kiril Gashteovski, Carolin Lawrence, Tongshuang Wu, and Graham Neubig. Large language models enable few-shot clustering. arXiv preprint arXiv:2307.00524 2023.
[51] Sida I Wang and Christopher D) Manning. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics ( (Volume 2: Short Papers), pages 90-99999990 94, 2012.
[52] Zihan Wang, Jingho Shang, and Ruiqi Zhong. Goal-driven explainable clustering via language deseriptions. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10626-10649, Singapore, December 2023. Association for Computational Linguistics. doi: 10.11863//1/2023. emnip-main.657. URL
orgpey..rg///// 2023. emn.p-main..5...
[53] Zihan Wang, Jingho Shang, and Ruiqi Zhong. Goal-driven explainable clustering via language descriptions. arXiv preprint arXiv:2305.1 13749, 2023.
[54] Lionel Wong, Jiayuan Mao, Pratyusha Sharma, Zachary S Siegel, Jiahai Feng, Noa Korneev, Joshua B Tenenbaum, and Jacob Andreas. Learning adaptive planning representations with natural language guidance. arXiv preprint arXiv:2312.08566, 2023.
[55] Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison- and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19187-19197, 2023.
[56] Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and Erik Cambria. Large language models for automated open-domain scientific hypotheses discovery. arXiv preprint arXiv:2309.022226 2023.
[57] Chao Zhang, Fangbo Tao, Xiusi Chen, Jiaming Shen, Meng Jiang, Brian Sadler, Michelle Vanni, and Jiawei Han. Taxogen: Unsupervised topic taxonomy construction by adaptive term embedding and clustering. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2701-2709, 2018
[58] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.
[59] Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: Im chatGPT interaction logs in the wild, In The Twelfih International Conference on Learning Representations, 2024. URL /tttps://optt
14

---

[60] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang. Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, al. Judging 1lm-ass -s---ue- with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024.
[61] Ruiqi Zhong, Charlie Snell, Dan Klein, and Jacob Steinhardt. Describing differences between text distributions with natural language. In International Conference on Machine Learning. pages 27099- 20199-27120 27116. PMLR, 2022.
[62] Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, and Jacob Steinhardt. Goal driven discovery of distributional differences via language descriptions. arXiv preprint WiXi::3302.14333 2023.
[63] Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, and Jacob Steinhardt. Goal driven discovery of distributional differences via language descriptions. Advances in Neural Information Processing Systems, 36, 2024.
[64] Yongchao Zhou, Andrei loan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910, 2022.
15

---

A More Related Work
LLM for Exploratory Analysis. Due to its code generation capability [10], large language models have been used to automatically generate programs analyze a dataset and generate reports from them [T] [00]. In comparison, our work focuses on generating natural language parameters to extract real-valued features from structured data.
Many prior works optimized discrete prompts to improve the predictive performance [4 [4], and some recent works demonstrated that L.MSs can optimize [6]. comparison, we focus on optimizing discrete predicates to explain patterns rather than improve task performance.
Discrete Prompt Optimization.
prompts to reach otate-of-the-art accuracy 66
Learning with Latent Language. first proposed learn in a hypothesis space of natural language strings to improve generalization, and later works in this area have focused on using natural language
guide the learning process improve downstream task performance 33 our work focuses on explaining datasets with natural language, rather than improving downstream task performance.
25
4 54
In contrast,
B Time Series Dataset
To sample texts from the All time series problem, we sample from the time series model de- scribed Section 3.2 we set ø to be all the 12 predicates, sort them first by attributes (e.g. topie/location/lanuuage then alphabets, and set the weight for the 1thh predicate to be sin function with period and evenly spaced phases, i.c.
t T
Wke sin (2T
(11)
K
As result, the weight for each predicate has evenly spaced phases and would peak at different time period.
C Surface form similarity prompt
We include our prompt used to evaluate the surface form similarity hetween the predicted predicate Se and the reference predicate of in Figure
Similarity
Is text_a a and text_ b similar in meaning? respond with yes, related,
Target:
Φ.
text a: has a topic of sports
text b: about sports
LM's
output: yes
generation
Figure 5: The prompt template used to evaluate the surface form similarity between the predicted predicate dy and the reference predicate o7.
D Additional Results on Our Benchmark
Our method is similar or better than classical methods such as topic modeling or K-means. We report the performance of K-means clustering and topic modeling under the clustering benchmark
16

---

FI/Surface
AGNews
DBPedia
NYT
Bills
Wiki
Average
OneHot
0.87/0.53
0,54/0.51
0.48/0.53
0.26/0.51
0.36/0.47
0.50/0.51
OtherEmb
0.85/0.70
0.62/0.54
0.59/0.53
0.43/0.59
0.48/0.53
0.60/0.59
Ours
0.86/0.62
0.68/0.54
0.70/0.63
0,45/0.52
0,51/0.53
0.64/0.57
K- K-means 0,83/
0.75/
0.72/
0.417
0.53/
0.657
TopicModel
0,56/-
0.52/
0.49/
0,25/
0.35/
0.43/-
Table 4: We compare method classical clustering approaches that do not generate natural language explanations (K (K-means and Topi cModel), where "-" means that the surface form metric undefined since these methods do not output natural language explanations. We find that on average. our method is close to K-means and significantly outperforms Topi cModel under the FI similarity metric, while generating natural language explanations for each cluster. We also compare our method to using one-hot text embedding, and find that our method is significantly better; this indicates that the use of informative text embedding is crucial to performance.
F1/Surface topic
lang
locat
all
time-avg
classification
One-hot
0.63/0.55
0.51/0.57
0.51/0.62
0.66/0.60
0.58/0.59
0.72/0.68
OtherEmb
0.68/0.58
0.56/0.59
0.49/0.68
0.71/0.68
0.61/0.63
0.73/0.67
Ours
0.67/0.57
0.62/0.70
0.55/0.68
0.72/0.64
0.64/0.65
0.73/0.70
Table 5: Our method consistently outperforms a variant that uses one-hot text encoding as ez rather than neural embeddings. This indicates that using informative text embedding is crucial to performance.
Table 4 on average, our method close to K-means and significantly outperforms Topi cModel under the FI similarity metric, while generating natural language explanations for each cluster.
Takeaway 5: Using informative text embedding crucial to performance. We used neural embeddings when optimizing the continuous representation of the predicates. Does our algorithm actually make use of the information in the feature embeddings? To investigate this question, we ran an ablation of using one-hot text embeddings rather than neural embeddings (OneHot), which do not encode any information about the similarity between text samples. We report the performance in Table 4 and 5 across all settings, using neural embeddings consistently outperforms OneHot.
To make that this takeaway is general and specific embedding model, method with another text embedding model, aal--ppet--base--e and report the performance as
Speed of optimization
6.1
Curs
No-relax
6.0
5.9
5.8
5.7
5.6
8
Iteration
Figure 6: We plot how the loss decreases across different iterations with and without relaxation (that explores using random predicates). We find that using relaxation significantly speeds up optimization.
17

---

the OtherEmb row. We find that using this neural embedding also outperforms OneHot in most cases, indicating that our conclusion is robust.
Takeaway 1,2,and 4 are statistically significant. To compare the performance between our method and each variant, we conduct a one-sided paired t-test on their performance (F1-similarity) on each dataset, where the performance on each dataset is the averaged performance across five runs. Takeaway 1, 2, 4 has a p-value of 5 x 10 2 10 and 6 10-3, respectively.
E Detecting Self-Reinforcing Trends in Machine Learning System
Machine learning systems sometimes have unintended side effects and reinforcement themselves. [9] illustrated an example failure mode, where group of users is discriminated against and thus leave a platform, causing a ML system to discriminate them further and hence drive them away.
As concrete illustration, let us imagine social platform Y where users post tweets and the platform will display the most engaging ones; suppose there are two groups of users, one conservative and one liberal, where both groups prefer more engaging tweets but also tweets that agree with their political stances. Y implements a recommender system, which trains a classifier to predict whether a tweet is likely be preferred by random user, and then the platform Y will promote these tweets. If the two groups of users are balanced, the optimal classifier will make Y promote tweets that are engaging and place little weights on the political slant.
However, if there are fewer liberal users, the classifier will be biased and Y will promote conservative tweets more and focus less on whether the tweet is engaging or not. The liberal users will find the promoted tweets less attractive, thus leaving the platform Y. As a result, fewer liberal users will stick to Y, thus making the classifier more biased.
Now we provide a proof-of-concept experiment to illustrate how our time series model can be applicd to detect such reinforcing trend. We first simulate the setup above and obtain the tweets promoted by platform Y across time, and then apply our time series model to extract temporal trends from these tweets. Suppose there two groups of users, liberal and conservative. At t = 0, the fraction of liberal users is Ao = 0.5 and is the same as that of conservative users. To simulate the setup above and obtain the tweets promoted by platform Y across time, we assume that at each time step t, will sample 2,000 tweets, where each tweet is a 2D datapoint with the æ-value a random integer from [-1, 1] indicating whether it is liberal, non-political, or conservative, and y-value a random integer from [-2, 2] indicating how engaging the tweet is. For each tweet, we obtain a label of y = 1/0 if the user likes tweet, and the user's probability for liking tweet is defined by our 0.5g), where = 1 if the user is conservative and 0 otherwise. We then train a logistic regression classifier to predict whether a random user will like tweet and the platform Y will promote the tweets with the top 20% score. Let the fraction of tweets non-liberal tweets be a and non-conservative tweets be b, then the fraction of liberal users for the next round will be determined by:
bE
Ae+1 b.d. + ar >x))
(12)
which models how the group size will increase/decrease depending on whether the platform promotes tweets that agree with their views. We run this process for T = 10 and gather all the 2D datapoints promoted by platform Y.
We then turn these two-dimensional datapoints into text samples x. We ask the gpt-4o to write a liberal, non-political, or conservative tweet based on the z-value; then we ask gpt-4o to make it more/less engaging based on the y-value. For example, for a 2D value of (-1, 2). we ask gpt-4o to write a liberal tweet and ask it to make it more engaging two times; if the value is (1, -2), we ask got-4o to write a conservative tweet and then ask it to make it less engaging two times.
We now have a list of tweets across time, and we directly apply our time series model with K = 3 to extract trends from them. Our time series model find that there an increasing amount of tweets that "expresses patriolic sentiments" and "champions specific policies", but decreasing amount "poses question to engage the audience". These predicates exactly recover all the underlying trends, that the self- reffrreinorring effect make the tweets more conservative, less nol-toli,,al, and less engaging.
18

---

Learned Predicates
Weights
portrays sense tranquiityy e.g. image captures a serene sureet over a calm lake, with soft orange and pink hues the sky"
-0.64
Class Not Memorable
uses mix of colors and textures; for example, the cluster of dark blue beries nestled among vibrant red branches
+0.19
Class 1 Memorable
"highlights specific emations expressions; example, they hold an old-atyle fllp phone."
XOX
expression
+0.43
Figure 7: We apply our classification model from Section 3.2 to explain what visual features make images more memorable [24]. Consistent with previous findings, find that tranquil scenes make an image less memorable, while emotions and expressions are more memorable.
Ours
Classical Method
1. involves algebraic manipulation:
asy. draw, axis, operatorname, tabular Unclear what this mears
2. involves probability combinatorics
divisors, probability, many, letters, unique Mayoe combinatirics
3. requires geametric reasoning;
decimal, det, compute, evaluate, power - Probatsy algebra
pertains number theory; involves calculus or limits;
solutions, roots, polynomial, solution, minimum
Anothar algebra cluster?
5.
5. hyperbola, corresponds, proportional, vertices, points Maybe goometry
Directly explainable
Vague
21 and compare our method (left) to a classical method (right), which first clusters via K-means and then explains each cluster via unigram analysis. Our method directly explains complex concepts, while the classical method delivers vague explanations.
Figure 8:
We cluster the MATH dataset
F
More Applications
F.1 Applying Our Classification Model to Images: Explaining Memorable Visual Features
Our framework is applicable to the vision domain since natural language predicate á can extract binary values from an image æ. For example, for the rightmost image Figure the predicate "portrays a person" evaluates to 1, i.e. [x) = 1, while "contains texts" evaluates to O.
We present application of classification model from Section to images, which learns linear weights over set of visual features described by natural language predicates. This model has also appeared in prior works: our model is equivalent the language-based concept bottleneck model proposed by (55] 41; additionally, when = and C = 2, our model equivalent to the VisDiff framework 15 which finds single predicate discriminate samples from two classes of images.
We apply our classification model to the LaMem dataset [24] to understand what visual features make an image more memorable, interesting cognitive science question. We define the samples a; and their class labels yi to run our classification model. In LaMem, each image associated with score of how memorable it is measured by whether humans can remember seeing it in the past; to make implementation easier, we set x; to be the caption of the image and y:: = if x; has an above median score and y, 0 otherwise. To fit our classification model, set K 6, use gpt-4o the discretizer, and use cpt-4o-mine to compute denotation.
We present three learned predicates in Figure We find that an image is less memorable if it "portrays sense of tranquility; e.g. the image captures a serene sunset over a calm lake, with soft orange and pink hues in the sky ...", and more likely to be memorable if it highlights specific emotions or expressions; for example, the child has a curious expression... ". These results are consistent with the previous manual analysis from [24, suggesting the validity of our results.
F.2 Explaining Abstract Properties via Easy Steering: Clustering Problems Based on Subarea
Can our framework explain more abstract aspects sample æ: e.g. subarea, the type knowledge required to solve a math problem x? We show this is feasible by applying our model from Section 3.2 to cluster math problems and steering to focus on explaining subareas. Meanwhile, classical methods struggle explain abstract aspects.
19

---

We apply our clustering model from Section 3.2 to cluster the MATH dataset [2]] based on subareas. The MATH dataset contains five labeled subaread and hope our model can recover all of them: Algebra, counting. and_ probability, geometry, number_ theory, and precalculus. To steer our clustering model to explain subareas, we simply prompt the discretizer L.LM "/ want to cluster these math problems based on the type of skills required to solve them." " We set K 5, using gpt-4o to discretize and gpt-4o-ming to compute denotation.
We present the outputs of our model on the left of Figure g With simple prompting, our model successfully steered cluster based on subareas and recovers all five labeled subareas from the MATH dataset. Note that our explanations can explain abstract properties that have no word overlap with the samples that match them: for example, the math problems that "requires geometric reasoning" (Figure left 3) usually contain neither of the word "geometric" or "reasoning".
We compare method to classical baseline that first clusters the samples and then explains each cluster with representative words. this baseline, we first perform K-means clustering on the neural embeddings of and assign each sample to a cluster; we then extract representative words by first running a unigram regression to predict whether a sample belongs to the cluster and then selecting words with the most positive weights. We present the word-based explanations on the right in Figure 8 Overall, significant guesswork is needed to interpret the meaning of each word-based cluster (e.g., it is unclear what cluster 1 represents Figure 8 right), while the predicates generated by our algorithm are directly explainable. Our framework can be steered to explain more abstract aspects of sample and significantly improve over classical methods.
G Implementation Details on Open-Ended Applications
To obtain the best outputs, we discretize with gpt-4o and compute denotations with -laude33.5-somm Since aim to analyze user queries, we explicitly prompt gpt-4o to generate detailed predicates about use cases when discretizing continuous predicates. See for the full prompt.
G.1 Taxonomizing User Applications.
Implementation Details We cluster 1024 dialogue with 6 and S We only cluster on a small set of dialogue turns because it is slow to compute denotations: 1) we in total explored around 100 predicates and this amounts to 100 x 1024 = 100K LLM API calls, and 2) we used language model API (claude-3, 3. rather than local small language model (google 1 xx11), to compute denotations, since this is the cheapest model that we feel confident that can handle more sophisticated predicates.
Full Results. We present the full results I0 Overall, we find that our framework can generate sophisticated explanations that classical methods cannot generate. However, some cluster descriptions are significantly overlapping (e.g., category 0, 1, and O.D); additionally, some sub-clusters are not indeed subsets of their parent categories (e.g., subcategory 2.D does not belong to category 2). Future work can improve the taxonomy by 1) deduplicating semantically similar descriptions or more heavily penalizing cluster overlaps, and 2) steer the predicate generation process so that the descriptions for the subclusters are indeed subsets of their parent descriptions.
G.2 Characterizing Temporal Trends.
Implementation Detail. We run our time series model on 1K dialogue turns with K = 4 and the number of iterations S to be 10 identify temporal trends in user applications. We obtain the smoothed frequency curve by updating with the follow equation:
100
1
ft 0.99 ft-1 (x0]) fo
Σ [oN](Ir))
(13)
100
We obtain the shaded area in Figure by shuffling x, and find the highest and lowest values across 100 random runs.
after merging similar categories that differ in levels of difficulty
20

---

Here is a corpus of user queries each associated with score. The queries are sorted from the lowest to the highest score.
(samples_in_ sGanples_in_oromttt..oooo
I an nachine learning researcher that builds chat bots. Here is a list of first turns of user quaries, want cluster based their apslications. Note for exemble 'refers to pop culture' is not an application, but 'wants to ask for information about a po wents
only interested applications:
culture entity'
application.
description should
with
understand what kind of queries acnieve higner score, please suggest descriptions about the queries that are more likely to achieve higher scores.
(nun frun_descriptions_uu pestions_pt_ppooptt descriptians, in line, starting with "=" nd surrounded by quotes ",, Each of then needs to be a predicate about quezy followed by an exclanatio and an example that satisfies the predicate, for example:
Please suggest
user wants to raquast for amail massage conpesition; spacifically, the query involves a reques for drafting an email or nessage with specific content and intent. For example, 'writa an email to ter cosperation....
11e with propesal
"the user wants to request technical code/szript writing; specifically, the query demands the creation script
rodatictiion
For example, 'create dockerfile based this script.
your axample. (Nota that tha examples might nat be goal related, and your response should be both formatted correct as abave and related to tha goal
not output anything else. Please do mention
Plesse generate the response based the given datapoints as much possible. We want the descristions to be relstively objective and can be validated essily, e.g. "is surprising" neans different things for different people, so want to avoid such dascriptions. It should also be predicate on singla quer examale, instead saying. *uses more polite languag e", the generation should be "uses polite language*. Semztines KeyInfo is pravided to h:lp you make come unavailable).
y (rather than statement about comparisen);
up with better responses (though night
Again, here's
goal.
I an a naching learning researcher that builds chat bots. Here is a list of first turns of user quaries, cluster then based their apalications. Note that only interested applications: for exumple 'refers to pop culture' is not an application, but 'wants to ask for infermation about a po culture entity' is application. Each description should with 'the wants
and
Your responses are:
Figure 9: A discretizer prompt that explicitly asks LLM to explain applications. E.g., at the end of the prompt, we explicitly requested the predicates to start with "the user wants to...".
G.3 Advantages and Limitations
Overall, we find that our framework allows us to define sophisticated models (e.g. time series) and can output highly sophisticated predicates, which can include detailed explanations and examples. Therefore, when implemented perfectly, its utility has much higher upperbound than classical methods such as n-gram Bayes/regression or topic models.
However, the comparison between our method and classical methods is only qualitative: we only eye- balled the outputs from our method and the classical methods in Section and did not quantitatively measure how useful they are in practice. Therefore, even if our method does outperform classical methods such as topic model on our benchmark (Table it might not directly translate how useful it is in real-world applications. Additionally, we did not compare to modern taxonomy construction method such as (57), which involves a of task-specific engineering; our method is model-agnostic and was applied out-of-the-box to construct the taxonomy. Section 6 only shows that our method can generate more sophisticated natural language explanations, which presents a higher upperbound of what method could potentially achieve.
In terms of the weakness of our method, our method is currently slow, as its performance highly depends on LLMs to compute denotations correctly, it outputs semantically similar predicates that add little information, and it is hard control the predicates satisfy certain properties (e.g. being subset of parent category). We look forward to future works that can address these problems and realize the full potential of this framework. For example, to remove similar predicates, one could prompt language model to check the pairwise surface similarity between two predicates; to speed up inference, one can distill smaller but much more efficient model specialized for computing denotations.
21

---

user wants iied beckground, for cxerple, 'Wiite a very long, schezent, clabocete, descrietive and detailes aczeemslay....
descristive screenplay writing: specifically, the aska for the genazation of scruunplays including dislogur and deta
request narrative writing "mrits detailed, ne feem the first pecsen perspretive.
rivst-serton persecttive speriicaaly, the entr aske sarrated by Calesti insginative and
one and thoughts free the vicnssist
user ctare. eccule, Neell request role-nlaring sperifically, warr involves enpating in natrative relenlayino sostario gretefined froe
dest
provide
for Cirnentine I'll take rale of
uis...!
sole, sgrite user vants to request dialogue writing: sprcitioall, the every faoures on generating speciric cowerrations beteeen chorecters, for oxe the arteated "Jane", shere
als Jame, Jine's
girifriand Jane's t6
d neighnoue Ascon king and Aazon's 12 year ole sister Holly Boss Line hanging out at school..."
user vants to For exaple, 10an reurite 19791e
content rentiting; asecificelly. the every siks altering er reisapining existing content
nee context. Karpar?
Talee Crypt
gate the
write the
user assistance writing actting text; apecifically, the inuslves tasks like composing positive feedbook or reariting contant. exeepln, "Plesse re-arite this as positive fescback for Stelins, praizing her.
anple, "write user vants cceate fictionsi content; specifirally
quesy imvovees gerezating ariginel stariss, scripts, or chazactsrs. far old pirifrien
Jare, Jare's
Jane's
and neighosur Aaron Ling hanging out at ackool shet Jane and San rinds cut Airon his a crust."
1.5 user vants request dialogue writing fistiamal charecters; aperifically, the query zequests the crestion of conversation betaren "Jase"...".
harestors
scenarie,
1.C the user vante to requect stort recsence of reply compoxition; specifically, the query invsives crafting conciss eessage to sand in di acuasion. For exmple. "give cexporue emiiiase Yas, I'm deing will. about yaut". content
invsives insroving language, structure, adding references to estating test. For exeplle, "nunaarize this parte for notted section in in article and reurite it to teprove and atd cone refarances'.
ssecifitally,
2 the user wants requect feechack or ovalustion; spectfically, the query invelves aoking for re-eriting or giving feedback on a place of text, For esa apla, 12lasse za-arite this a positive feecback Stalins, praising her."
sprcifically,
involves crofting
and direct
ly for given
fiscussion, for oxeelle 'give respince to atsolutely. Conpession, inderstanding, and respect...
2.5 user
to request grammer and sselling correction; speciiically query invelses correcting grasestical and spelling arrars i in acatenio rowrite of text) specifioally, the invoives rephresing content to nest scstesd stardards esp 8 [taxt in (hinase)."
2.0 the user vants
recatons. for excpple, "8 T H..
involves condersing of sumeary. for example, "In point fore, simply sumnarizs key insights on botriers to porticipate in procurement from the selse text at ath-grade read fng level and proviie beter supporting quites with in neterre..
specifically,
into
bri
the user
request testntca) cade/sorist writing: sperifically, the query denands the creation or andification of a script or cote. For example,
vants to request date aanipuistion er computation: specifically, the vurry inrolves operations like celculations, data precassing, or data transfomation for exmple, The to get pezcentage nashealth to nealth in jaxe'
2.
cebuazing assistance; specifically, query invoives aaking help identify iven code. for crample, "N 4 a 88 it i
iasues
3.0 the user vants to create or othance ui/ux companents withis web develapeent; specificeall, the avety requires designing, styling, er insrovi
code/scrist serfor particular task: specifically, the query aska for writing code from scratch to othieve specific furetionaliyy. For exeppee, terite a code for des Exeryption free scrotch in sytton'.
2.0
vants
the creation
4 the user vonts to fornst diseciption of an image gentration pronpt: specifically, the query invelves stractuzing a prorpt for use with image genzroti aistoumey create Az-genezated content based language and lncaliration; specitically. the query invoives creating prospts in diffe gent languages shile mainteining specific design instructions, for examlle, 'nagite you're in expest ceushic Designes and have expezience in W t t-ahiz printing and
toels. for
farmlla..."
prospt expert Kidjorrny creative content gateration; specitically, the query invelves asking for astenaive and inaginative descriation oz stories about factiorul craractses or scanacias. for cruesie, "ezite a deseiled tusion scenazio of two snis chasecters.
4.0
user wants resuest graphic design pranpts Midjouney All specifitally, the invalves generating srongts for creating spectri tyoes of imaget. scsepla Genocative pesnpt sritez......
Graphic
have
BB t-shiet printing also
Hidjoumny
user vants to request cospazison or faslan different art styles chazacters; srecifirell. the quety inveives nerping and
ylee chsractors and his guards. past 11.
for nxaaale
and Bragon tall: doks
Freedos planat with whit, 70to
5 the user wasts to request information or antmers regarsing a specific topls; spacifically, the query involves seeking knzuledge or clarification, for e xample,
arms osen source oserating systems.
for historica) Information a fros the past. for asarple, Can you give es cons sumerized info on sugar care 100 years ago?
spesific tople: specifically, the qiery steks details asout
practions, Sten
5.5 usez vants ask for definitions er distinctions: sparificully. polytules?'
eusry sesks to clazify the meening differences beteren tarms. to
example,
oilygmy, difference sclantific or factial informatio; specifically, the query is casking sccurate cata or asplanations relevant to aca
polygaey,
f.c the user vants, to dhrals denic scientific fisids. for example. ddescriee cresture scientific rame is felis schlsncheers evolved from ferel creatures of felis catus, includin species
6.D the user vonte to requect cecentcal teme resalutionn; specifically, the query involves traablesttotigg or colving problem with sefrware or hardears. For exampla, hhww do make circle CoS that sits on too other alements and spoliss gray scale filter to all of the alements interase ting its
Figure 10: The full taxonomy that our algorithm generates to categorize user applications from a corpus of user chatbot queries.
H A Formal Description of Our Algorithm
formal description of our algorithm can be seen in Algorithm H
I Additional Details of Our paper
I.1 Limitations of Our Framework and Our Experiments
our current system is slow, as its performance highly depends on the LLMs to compute denotations correctly, it outputs semantically similar predicates that add little information, and it is hard to control the predicates to satisfy certain properties (e.g. being a subset of parent category). Our experiments are limited since it assumes that the datasets and statistical models we used are reflective of real world application. We made our best effort to gather text clustering datasets that are commonly used in the literature (e.g. from 52 38) and defined models that are plausibly useful for practitioners. Additionally, note that our evaluation on topic clustering is more comprchensive than the prior work (52) by including two new datasets (Bills and Wiki);
As mentioned in Appendix
22

---

Algorithm A formal description of our algorithm. Argument: is the number of steps we use to run our algorithm. Output: var_@1.... the list of K predicates that we maintain, optimize, and return at the end of the algorithm. v are other parameters.
We first optimize the relaxed continuous predicates and discretize them (Line B and then iteratively refines the predicates (Line 2 During iterative refinement, we first find the least useful predicate (Line 12), then we only optimize the continuous representation of the least useful predicate while fixing other discrete predicates (Line 14 17 finally we discretize the hthh predicate
(Line [8
1: Arguments: S
2: Output: var_ var_ø1...,; à
3: $1...
randomly sample K embeddings er to initialize ő
4: for t = 1 to 10
5: t < Optt((i....k)
6: @1...K K -ptRellxeePPhi(ii)
7: end for
8: for k == 1 K do
9: var_Ø; + Discretize(o:]0]
end for 11: for arrmma,.FFinnesst Fitness(vaarl o- [ow] = 0)
to S do k argmax
13: Φx + randomly sample an embeddinge,
14: for
to 10 do
15:
<- Opt δk)
16:
Φk + OptRelaxedPhiivar_oog ú)
17:
end for
18:
C₁ c Discretize((e)
var_Øı t argmax Fitneraasssiiiiisssesiee Fitness(yaa_ = )¹)
t O-OptWvvar_g)
21: end for return var
additionally, we used the exact same hyper-parameter across all clustering tasks, while 52 changed the srpeepprrameters for different datasets.
1,2 Cost of the Experiments
All of the experiments ran in Section 5 are estimed cost most 200 GPU hours on an A100 GPU with 40GB memory, and cost less than $20 of API credit for gpt-3. t-rbr.. The experiments in Section 6 costs most $50 of API inference credit, but we are constrained by rate limit.
1.3
Licenses for Existing Datasets
AG-News (58] has unknown license, the DB-Pedia dataset is released under Creative Commons Attribution Share Alike 3.0, the NYT dataset is distributed by LDC under the LDC's generic non-member license, the Bills dataset [X] are considered public domain works, and the Wiki dataset is licensed under BY-SA 4.0.
40
58
32
23
License for the Assets Provided by Our Paper 1.4
Our code will shared under CC BY-SA 4.0.
1.5
Broader Impacts
This paper presents work whose goal is to advance the field of Machine Learning. Our framework could potentially make machine learning systems more explainable, thus making them safer, more trustworthy and casily auditable. On the other hand, however, the learned predicates only reflect
23

---

correlation rather than causations learned from data, and hence requires careful interpretation. Given that the performance of our model-agnostec method is still far from perfect and it is unclear how human users would use them in real world applications, the algorithm presented in this paper should only be used for research and not deployed in practice.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main contributions are (a) defining family of models parameterized by predicates (justified by Section 3.2
(b) proposed an optimization algorithm based on iterative refinement and continuous relaxation, both of which improves the performance (justified by Section Takeaway 1 and 2)
(c) our method achieves similar performance as the previous method specialized for explainable clustering (justified by Section | Takeaway 3), and (d) our framework generate sophisticated predicates (justified by Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The practical limitations are hinted at the end of Section 6 and more compre- hensively discussed in Appendix G and I.]
Guidelines:
The answer NA means that paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well well-specification asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
25

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification:
We included code reproduce our experiments in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
26

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We included code to reproduce experiments in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cc.
cotpssiynnoooiyl for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips. gwtps/cccppppgguuuaaeeeeae httpc//ggnddeeeooeouuuuumbibeie ootpee///oooeeeuumiiiiiiommme assionPoicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
We included code to reproduce our experiments in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We ran statistical tests in Appendix
D
Guidelines:
The answer NA means that the paper does not include experiments.
27

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have described the cost our experiments Appendix I.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer:
[Yes]
Justification:
We have reviewed the NeurlPS Code of Ethics and observed anonymity requriements.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discussed them in Appendix L.5
28

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cited the existing datasets and mentioned their license in Appendix I.3
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
29

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We discuss the license for the new assets introduced by our paper in Appendix
1.4
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
30

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "XgGHoq6rVk", "reviewer": "Reviewer_fF31", "summary": "The paper introduces a a framework for a family of models parameterized by natural language predicates. This models allow for a language-based interpretation of text distribution. Such framework can easily instatiate models include clustering, time-series, and classification models. The authors develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradient descent and then discretizes them by prompting language models (LM). This approach is demonstrated to be effective in taxonomizing news dataset as well user chat dialogues while also characterizing their evolution over time.", "questions": "- How are you selecting language predicates lengths? Given the NLG component in the LLM call, can predicate length evolve over time? Have you done an analysis of such change in your optimization process?\n\n- When you mentioned ranking by correlation scores: the correlation is computed between $[[\\phi]]$ and the dot product between a predicate relaxation and a sample embedding. Are you correlating continuous vs discrete values? Is $[[\\phi]]$ a binary value? Could you please clarify your correlations here?\n\n- In the no-refine case, am I reading the results correctly If I understood that you select random predicates and discretize after one full round of optimization (of $w$ and $\\phi$ them once? \n\n- In the no-relax case, you are still discretizing using a language model, is that correct? Would this just mean that the optimization process would take longer given that you are skipping the correlation-based selection criterion?", "limitations": "Yes, in section 6, E.3 and G.1.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "I find the paper to have strengths along several axes:\n\n- Novelty: The general framework is, to the best of my knowledge a novel approach to describe text datasets in a way that is interpretable and scalable.\n\n- Model-Agnostic Algorithm: The algorithm developed to optimize the models proposes is versatile and can be applied to various types datasets under different approaches, namely classification, clustering and time-series analysis.\n\n- Extensive experiments and interesting ablations: The authors demonstrated the practical utility of the approach by applying it to real-world datasets (user-LLM chats), as well as news data for comparison purposes.\n\n- The results show improvements over the baseline and when this is not the case Bills and NYT dataset for clustering in table 2, the difference is either minimal or compensated with the generality of the proposed method.\n\nOverall the paper is well written and easy to follow. The results well presented and support the claim made in the paper. The limitation as well as weaknesses of the work are presented and discussed.", "weaknesses": "I did not find major weaknesses in this paper. \n\n- The main one would be in the addition computational and money costs for the proposed method compared to other classical approaches. However, in my opinion, the additional costs are counter balanced by performance and generality of the method.\n\n- Another weakness is the use of a single text embedding model. I do not think this addition experiment is necessary for accepting this paper, but I do think that testing the robustness of this method with various text embedding would make the presented claims stronger.\n\n- Additionally, given the experiments are averages across several seeds, I would report the variance of the results.\n\n----\n\nMINOR\n\n- Please adjust the citation format, in the text there's a number-based format but there are not numbers in the references\n\n- Very minor: then I tried to read table table 1 (pag 6), I was confused about what metric surface was. This was explained later in page 7. I would add a brief explanation of surface in the caption of the table, make a reference to the section when surface is explained or move the table closer the experiments section.\n\n- Typo in caption of table 2: relaxiation -> relaxation\n\n- line 277 general purpose -> general-purpose\n\n- line 648: you refer section G.1 from section G.1, did you mean section E.3?\n\n---\n\nRecomendation \n\nOverall I consider this a strong a paper and I vote in favor of its inclusion to NeurIPS", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for appreciating the strength of our paper! We will address each of your questions below.\n\n### Checking whether our conclusion generalizes to other embedding models\n\nWe report the performance of another embedding model, all-mpnet-base-v2, and report the results in Table 1 & 2 in the author response pdf. In all but 2 datasets, using this embedding model outperforms not using an embedding-based model (one-hot), indicating the robustness of our approach.\n\n### Reporting the cost of fitting our model\n\nWe report the cost of our experiments in Appendix G.2. If we use a local validator of google/flan-t5-xl (1.5B), fitting each model reported in Table 2 and 3 takes around one hour, and clustering a corpus with 1M tokens with K=8 clusters takes around $9 using the latest GPT-4o-mini model. Indeed, it is currently much more expensive than traditional methods like K-means clustering (e.g. taking around 1 minutes to cluster and find salient words). As you have mentioned, the additional costs are counterbalanced by performance and generality of the method. Finally, our method is likely to become cheaper in the future – the cost for API calls given the same capability level has decreased drastically (e.g.~10^4) since GPT-3 has initially released, and we expect the cost to continue going down.\n\n### Reporting the variance of the experiments\n\nWe will report the standard deviation of our method in our updated version.\n\n### How is the correlation between the continuous predicate and the discrete predicate calculated?\n\nThe continuous predicate is a continuous function that maps from a text sample to a real-value, and the denotation of the discrete predicate is a binary function that maps from a text sample to an integer of 0/1. For models where $w$ is only non-negative (e.g. $\\infty$), we compute the pearson-r correlation between the two functions across all text samples; for models where $w$ can be negative, we compute the absolute value of the pearson-r correlation.\n\n### Clarification in the no-refine case\n\nThe no-refine algorithm is described in Section 4.3, the initialization stage. We would: \n- Alternatively optimize two variables: 1) $w$, and 2) the continuous representation of ALL the predicates (instead of random ones)\n- Discretize each continuous representation and find one most correlated predicate\n\nThe continuous representation of ALL the predicates are randomly initialized by randomly selecting K embeddings of the text samples (K-means++initialization[1] ). \n\n### Does the predicate length evolve throughout time?\n\nAveraged across all tasks reported in Table 2-3, the average predicate length after the first discretization is 5.99, while it is 6.03 at the end. Ex-ante it is plausible that the predicates will become longer as LLM “refines” their predicates by making them more precise; however, empirically the loss usually decreases because the LLM discovers a completely new feature (e.g. cluster) that has little correlation with the feature before the update.\n\n### Clarification on the no-relax case. Do you still use the language model? Do they take longer to converge?\n\nYes, we are still using the language model; however, without any scoring on the text samples, this is similar to proposing random predicates conditioned on all the text samples, in an uninformed manner. We average across the loss curve across all tasks and plot it in Figure 1 our author response pdf. We find that the speed of convergence is significantly faster with relaxation.\n\nThanks a lot for your other feedback on writing! These are really useful feedback and we plan to incorporate them in our updated version. \n\n[1] k-means++: the advantages of careful seeding", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BAsRRLoftu", "reviewer": "Reviewer_ZMnS", "summary": "This paper proposes to use \"natural language predicates\" to explain text datasets. Authors develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradient descent and discretizes them by prompting language models. With this method, the authors can characterize the dataset with a distribution over predicates. The authors apply it to taxonomize real-world user chat dialogs and show how the text data's features can evolve across time, demonstrating the potential of the method in understanding text data.", "questions": "1. Can you provide more insights on the motivation of the paper? Why do people want to explain text datasets? What are the potential applications of the method apart from the two demonstrated in the paper?\n\n2. What is the benefit of using \"predicate\" to explain text datasets? How is the predicate better than other forms of explanations?", "limitations": "Limitations are being discussed in the paper.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper proposes a new method to explain text datasets. There are a few new concepts in the paper, such as \"Predicate-Conditioned Distribution\" and the corresponding optimization algorithm that involve both continuous and discrete optimization.\n\n2. A practical application of the method is demonstrated on real-world user chat dialogs. \n\n3. The problem is well-formulated and backed by enough mathematical and empirical evidence.", "weaknesses": "1. The motivation of the paper is somewhat unclear. It's not well explained in the Introduction section why people want to \"Explain text datasets\". Is it for controlling the dataset quality for pretraining? Or tailor the domain of dataset for finetuning LM? Although the authors showcase the method on a dataset of real-world user-ChatGPT dialogues, the results look arbitrary and not very insightful. Only \"Taxonomizing User Applications via Clustering\" and \"Characterizing Temporal Trends via Time Series Modeling\" are still not fully demonstrating the usefulness of the method.\n\n2. The method should be compared with more baseline methods. It's unknown what's the benefit of predicate-conditioned distribution over other methods. The authors should also explain why the \"predicate\" is the key to explain text datasets, instead of other factors such as just verbs, nouns, text representations, etc.\n\n3. The datasets being tested are relatively small and not showing the scalability of the method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thoughtful review. We will\n- Clarify our motivation to explain datasets\n- We compared our method to the approach you have recommended (representing clusters with verbs/nouns). **We find that natural language predicates can provide much better explanations than individual words**. \n- We ran the experiment above on a dataset with 12.5K examples, mitigating your concern that our method only works on small datasets.\n\n## The motivation for explaining datasets?\n\nThere are many existing machine learning and data (social) science applications for explaining datasets.\n\nIn the social science/data science domain, \n- [1] categorizes online discussions about political topics by generating natural language descriptions.\n- [6] clustered datasets to mine arguments in debates [2] and customer feedbacks in product reviews [9]\n\nOur framework can also be potentially applied to \n- understand what makes human thinks that a text is machine-written (with our classification model) [3]\n- analyze Google search trend similar to [4]\n\nPrior works [5] and [6] include comprehensive overviews of how clustering and classification models with natural language parameters are useful, separately. Our work contributes by connecting these separate models and creating a unified formalization and optimization framework.\n\n## The significance of taxonomizing LLM applications. \n\nThis is a growing need from the community that hosts Chatbots for users. \n- Industry: LLM companies are building internal prototype systems like this to understand how their LLM is used with real user data. \n- Academia: The ChatbotArena used naïve method to cluster user queries (Figure 3 of [7]) and prompt LLM to summarize the clusters, indicating that this is a realistic application that some machine learning researchers care about. They are currently trying to taxonomize these queries to create more fine-grained Elo ratings.\n\n## Why natural language predicates, but not other representations like verbs or adjectives?\n\nIn L266 and Appendix Table 4, we show that the word-based topic model significantly underperforms our method. \n\nPrior work [8] has also found that it is hard to interpret meaning from a list of representative words since they might not be semantically coherent, and [6] shows that natural language predicates are more explainable than lists of words for humans.\n\nWe illustrate this by including an additional experiment suggested by the reviewer krnw: clustering the MATH dataset based on their subareas. We will first present the results and then describe the experimental details. Overall, **we find that significant guesswork is needed to interpret the meaning of each word-based cluster (e.g. unclear what cluster 1 represents), while the predicates generated by our algorithm are directly explainable.**\n\nOutputs of our model:\n```\n- involves algebraic manipulation;\n- involves probability or combinatorics\n- requires geometric reasoning; \n- pertains to number theory; \n- involves calculus or limits; \n```\n\nWord-based cluster representation (presenting two based on space limit):\n```\ncluster 0:\n> 'ADJ': ['shortest', 'cartesian', 'parametric', 'polar', 'correct'],\n\n> 'ADV': ['shortest', 'numerically', 'downward', 'upward', 'meanwhile'],\n\n> 'NOUN': ['areas', 'option', 'foci', 'endpoints', 'perimeter'],\n\n> 'VERB': ['lie', 'enclosed', 'corresponds', 'vertices', 'describes']},\n\ncluster 1: \n\n> {'ADJ': ['decimal', 'base', 'sum_', 'residue', 'prod_'],\n\n> 'ADV': ['90', 'recursively', 'mentally', 'dots', 'left'],\n\n> 'NOUN': ['operation', 'sum_', 'denominator', 'dbinom', 'div'],\n\n> 'VERB': ['evaluate','simplify', 'rationalize','calculate', 'terminating']},\n```\n\n## Experimental details: \n\nWe run the experiment suggested by reviewer krnw: clustering the MATH dataset based on their subareas. We directly compared our clustering algorithm to the approach you recommended: concretely, we first cluster the problem description based on K-means; then for each cluster, we filter out the nouns/verbs/adjectives, learn a unigram regression model to predict whether a sample comes from a cluster, and present the unigrams with the highest coefficient.\n\n[1] Opportunities and Risks of LLMs for Scalable Deliberation with Polis\n\n[2] DebateSum: A large-scale argument mining and summarization\n\n[3] Human heuristics for AI-generated language are flawed\n\n[4] Google Trends\n\n[5] Goal-Driven Explainable Clustering via Language Descriptions\n\n[6] Goal Driven Discovery of Distributional Differences via Language Descriptions\n\n[7] Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference\n\n[8] Reading Tea Leaves: How Humans Interpret Topic Models\n\n[9] Huggingface dataset: ​​McAuley-Lab/Amazon-Reviews-2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wH3UDHv26h", "reviewer": "Reviewer_krnw", "summary": "The authors propose a general framework to generate explanations of different text datasets, by parameterizing the data distribution with textual predicates. The textual predicates, along with their weights, can be viewed as an explanation of the data. Three different tasks are used as examples: clustering, time-series modeling, and multi-class classification. To learn the textual predicates and their associated weights, the authors first iteratively learn the predicates in the embedding space along with the weights, and then use a pre-trained language model to discretize the predicate embeddings into textual space. Experiments are conducted on three datasets that have ground-truth predicates. A qualitative experiment with user-chatgpt dialogues is also presented.", "questions": "See weaknesses.", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed framework nicely generalizes from the classical Bayesian text analysis methods and makes them more relevant/useful in the LLM era.\n\n2. Comprehensive experiments are performed on four different datasets/tasks.\n\n3. The paper is well-written and easy to follow.", "weaknesses": "1. Missing pure LLM baselines. It is fairly common nowadays that LLMs are directly used to generate explanations given the data, which can be applied to any dataset and any task. More evidence is needed to show the advantages of the proposed method compared to direct prompting. For example, for the clustering task, what if you perform clustering first, and then prompt GPT4 to directly explain each cluster? \n\n2. The explanations generated are mostly topic descriptions, and the datasets used for qualitative results are quite simple. Can the proposed method generate more complex descriptions for more difficult datasets? For example, for a STEM problem-solving dataset (e.g. MATH), can the proposed method explain the dataset by the theorem used/subarea of the subject/similar solving strategy? \n\nIn summary, I like the way the proposed method combines the classical Bayesian method with LLM prompting, but I'm not totally convinced that the proposed method is significant as there is no direct comparison with LLM prompting, or evaluation on harder datasets. I'm willing to raise my score if my concerns are properly addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review and recommended experiments!\n\n## Summary of Our Response\n\n### Re: pure LLM prompting baseline.\n\n- This baseline is similar to our no-refine and no-relax baseline, and we showed that both refinement and relaxation were helpful (Takeaways 1 & 2).\n- We **directly ran the naïve prompting experiment you recommended** (Tables 1 & 2 in the author response pdf), and **our method is significantly better**.\n\n### Re: clustering on more complicated datasets (e.g. MATH). \n\nAs you recommended, we ran our clustering model on the MATH dataset. \n- Our clustering model **recovered all the 5 subareas predefined in the MATH dataset**\n- Our model was able to cluster solutions based on the strategies to solve the problems\n\nBelow is our more detailed response.\n\n## Re: Missing Pure LLM baseline\n\nOur paper implicitly compared our method to this baseline. Effectively, no-refinement + no-relaxation ~=pure LLM prompting. Without relaxation, the language model would propose predicates only based on the text samples, and we will directly output these predicates under no-refinement. Our paper has shown that both refinement and relaxation are helpful (Takeaways 1 and 2 in Section 5), implying that our approach will outperform the naïve prompting baseline.\n\nStill, we completely agree that including the pure LLM prompting baseline can better highlight the strength of our algorithm, and indeed we find that **our algorithm significantly outperforms this baseline**. \n\nSpecifically, we directly prompted the language model to generate a list of natural language predicates. Here is a part of our prompt for clustering: “We want to come up with some features (expressed in natural language predicate) such that they can be used for clustering”. To control for the randomness in random text samples in the language model context, we ran this baseline approach 5 times and report the average performance in Table 1 and 2 in the author-response pdf.\n\nAcross all datasets, the performance of prompting LLM lags significantly behind our approach, indicating the necessity of a sophisticated approach. Additionally, the max of 5 seeds performance (F1-score) for clustering/time-series/classification is 0.36/0.45/0.58 on average, which still significantly lags behind our approach (0.64/0.64/0.73). This implies that our method is robustly better than the naïve, pure LLM baseline.\n\n\n## Applying our method to more complicated tasks (e.g. sub-area/solution strategy on the MATH dataset)\n\n\nOur evaluation focused on topic-based datasets since it is a standard practice in the clustering literature [1,2] and these datasets have ground truth. However, indeed our method is not limited to topic clustering. **As you have recommended, we ran our clustering model on the MATH dataset**. We will first present the results, and then describe the experimental details. \n\nWhen clustering problems based on subareas, **our method recovered all of the 5 subareas in the MATH dataset as defined by the original authors**. The MATH dataset was released under five pre-defined categories (after merging different difficulty levels of algebra): \n\n```[Algebra, counting_and_probability, geometry, number_theory, precalculus]```\n\nHere is the output of our clustering model:\n```\n- involves algebraic manipulation;\n- involves probability or combinatorics\n- requires geometric reasoning; \n- pertains to number theory; \n- involves calculus or limits; \n```\nWe then clustered the geometry solutions based on their strategies. While we do not have ground truth clusters to compare against, we qualitatively examined a few examples for each cluster and most of the samples satisfy the corresponding cluster descriptions. Here is our outputs\n```\n- employs similarity or congruence in triangles;\n- uses algebraic manipulation to solve for unknowns; \n- involves area or volume calculation; \n- uses geometric diagrams or visuals;\n```\n\nTo conclude, we directly ran the experiments you recommended, showing that our method significantly outperforms the naïve prompting baseline and can be applied to more challenging tasks beyond topic clustering.\n\n**Experimental Details and Results:**\n\nWhen clustering the problem descriptions based on subareas, we prompt the LM “I want to cluster these math problems based on the type of skills required to solve them.” to discretize. When clustering the solutions based on the strategies, we used the following prompt: \"I want to cluster these math solutions based on the type of strategies used to solve them.\"\n\n[1] Goal-driven explainable clustering via language descriptions\n\n[2] TopicGPT: A Prompt-based Topic Modeling Framework", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thanks to all the reviewers for the feedback! We are glad that overall the reviewers like our formulation, method, and writing. In this response, we supplement new experiments to strengthen our paper.

- Comparison against naive prompting (reviewer krnw): while our paper has implicitly compared to this baseline, we followed the recommendation to directly run this experiment. **We find that our method is significantly better than this naïve prompting baseline**.
- Clustering on the MATH dataset (reviewer krnw, ZMns): we directly ran the experiment suggested by krnw, and **our method uncovers all the 5 ground truth subarea in MATH**. Additionally, we compared our approach to the verb/noun-based method suggested by reviewer ZMns and found that our approach is more explainable. 
- Generalization to other embedding models (reviewer fF31): we ran our experiments on another embedding model (all-mpnet-base-v2) and found that it also significantly outperforms the one-hot-embedding baseline, further validating the importance of using informative text embeddings.

We have directly addressed most of the concerns raised by the reviewers. If you have any further questions, feel free to let us know!

Meta-review: This paper proposes a general framework to generate explanations of text datasets, by parametrizing the data distribution with textual predicates. The textual predicates, along with their weights, can be viewed as an explanation of the data. To learn the textual predicates and their associated weights, they iteratively learn the predicates in the embedding space along with the weights, and then use a pre-trained language model to discretize the predicate embeddings into textual space.

Most reviewers agree that this is an interesting and novel contribution. Most of the concerns seem to have been addressed convincingly in the author rebuttal and in the additional experiments. I recommend acceptance.
Decision: Accept (poster)