Explaining Datasets in Words: Statistical Models with Natural Language Parameters

Authors: ruiqi  zhong
heng  wang
dan  klein
jacob  steinhardt
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 02:58:36

Abstract: To make sense of massive data, we often first fit simplified models and then interpret the parameters; for example, we cluster the text embeddings and then interpret the mean parameters of each cluster.
However, these parameters are often high-dimensional and hard to interpret.
To make model parameters directly interpretable, we introduce a family of statistical models---including clustering, time series, and classification models---parameterized by *natural language predicates*. 
For example, a cluster of text about COVID could be parameterized by the predicate ``*discusses COVID*''.
To learn these statistical models effectively, we develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradient descent and discretizes them by prompting language models (LMs).
Finally, we apply our framework to a wide range of problems: taxonomizing user chat dialogues, characterizing how they evolve across time, finding categories where one language model is better than the other, clustering math problems based on subareas, and explaining visual features in memorable images.
Our framework is highly versatile, applicable to both textual and visual domains, can be easily steered to focus on specific properties (e.g. subareas), and explains sophisticated concepts that classical methods (e.g. n-gram analysis) struggle to produce.

Main paper: 
 introduction: 
 To analyze massive datasets, we often fit simplified statistical models and interpret the learned parameters. For example, to categorize a set of user queries, we might cluster their embeddings, look at samples from each cluster, and hopefully each cluster corresponds to an explainable category, e.g. "asks about COVID symptoms" or "discusses the U.S. Election". Unfortunately, each cluster might contain an uninterpretable group of queries, thus failing to explain the categories. Such a failure is not an isolated incident: many models explain datasets by learning high dimensional parameters, but these parameters might require significant human effort to interpret. For example, BERTopic [18] learns uninterpretable cluster centers over high-dimensional neural embeddings. LDA [7] , Dynamic Topic Modeling [6] (time series), and Naive Bayes (classification) learn weights over a large set of words/phrases, which do not directly explain abstract concepts [9, 52, 63] . We want model parameters that are more interpretable, since explaining datasets is important in machine learning [60] , business [4] , political discussion [47] , and science [17, 34] .To make model parameters directly interpretable, we introduce a family of models where some of their parameters are represented as natural language predicates, which are inherently interpretable. Our core insight is that we can use a predicate to extract a 0/1 feature by checking whether it is true on a sample. For instance, given the predicate ϕ = "discusses the U.S. Election", its denotation ϕ is a binary function that evaluates to 1 on texts x discussing the U.S. Election and 0 otherwise: ϕ : "discusses the U.S. Election" (x : "Is Georgia a swinging state this year?") = 1. Using these 0/1 feature values, we define a wide variety of models, including clustering, classification, and time series modeling, all parameterized by natural langauage predicates (Figure 1 ). Learning these predicates ϕ requires optimizing them to maximize the log-likelihood of the data. This is challenging because ϕ are discrete and thus do not admit gradient-based optimization. We propose a general method to effectively optimize ϕ: we create a continuous relaxation φ of ϕ and optimize φ with gradient descent; then we prompt an LLM to explain the behavior of φ, thus converting it back to discrete predicates (Section 4). We repeat this process to iteratively improve performance.To evaluate our optimization algorithm, we create statistical modeling problems where the optimal predicate parameters are known, so we can use them as the ground truth. We evaluated on three different statistical models (clustering, multilabel classification, and time series modeling, as illustrated in Figure 1 ) and used five different datasets (NYT articles, AG-News, DBPedia, Bills, and Wiki [40, 58, 32, 23] ). We found that both continuous relaxation and iterative refinement improve performance; additionally, our model-agnostic algorithm matches the performance (2% increase in F1 score) of the previous algorithm specialized for explainable text clustering [53] .Finally, we show that our framework is highly versatile by applying it to a wide range of tasks: taxonomizing user chat dialogues [59] , characterizing how they evolve, finding categories where one language model is better than the other, clustering math problems [21] based on their subareas, and explaining what visual features make an image memorable [24] . Our framework applies to both text and visual domains, can be easily steered to explain specific abstract properties, and explains complicated concepts that classical methods (e.g. n-gram regression/topic model) struggle to produce. Combining LLM's ability to generate explanations along with traditional statistical models' ability to process sophisticated data patterns, our framework holds the promise to help humans better understand the complex world. 
related work: 
 Statistical Modeling in Text. Statistical models based on n-gram features or neural embeddings are broadly used to analyze text datasets. For example, logistic regression or naïve Bayes models are frequently used to explain differences between text distributions [51] ; Gaussian mixture models on pre-trained embeddings can create text clusters [2] ; topic models can mine major topics across a large collection of documents [7] and across time [6] . However, since these models usually rely on high-dimensional parameters, they are difficult to interpret: for example, human studies from [9] show that the most probable words for a topic might not form a semantically coherent category. To interpret these models, prior works proposed to explain each topic or cluster by extracting candidate phrases either from the corpus or from Wikipedia [8, 49, 57] . Our work complements these approaches to explain models with natural language predicates, which are potentially more flexible.Prompting Language Model to Explain Dataset Patterns. Our algorithm heavily relies on the ability of LLMs to explain distributional patterns in data when prompted with datasets [39, 46] . [61, 62, 15] have prompted LLMs to explain differences between two text distributions; [53, 38, 50, 27] prompted LLMs to generate topic descriptions over unstructured texts; [44, 22, 64] prompted LLMs to explain the function that maps from an input to an output; [45, 5] prompted LLMs to explain what inputs activate a direction in the neural embedding space. However, these works focused on individual applications or models in isolation; in contrast, our work creates a unifying framework to define and learn more complex models (e.g. time series) with natural language parameters.Concept Bottleneck Models (CBM). CBMs aim to achieve explainability by learning a simple model over a set of interpretable features [26] , and recent works have proposed to extract these features using natural language phrases/predicates [3, 55, 30, 12, 41] . While most of these works focus on classification tasks, our work formalizes a broad family of models-including clustering and time series -and proposes a model-agnostic algorithm to learn them. Additionally, these prior works focus on downstream task performance (e.g. classification accuracy), thus implicitly assuming that the model grounds the feature explanations in the same way as humans; in contrast, since our focus is on explanations, we focus on our algorithm's ability to recover ground truth explainable features.We discuss more related work on discrete prompt optimization, exploratory analysis, and learning with latent language in Appendix A.3 Mathematical FormulationQuestion: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: The practical limitations are hinted at the end of Section 6 and more comprehensively discussed in Appendix G and I.1.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
predicate-conditioned distribution: 
 In order to model text distributions with natural language parameters, we introduce a new family of distributions, predicate-conditioned distributions; these distributions will serve as building blocks for the models introduced later, just like normal distributions are building blocks for many classical models like Gaussian Mixture or Kalman Filter. Predicate-conditioned distributions p are supported on the set X of all the text samples we observe from the dataset, and they are parameterized by (1) a list of K predicates ⃗ ϕ ∈ Φ K , and (2) real-valued weights w ∈ R K on those predicates. Formally,EQUATIONWe now explain how to (1) extract a feature vector from x using ⃗ ϕ, (2) linearly combine ⃗ ϕ by re-weighting with w, and (3) use the reweighted values to define p(x | w, ⃗ ϕ).Natural Language Parameters ⃗ ϕ. Each predicate ϕ ∈ Φ is a natural language string and its denotation ϕ : X → {0, 1} maps samples to their value under the predicate. For example, if ϕ = "is sports-related", then ϕ ("I love soccer.")= 1. Since a model typically requires multiple features to explain the data, we consider vectors ⃗ ϕ ∈ Φ K of K predicates, where now ⃗ ϕ maps X to {0, 1} K :EQUATIONTo instantiate • computationally, we prompt a language model to check whether ϕ is true on the input x, following the practice from prior works [61, 62] . See Figure 2 (left) for the prompt we used.Reweighting with w. Consider the following example:EQUATIONThen w T ⃗ ϕ has a value of -5 • 1 + 3 • 0 = -5 for an English, non-sports related sample x. More generally, w T ⃗ ϕ (x) is larger for non-English sports-related samples.Defining p(x | ⃗ ϕ, w). According to Equation 1, p(x | ⃗ ϕ, w) is a distribution over X, all the text samples we observe, but it puts more weights on x with higher values of w T ⃗ ϕ (x). Using the example w and ⃗ ϕ above, p(x | ⃗ ϕ, w) has higher probability on non-English sports-related texts.Finally, we define U (x) as the uniform distribution over X for later use. Please suggest predicates about the text samples that are more likely to achieve higher scores. Your responses are:-"has a topic of art" -"has a topic of sports" -….Discretization: Denotation:← ϕ ← x x ∼ U(x) ↑ cos(e x , φ) ↑ [[ϕ]](x) Discretize( φ)The input variables illustrated in blue and the output of each prompt in boldFigure 2 : Left. The prompt to compute ϕ (x). Right. The prompt to Discretize φk , which generates a set of candidate predicates based on samples x from U and their scores cos(e x , φk ).We introduce three models parameterized by predicates: clustering, time series, and multi-label classification. For each model, we explain its input, the learned parameters ⃗ ϕ and w, the log-likelihood loss L, and its relation to classical models.Clustering. This model aims to help humans explore a large corpus by creating clusters, each explained by a predicate. Such a model may help humans obtain a quick overview for a large set of machine learning inputs [60] , policy discussions [47] , or business reviews [4] . Given a set of text X, our model produces a set of K clusters, each parameterized by a learned predicate ϕ k ; for example, if the predicate is "discusses the U.S. Election", then the corresponding cluster is a uniform distribution over all samples in X that discuss the U.S. Election. Similar to K-means clustering, each sample x is assigned to a unique cluster. We use a one-hot basis vector b x ∈ R K to indicate the cluster assignment of x, and set w x = τ • b x , where τ has a large value (e.g. 10). We maximize the total log-likelihood:L( ⃗ ϕ, w) = - x∈X log(p(x | ⃗ ϕ, w x )); w x = τ • b x ,where τ → ∞ and b x is a basis vector.However, some samples might not belong to any cluster and thus have 0 probability; to prevent infinite loss, we add another "background cluster" U (x) that is uniform over all samples in X; therefore, each sample x can back off to this cluster and incur a loss of at most -log U (x) = log(|X|).Time Series Modeling. This model aims to explain latent variations in texts that change across time; for example, finding that an increasing number of people "search about flu symptons" (ϕ) can help us forecast a potential outbreak [16] . Formally, the input is a sequence of T text samples X = {x t } T t=1 . Our model produces K predicates ϕ k that capture the principle axes of variation in x across time. We model w 1 . . . w T as being drawn from a Brownian motion, i.e.,EQUATIONwhere λ is a real-valued hyper-parameter that regularizes how fast w can change. The loss L is henceEQUATION)Multiclass Classification with Learned Feature Predicates. This model aims to explain the decision boundary between groups of texts, e.g. explaining what features are more correlated with the fake news class [35] compared to other news, or explaining what activates a neuron [5] . Suppose there are C classes in total; the dataset is a set of samples x i each associated with a class y i ∈ [C]. Our model is hence a linear logistic regression model on the feature vectors extracted by ⃗ ϕ, i.e.i ) = W • ⃗ ϕ (x i ); L( ⃗ ϕ, W ) = - i log( e logits(xi)y i C c=1 e logits(xi)c),where W ∈ R C×K is the weight matrix for logistic regression.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [NA]Justification:Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
method: 
 We can now learn the parameters for each model above by minimizing the loss function L. Formally,EQUATIONHowever, optimizing ⃗ ϕ is challenging, since it is discrete and therefore cannot be directly optimized by gradient-based methods. To address this challenge, we develop a general optimization method, which we describe at a high level in Section 4.1, introduce its individual components in Section 4.2, and explain our full algorithm in Section 4.3.Our framework pieces together three core functions that require minimal model-specific design:1. OptW, which optimizes w. 2. OptRelaxedPhi, which optimizes a continuous relaxation φk for each predicate ϕ k . 3. Discretize, which maps from continuous predicate φk to a list of candidate predicates.Using these three components, our overall method initializes the set of predicates by first optimizing w and φ using OptW and OptRelaxedPhi and then discretizing φ with Discretize. To further improve the loss, it then iteratively removes the least useful predicate, re-optimizes its continuous representation, and discretizes it back to a natural language predicate.To provide more intuition for these three components, we explain what they should achieve in the context of clustering. OptW should optimize the 1-hot choice vectors w x by assigning each text sample to the cluster with maximum likelihood. OptRelaxedPhi should find a continuous cluster representation φk similar to the sample embeddings assigned to this cluster, and Discretize generates candidate predicates that explain which samples' embeddings are similar to φk . Next, we introduce these three components formally for general models with predicate parameters.OptW optimizes w while fixing the values of ⃗ ϕ. Formally, OptW( ⃗ ϕ) := argmin w L( ⃗ ϕ, w).This function needs to be designed by the user for every new model, but it is generally straightforward: in the clustering model, it corresponds to finding the cluster that assigns the highest probability for each sample; in classification, it corresponds to learning a logistic regression model; in the time series model, the loss is convex with respect to w and hence can be optimized via gradient descent.For later use, we define the fitness of a list of predicates ⃗ ϕ as the negative loss after w is optimized:EQUATIONNext, we discuss OptRelaxedPhi. The parameters ⃗ ϕ are discrete strings, so the loss function is not differentiable with respect to ⃗ ϕ. To address this, we approximate ⃗ ϕ (x) with the dot product of two continuous vectors, φk • e x , where e x ∈ R d is a feature embedding of x normalized to unit length (e.g. the last-layer activations of some neural network), and φk ∈ R d is a unit-length, continuous relaxation of ϕ k . Intuitively, if the optimal ϕ = "is sports-related" and x is a sports-related sample with ϕ (x) = 1, then we hope that φ would correspond to the latent direction encoding the sports topic and it has high similarity with the embedding e x of x. Under this relaxation, L becomes differentiable with respect to φk and can be optimized with gradient descent.Formally, OptRelaxedPhi optimizes all continuous predicates φ1...K given a fixed value of w:EQUATIONWe sometimes also use it to optimize a single continuous predicate φk given a fixed w and all discrete predicate variables other than ϕ k (denoted as ϕ -k ): 1 : We compare the reference predicates and our learned predicates when clustering the DBPedia dataset. We abbreviate the predicates, e.g. "art" = "has a topic of art". For each reference, we match it with the learned predicate that achieves the highest F1-score at predicting the reference denotation. We also report the surface similarity (defined in Section 5.2) between the learned predicate and the reference. Our learning algorithm mostly recovers the underlying reference predicates, though it sometimes learns larger/correlated cluster that disagrees with the reference but is still meaningful.EQUATIONFinally, Discretize converts φk into a list of M discrete candidate predicates to update the variable ϕ k . Our goal is to find ϕ whose denotation is highly correlated with the dot product simulation φk • e x .To discretize φk , we prompt a language model to generate several candidate predicates and then re-rank them. Concretely, we draw samples x ∼ U (x) 3 and sort them based on their dot product φk • e x . We then prompt a language model with these sorted samples and ask it to generate candidate predicates that can explain what types of samples are more likely to appear later in the sorted list (Figure 2 bottom). To filter out unpromising predicates, we re-rank them based on the pearson-r correlations between ϕ and φk • e x on U if w cannot be negative (e.g. clustering), and the absolute value of pearson-r correlation otherwise. We then keep the top-M predicates.Our algorithm has two stages: we first initialize all the predicate variables and then iteratively refine each of them. During initialization, we 1. randomly initialize continuous predicates φ to be the embedding of random samples from X 2. optimize L( φ, w) by alternatively optimizing w and all the continuous predicates φ with OptW and OptRelaxedPhi, and 3. set ϕ k as the first candidate from Discretize( φk )During refinement, we repeat the following steps for S iterations:1. find the least useful predicate ϕ k ; we define the usefulness of ϕ k as how much the fitness would decrease if we zero it out, i.e. -Fitness( ⃗ ϕ -k , 0). 2. optimize φk using OptRelaxedPhi and choose the fittest predicate from Discretize( φk )We include a formal description of our algorithm in Appendix Algorithm H.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: We included code to reproduce our experiments in the supplementary material. 
experiments: 
 In this section, we benchmark our algorithm from Section 4; we later apply it to open-ended applications in Section 6. We run our algorithm on datasets where we know the ground truth predicates ⃗ ϕ and evaluate whether it can recover them. On five datasets and three statistical models, continuous relaxation and iterative refinement consistently improve performance. Our general method also matches a previous specialized method for explainable clustering [53] .We design a suite of datasets for each of the three statistical models mentioned from Section 3.2. Each dataset has a set of reference predicates, and we evaluate our algorithm's ability to recover them.Clustering. We consider five datasets, AGNews, DBPedia, NYT, Bills, and Wiki [40, 58, 32, 23] . The datasets have 4/14/9/21/15 topic classes each described in a predicate, and we sample 2048 examples from each for evaluation.Multiclass Classification. We design a classification dataset with 5,000 articles and 20 classes; its goal is to evaluate a method's ability to recover the latent interpretable features useful for classification. Therefore, we design each class to be a set of articles that satisfy three predicates about its topic, location, and language; for example, one of the classes can be described by the predicates "has a topic of sports", "is in Japan", and "is written in English". We create this dataset by adapting the New York Times Articles dataset [40] , where each article is associated with a topic and a location predicate; we then translate them into Spanish, French, and Deutsch. We consider in total 4 + 4 + 4 = 12 different predicates for each of the topic/location/language attributes and subsample 20 classes from all 4×4×4 = 64 combinations. Time Series modeling. We synthesize a time series problem by further adapting the translated NYT dataset above. We set the total time T = 2048 and sample x 1 . . . x T according to the time series model in Section 3.2 to create the benchmark. We set ⃗ ϕ to be the 12 predicates mentioned above and the weight w •,k for each predicate ϕ k to be a cosine function with a period of T to simulate how each attribute evolves throughout time. In addition, we included three simpler datasets where there is only variation on one attribute (i.e. varies only on one of topic/location/language). We name these four time series modeling all, topic, locat, and lang, respectively. See Appendix B for a more detailed explanation.To evaluate our algorithm, we match each learned predicate φk with a reference ϕ * k ′ , compute the F1-score and surface similarity for each pair, and then report the average across all pairs. To create the matching, we match φk to the ϕ * k ′ with the highest overlap (number of samples where both are true); formally, we define a bi-partite matching problem to match each predicate in φ with one in ϕ * , define the weight of matching ϕ * k ′ and ϕ * k ′ to be their overlap, and then find the maximum weight matching via the Hungarian algorithm. We now explain the F1-score and surface similarity metric.F1-score Similarity. We compute the F1-score of using φ(x) to predict ϕ * (x) on X, the set of samples we observe. This is similar to the standard protocol for evaluating cluster quality [28] .Surface Form Similiarity. We can also directly evaluate the similarity between two predicates based on their string values, e.g. "is about sports" is similar in meaning to "has a topic of sports", a metric previously used by [62] . For a pair of predicates, we ask gpt-4 to evaluate whether they are similar in meaning, related, or irrelevant, with each option associated with a surface-similarity score of 1/0.5/0. We display the prompt in Figure 5 and example ratings in Table 1 .We now use these metrics and datasets to evaluate the optimization algorithm proposed in Section 4 and run ablations to investigate whether continuous relaxation and iterative refinement are effective. We will first introduce the overall experimental setup, and then discuss individual takeaways supported by experimental results in each paragraph.Experimental Setup. When running the algorithm, we generate candidate predicates in Discretize with gpt-3.5-turbo [37]; to perform the denotation operation ϕ (x), we use flan-t5-xl [13] ; we create the embedding for each sample x with the Instructor-xl model [48] and then normalize it with ℓ 2 norm. We set the number of candidates M returned by Discretize to be 5 and the number of optimization iteration S to be 10. To reduce noises due to randomness, we average the performance of five random seeds for each experiment. Takeaway 0: Is our method better than naïvely prompting language model to generate predicates? How does our approach compare to a naïve baseline approach, which directly prompts the language model to generate predicates based on dataset samples? For this baseline, we repeatedly prompt a language model to generate more predicates until we obtain K predicates, compute their denotation, evaluate them using the metrics in Section 5.2, and report the performance in Table 2 and 5, the Prompting row. Across all entries, our approach significantly outperforms this baseline.Takeaway 1: Relax + discretize is better than exploring randomly generated predicates. Our optimization algorithm explores the top-5 LLM-generated predicates that have the highest correlations with φk •e x . Would choosing a random predicate be equally effective? To investigate this question, we experimented with a variant of our algorithm that randomly chooses five predicates without utilizing the continuous representation φk (No-Relax). In Table 2 and 3 , No-Relax underperforms our full algorithm (Ours) in all cases. In Appendix Figure 6 , we plot the loss after each iteration averaged across all tasks, and we find that Ours converges much faster than No-Relax.Takeaway 2: Iterative refinement improves the performance. We considered a variant of our algorithm that only discretizes the initial continuous representations and does not iteratively refine the predicates (No-Refine). In Table 2 and 3 , No-Refine underperforms the full algorithm in all cases.Takeaway 3: Our model-agnostic method is competitive with previous methods specialized for explainable clustering. We compare our method to GoalEx from [53] , which designs a specialized method for explainable clustering based on integer linear programming. Even though our method is model-agnostic, it matches or outperforms GoalEx on four out of five datasets and improves F1 by 0.02 on average.Takeaway 4: Our method accounts for information beyond the set of text samples (e.g. temporal correlations in the time series). We investigate this claim using the time series datasets, where we shuffle the text order and hence destroy the time-dependent information a model could use to extract informative predicates (Shuffled). Table 3 finds that Ours is better than Shuffled in all cases, indicating that our method does make use of temporal correlations.Appendix D includes additional results: 1) compared to topic modeling and K-means, our method achieves similar or better performance while being explainable; 2) we ran ablations on the effect of neural embeddings and show that informative embeddings are crucial to good performance; 3) Takeaways 1, 2, and 4, are significant with p < 1% under paired t-tests. Although some topics are plausibly related to certain applications, they are still ambiguous. 
open-ended applications: 
 We apply our framework to a broad range of applications to show that it is highly versatile. Our framework can monitor data streams (Section 6.1), apply to the visual domain (Section F.1), and be easily steered to explain specific abstract properties (Section F.2). Across all applications, our framework is able to explain complex concepts that classical methods struggle to produce.We apply our models from Section 3.2 to monitor complex data streams of LLM usages. In particular, we recursively apply our clustering model to taxonomize user queries into application categories, apply our time series model to characterize trends in use cases across time, and apply our classification model to find categories where one LLM is better than the other. Due to space constraints, we present the key results in the main paper and the full results in Appendix G.Taxonomizing User Applications via Clustering. LLMs are general-purpose systems, and users might applyLLMs in ways unanticipated by the developers. If the developers can better understand how theLLMs are used, they could collect training data correspondingly, ban unforeseen harmful applications, or develop application-specific methods. However, the amount of user queries is too large for individual developers to process, so an automatically constructed taxonomy could be helpful.We recursively apply our clustering model to user queries to the ChatGPT language model. We obtain the queries by extracting the first turns from the dialogues in WildChat [59] , a corpus of 1M real-world user-ChatGPT dialogues. We use gpt-4o [36] to discretize and claude-3.5-sonnet [1] to compute denotations. We first generate K = 6 clusters on a subset of 2048 queries; then we generate K = 4 subclusters for each cluster with > 32 samples.We present part of the taxonomy in Figure 3 (left) and contrast it with the taxonomy constructed by directly applying LDA recursively (right). Although some LDA topics are plausibly related to certain applications, they are still ambiguous; for example, it is unclear what topic 1 "ar prompt description detailed" means. After manually inspecting the samples associated with this topic, we found that they were related to the application of writing prompts for an image-generation model. In contrast, our framework can explain complicated concepts that are difficult to infer from individual words; for example, it generates "requesting graphic design prompts" for the above application, which is much clearer in its meaning when explained in natural language.Characterizing Temporal Trends via Time Series Modeling. Understanding temporal trends in user queries can help forecast flu outbreaks [16] , prevent self-reinforcing trends [19] , or identify new application opportunities. We run our time series model on 1000 queries from WildChat with K = 4 to identify temporal trends in user applications, and report part of the results in Figure 4 . Based on the blue curve, we find that an increasing number of users "requests writing or content creation .... creating stories based on given prompts.'. This helps motivate systems like Coauthor [29] to assist with this use case.Finding Categories where One Language Model is Better than the Other. One popular method to evaluateLLMs is crowd-sourcing: an evaluation platform (e.g. ChatBotArena [11] ) / or a company (e.g. OpenAI) accepts prompts from users, shows users responses from two different LLM systems, Trends on WildChat φ0: request information or translation in a different language; i.e. the query involves requesting content in another language or translating text φ1: engage in academic or scholarly discussion; i.e. the query involves seeking information or topics related to academic subjects or discussions φ2: request writing or content creation in a specific style or genre; i.e. the query involves asking for creating stories or other written content based on given prompts φ3: request general information or answers to factual questions; i.e. the query involves seeking answers to broad or factual questions Figure 4 : We analyze WildChat queries with our time series model. For each learned predicate, we plot how its frequency evolves and the 99% confidence interval of the average frequency (shaded). and the users indicate which one they like better. The ranking among the LLM systems is then determined by Elo-rating, i.e. how often they win against each other. However, aggregate Elo-rating omits subtle differences between LLM systems. For example, LLama-3-70B achieved a similar rating as Claude-3-Opus, and the LLM community was excited that open-weight models were catching up. However, is LLama-3-70B similarly capable across all categories, or is it significantly better/worse under some categories? Such information is important for downstream developers, since some capabilities are more commercially valuable than others: e.g. a programmer usually does not care about LLM's capability to write jokes. We need a more fine-grained comparison.We directly apply the classification model from our framework to solve this task. To understand the categories where LLama-3-70B is better/worse than Claude-3-Opus, we gather user queries x from the ChatBotArena maintainers (personal communication), set y = 1 if the LLama-3-70B's response to x is preferred and y = 0 otherwise. We set K = 3.Our model finds that LLama-3-70B is better when the query "asks an open-ended or thoughtprovoking question" but worse when it "presents a technical question" or "contains code snippets". These findings are corroborated by manual analysis by the ChatBotArena maintainers, who also found that Llama-3 is better at open-ended and creative tasks while worse at technical problems 4 . We hope that our model can automatically generate similar analysis in the future when a new LLM is released, thus saving researchers' efforts.To summarize, our framework 1) enables us to define a time series model to explain temporal trends in natural language, and 2) outputs sophisticated explanations that LDA fails to generate. However, it is far from perfect: it is slow to compute denotations for all pairs of x and candidates ϕ since it involves many LLM API calls, and the predicates themselves are sometimes redundant. We describe these limitations and potential ways to improve them in Appendix G.Due to space constraints, we present applications in explaining visual features to make images memorable to humans and clustering math problems based on subareas in Appendix F.1 and F.2. 
conclusion: 
 In this work, we formalize a broad family of models parameterized by natural language predicates. We design a learning algorithm based on continuous relaxation and iterative refinement, both of them effective based on our ablation studies. Finally, we apply our framework to a wide range of applications, showing that it is highly versatile, practically useful, applicable to both text and vision domains, and explains sophisticated concepts that classical methods struggle to produce. We hope future works can make our method more computationally efficient and apply it to more realistic applications, thus assisting humans to discover and understand complex patterns in the world. 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: We discussed them in Appendix I.5.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification:Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cited the existing datasets and mentioned their license in Appendix I.3.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [Yes] Justification: We discuss the license for the new assets introduced by our paper in Appendix I.4.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification:Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification:Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: LLM for Exploratory Analysis. Due to its code generation capability [10] , large language models have been used to automatically generate programs to analyze a dataset and generate reports from them [31, 20] . In comparison, our work focuses on generating natural language parameters to extract real-valued features from structured data. Discrete Prompt Optimization. Many prior works optimized discrete prompts to improve the predictive performance [43, 14] , and some recent works demonstrated that LLMs can optimize prompts to reach state-of-the-art accuracy [64, 56] . In comparison, we focus on optimizing discrete predicates to explain patterns rather than improve task performance.Learning with Latent Language. [3] first proposed to learn in a hypothesis space of natural language strings to improve generalization, and later works in this area have focused on using natural language to guide the learning process to improve downstream task performance [33, 25, 42, 54] . In contrast, our work focuses on explaining datasets with natural language, rather than improving downstream task performance.To sample texts from the All time series problem, we sample from the time series model described in Section 3.2; we set ⃗ ϕ to be all the 12 predicates, sort them first by attributes (e.g. topic/location/language) then alphabets, and we set the weight for the k th predicate to be a sin function with period T and evenly spaced phases, i.e.EQUATIONAs a result, the weight for each predicate has evenly spaced phases and would peak at different time period.We include our prompt used to evaluate the surface form similarity between the predicted predicate φk and the reference predicate ϕ * k in Figure 5 .Is text_a and text_b similar in meaning? respond with yes, related, or no. Target: text_a: has a topic of sports text_b: is about sports output: yes ϕ * φ {Our method is similar or better than classical methods such as topic modeling or K-means. We report the performance of K-means clustering and topic modeling under the clustering benchmark 4 : We compare our method to classical clustering approaches that do not generate natural language explanations (K-means and TopicModel), where "--" means that the surface form metric is undefined since these methods do not output natural language explanations. We find that on average, our method is close to K-means and significantly outperforms TopicModel under the F1 similarity metric, while generating natural language explanations for each cluster. We also compare our method to using one-hot text embedding, and find that our method is significantly better; this indicates that the use of informative text embedding is crucial to performance. 5 : Our method consistently outperforms a variant that uses one-hot text encoding as e x rather than neural embeddings. This indicates that using informative text embedding is crucial to performance.in Table 4 . on average, our method is close to K-means and significantly outperforms TopicModel under the F1 similarity metric, while generating natural language explanations for each cluster.Takeaway 5: Using informative text embedding is crucial to performance. We used neural embeddings when optimizing the continuous representation of the predicates. Does our algorithm actually make use of the information in the feature embeddings? To investigate this question, we ran an ablation of using one-hot text embeddings rather than neural embeddings (OneHot), which do not encode any information about the similarity between text samples. We report the performance in Table 4 and 5 ; across all settings, using neural embeddings consistently outperforms OneHot.To make sure that this takeaway is general and not specific to one embedding model, we run our method with another text embedding model, all-mpnet-base-v25 and report the performance as Figure 6 : We plot how the loss decreases across different iterations with and without relaxation (that explores using random predicates). We find that using relaxation significantly speeds up optimization.the OtherEmb row. We find that using this neural embedding also outperforms OneHot in most cases, indicating that our conclusion is robust.Takeaway 1,2,and 4 are statistically significant. To compare the performance between our method and each variant, we conduct a one-sided paired t-test on their performance (F1-similarity) on each dataset, where the performance on each dataset is the averaged performance across five runs. Takeaway 1, 2, 4 has a p-value of 5 × 10 -4 , 2 × 10 -4 , and 6 × 10 -3 , respectively.Machine learning systems sometimes have unintended side effects and reinforcement themselves.[19] illustrated an example failure mode, where a group of users is discriminated against and thus leave a platform, causing a ML system to discriminate them further and hence drive them away.As a concrete illustration, let us imagine a social platform Y where users post tweets and the platform will display the most engaging ones; suppose there are two groups of users, one conservative and one liberal, where both groups prefer more engaging tweets but also tweets that agree with their political stances. Y implements a recommender system, which trains a classifier to predict whether a tweet is likely to be preferred by a random user, and then the platform Y will promote these tweets. If the two groups of users are balanced, the optimal classifier will make Y promote tweets that are engaging and place little weights on the political slant.However, if there are fewer liberal users, the classifier will be biased and Y will promote conservative tweets more and focus less on whether the tweet is engaging or not. The liberal users will find the promoted tweets less attractive, thus leaving the platform Y. As a result, fewer liberal users will stick to Y, thus making the classifier more biased.Now we provide a proof-of-concept experiment to illustrate how our time series model can be applied to detect such a reinforcing trend. We first simulate the setup above and obtain the tweets promoted by platform Y across time, and then apply our time series model to extract temporal trends from these tweets. Suppose there are two groups of users, liberal and conservative. At t = 0, the fraction of liberal users is λ 0 = 0.5 and is the same as that of conservative users. To simulate the setup above and obtain the tweets promoted by platform Y across time, we assume that at each time step t, we will sample 2,000 tweets, where each tweet is a 2D datapoint with the x-value a random integer from [-1, 1] indicating whether it is liberal, non-political, or conservative, and y-value a random integer from [-2, 2] indicating how engaging the tweet is. For each tweet, we obtain a label of y = 1/0 if the user likes a tweet, and the user's probability for liking a tweet is defined by σ(ux + 0.5y), where u = 1 if the user is conservative and 0 otherwise. We then train a logistic regression classifier to predict whether a random user will like a tweet and the platform Y will promote the tweets with the top 20% score. Let the fraction of tweets non-liberal tweets be a t and non-conservative tweets be b t , then the fraction of liberal users for the next round will be determined by:EQUATION)which models how the group size will increase/decrease depending on whether the platform promotes tweets that agree with their views. We run this process for T = 10 and gather all the 2D datapoints promoted by platform Y.We then turn these two-dimensional datapoints into text samples x. We ask the gpt-4o to write a liberal, non-political, or conservative tweet based on the x-value; then we ask gpt-4o to make it more/less engaging based on the y-value. For example, for a 2D value of (-1, 2), we ask gpt-4o to write a liberal tweet and ask it to make it more engaging two times; if the value is (1, -2), we ask gpt-4o to write a conservative tweet and then ask it to make it less engaging two times.We now have a list of tweets across time, and we directly apply our time series model with K = 3 to extract trends from them. Our time series model find that there is an increasing amount of tweets that "expresses patriotic sentiments" and "champions specific policies", but a decreasing amount "poses a question to engage the audience". These predicates exactly recover all the underlying trends, that the self-reinforcing effect make the tweets more conservative, less non-political, and less engaging. We apply our classification model from Section 3.2 to explain what visual features make images more memorable [24] . Consistent with previous findings, we find that tranquil scenes make an image less memorable, while emotions and expressions are more memorable. [21] and compare our method (left) to a classical method (right), which first clusters via K-means and then explains each cluster via unigram analysis. Our method directly explains complex concepts, while the classical method delivers vague explanations.Our framework is applicable to the vision domain since a natural language predicate ϕ can extract binary values from an image x. For example, for the rightmost image x in Figure 7 right, the predicate "portrays a person" evaluates to 1, i.e. ϕ (x) = 1, while "contains texts" evaluates to 0.We present an application of our classification model from Section 3.2 to images, which learns linear weights over a set of visual features described by natural language predicates. This model has also appeared in prior works: our model is equivalent to the language-based concept bottleneck model proposed by [55, 41] ; additionally, when K = 1 and C = 2, our model is equivalent to the VisDiff framework [15] , which finds a single predicate to discriminate samples from two classes of images.We apply our classification model to the LaMem dataset [24] to understand what visual features make an image more memorable, an interesting cognitive science question. We now define the samples x i and their class labels y i to run our classification model. In LaMem, each image is associated with a score of how memorable it is as measured by whether humans can remember seeing it in the past; to make implementation easier, we set x i to be the caption of the image and y i = 1 if x i has an above median score and y i = 0 otherwise. To fit our classification model, we set K = 6, use gpt-4o as the discretizer, and use gpt-4o-mini to compute denotation.We present three learned predicates in Figure 7 . We find that an image is less memorable if it "portrays a sense of tranquility; e.g. the image captures a serene sunset over a calm lake, with soft orange and pink hues in the sky ...", and more likely to be memorable if it " highlights specific emotions or expressions; for example, the child has a curious expression ...". These results are consistent with the previous manual analysis from [24] , suggesting the validity of our results.Can our framework explain more abstract aspects of a sample x: e.g. subarea, the type of knowledge required to solve a math problem x? We show this is feasible by applying our model from Section 3.2 to cluster math problems and steering it to focus on explaining subareas. Meanwhile, classical methods struggle to explain abstract aspects.We apply our clustering model from Section 3.2 to cluster the MATH dataset [21] based on subareas.The MATH dataset contains five labeled subareas 6 and we hope our model can recover all of them: Algebra, counting_and_probability, geometry, number_theory, and precalculus.To steer our clustering model to explain subareas, we simply prompt the discretizer LLM "I want to cluster these math problems based on the type of skills required to solve them." We set K = 5, using gpt-4o to discretize and gpt-4o-mini to compute denotation.We present the outputs of our model on the left of Figure 8 . With simple prompting, our model is successfully steered to cluster based on subareas and recovers all five labeled subareas from the MATH dataset. Note that our explanations can explain abstract properties that have no word overlap with the samples that match them: for example, the math problems that "requires geometric reasoning" (Figure 6 left 3) usually contain neither of the word "geometric" or "reasoning".We compare our method to a classical baseline that first clusters the samples and then explains each cluster with representative words. In this baseline, we first perform K-means clustering on the neural embeddings of x and assign each sample to a cluster; we then extract representative words by first running a unigram regression to predict whether a sample belongs to the cluster and then selecting words with the most positive weights. We present the word-based explanations on the right in Figure 8 . Overall, significant guesswork is needed to interpret the meaning of each word-based cluster (e.g., it is unclear what cluster 1 represents in Figure 8 right), while the predicates generated by our algorithm are directly explainable. Our framework can be steered to explain more abstract aspects of a sample x and significantly improve over classical methods.To obtain the best outputs, we discretize with gpt-4o and compute denotations with claude-3.5-sonnet. Since we aim to analyze user queries, we explicitly prompt gpt-4o to generate detailed predicates about use cases when discretizing continuous predicates. See 9 for the full prompt.G.1 Taxonomizing User Applications.Implementation Details We cluster 1024 dialogue with K = 6 and S = 5 We only cluster on a small set of dialogue turns because it is slow to compute denotations: 1) we in total explored around 100 predicates and this amounts to ∼ 100 × 1024 = 100K LLM API calls, and 2) we used language model API (claude-3.5-sonnet), rather than a local small language model (google/flan-t5-xl), to compute denotations, since this is the cheapest model that we feel confident that it can handle more sophisticated predicates.Full Results. We present the full results in Figure 10 Overall, we find that our framework can generate sophisticated explanations that classical methods cannot generate. However, some cluster descriptions are significantly overlapping (e.g., category 0, 1, and 0.D); additionally, some sub-clusters are not indeed subsets of their parent categories (e.g., subcategory 2.D does not belong to category 2). Future work can improve the taxonomy by 1) deduplicating semantically similar descriptions or more heavily penalizing cluster overlaps, and 2) steer the predicate generation process so that the descriptions for the subclusters are indeed subsets of their parent descriptions.Implementation Detail. We run our time series model on 1K dialogue turns with K = 4 and the number of iterations S to be 10 to identify temporal trends in user applications. We obtain the smoothed frequency curve by updating with the follow equation:EQUATIONWe obtain the shaded area in Figure 4 by shuffling x t and find the highest and lowest f values across 100 random runs.Figure 9 : A discretizer prompt that explicitly asks LLM to explain user applications. E.g., at the end of the prompt, we explicitly requested the predicates to start with "the user wants to...".Overall, we find that our framework allows us to define sophisticated models (e.g. time series) and can output highly sophisticated predicates, which can include detailed explanations and examples. Therefore, when implemented perfectly, its utility has a much higher upperbound than classical methods such as n-gram Bayes/regression or topic models.However, the comparison between our method and classical methods is only qualitative: we only eyeballed the outputs from our method and the classical methods in Section 6 and did not quantitatively measure how useful they are in practice. Therefore, even if our method does outperform classical methods such as topic model on our benchmark (Table 4 ), it might not directly translate to how useful it is in real-world applications. Additionally, we did not compare to modern taxonomy construction method such as [57] , which involves a lot of task-specific engineering; our method is model-agnostic and was applied out-of-the-box to construct the taxonomy. Section 6 only shows that our method can generate more sophisticated natural language explanations, which presents a higher upperbound of what our method could potentially achieve.In terms of the weakness of our method, our method is currently slow, as its performance highly depends on LLMs to compute denotations correctly, it outputs semantically similar predicates that add little information, and it is hard to control the predicates to satisfy certain properties (e.g. being a subset of a parent category). We look forward to future works that can address these problems and realize the full potential of this framework. For example, to remove similar predicates, one could prompt a language model to check the pairwise surface similarity between two predicates; to speed up inference, one can distill a smaller but much more efficient model specialized for computing denotations.Figure 10 : The full taxonomy that our algorithm generates to categorize user applications from a corpus of user chatbot queries.A formal description of our algorithm can be seen in Algorithm H.As mentioned in Appendix G.3, our current system is slow, as its performance highly depends on the LLMs to compute denotations correctly, it outputs semantically similar predicates that add little information, and it is hard to control the predicates to satisfy certain properties (e.g. being a subset of a parent category). Our experiments are limited since it assumes that the datasets and statistical models we used are reflective of real world application. We made our best effort to gather text clustering datasets that are commonly used in the literature (e.g. from [52, 38] ) and defined models that are plausibly useful for practitioners. Additionally, note that our evaluation on topic clustering is more comprehensive than the prior work [52] by including two new datasets (Bills and Wiki);Algorithm 1 A formal description of our algorithm. Argument: S is the number of steps we use to run our algorithm. Output: var_ϕ 1...K is the list of K predicates that we maintain, optimize, and return at the end of the algorithm. ŵ are other parameters. We first optimize the relaxed continuous predicates and discretize them (Line 3-10), and then iteratively refines the predicates (Line 11 -21) . During iterative refinement, we first find the least useful predicate k (Line 12), then we only optimize the continuous representation of the least useful predicate while fixing other discrete predicates (Line 14 -17); finally we discretize the k th predicate (Line 18, 19 k ← argmax k ′ Fitness(var_ϕ -k ′ , ϕ k ′ = 0) 13:φk ← randomly sample an embeddinge x 14:for t = 1 to 10 do C k ← Discretize( φk ) 19: var_ϕ k ← argmax ϕ ′ ∈C k ∪{var_ϕ k } Fitness(var_ϕ -k , ϕ k = ϕ ′ ) 20:ŵ ← OptW(var_ϕ) 21: end for 22: return var_ϕ, ŵ ) additionally, we used the exact same hyper-parameter across all clustering tasks, while [52] changed the hyper-parameters for different datasets.All of the experiments ran in Section 5 are estimed to cost at most 200 GPU hours on an A100 GPU with 40GB memory, and cost less than $20 of API credit for gpt-3.5-turbo. The experiments in Section 6 costs at most $50 of API inference credit, but we are constrained by rate limit. [40, 58, 32, 23] ) AG-News [58] has unknown license, the DB-Pedia dataset is released under Creative Commons Attribution Share Alike 3.0, the NYT dataset is distributed by LDC under the LDC's generic non-member license, the Bills dataset [23] are considered public domain works, and the Wiki dataset is licensed under CC BY-SA 4.0.Our code will be shared under CC BY-SA 4.0.This paper presents work whose goal is to advance the field of Machine Learning. Our framework could potentially make machine learning systems more explainable, thus making them safer, more trustworthy and easily auditable. On the other hand, however, the learned predicates only reflect correlation rather than causations learned from data, and hence requires careful interpretation. Given that the performance of our model-agnostic method is still far from perfect and it is unclear how human users would use them in real world applications, the algorithm presented in this paper should only be used for research and not deployed in practice. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).Our code and dataset are at https://github.com/ruiqi-zhong/nlparami.e. uniformly draw samples x from all samples we observe from the datasethttps://lmsys.org/blog/2024-05-08-llama3/https://huggingface.co/blog/1b-sentence-embeddingsafter merging similar categories that differ in levels of difficulty 


Decision: This paper proposes a general framework to generate explanations of text datasets, by parametrizing the data distribution with textual predicates. The textual predicates, along with their weights, can be viewed as an explanation of the data. To learn the textual predicates and their associated weights, they iteratively learn the predicates in the embedding space along with the weights, and then use a pre-trained language model to discretize the predicate embeddings into textual space.

Most reviewers agree that this is an interesting and novel contribution. Most of the concerns seem to have been addressed convincingly in the author rebuttal and in the additional experiments. I recommend acceptance.

Reviews: review: paper_summary: paper_summary: The paper introduces a a framework for a family of models parameterized by natural language predicates. This models allow for a language-based interpretation of text distribution. Such framework can easily instatiate models include clustering, time-series, and classification models. The authors develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradient descent and then discretizes them by prompting language models (LM). This approach is demonstrated to be effective in taxonomizing news dataset as well user chat dialogues while also characterizing their evolution over time.
strength_weakness: strength_weakness: I did not find major weaknesses in this paper. 

- The main one would be in the addition computational and money costs for the proposed method compared to other classical approaches. However, in my opinion, the additional costs are counter balanced by performance and generality of the method.

- Another weakness is the use of a single text embedding model. I do not think this addition experiment is necessary for accepting this paper, but I do think that testing the robustness of this method with various text embedding would make the presented claims stronger.

- Additionally, given the experiments are averages across several seeds, I would report the variance of the results.

----

MINOR

- Please adjust the citation format, in the text there's a number-based format but there are not numbers in the references

- Very minor: then I tried to read table table 1 (pag 6), I was confused about what metric surface was. This was explained later in page 7. I would add a brief explanation of surface in the caption of the table, make a reference to the section when surface is explained or move the table closer the experiments section.

- Typo in caption of table 2: relaxiation -> relaxation

- line 277 general purpose -> general-purpose

- line 648: you refer section G.1 from section G.1, did you mean section E.3?

---

Recomendation 

Overall I consider this a strong a paper and I vote in favor of its inclusion to NeurIPS
questions: questions: - How are you selecting language predicates lengths? Given the NLG component in the LLM call, can predicate length evolve over time? Have you done an analysis of such change in your optimization process?

- When you mentioned ranking by correlation scores: the correlation is computed between $[[\phi]]$ and the dot product between a predicate relaxation and a sample embedding. Are you correlating continuous vs discrete values? Is $[[\phi]]$ a binary value? Could you please clarify your correlations here?

- In the no-refine case, am I reading the results correctly If I understood that you select random predicates and discretize after one full round of optimization (of $w$ and $\phi$ them once? 

- In the no-relax case, you are still discretizing using a language model, is that correct? Would this just mean that the optimization process would take longer given that you are skipping the correlation-based selection criterion?
limitations: limitations: Yes, in section 6, E.3 and G.1.
paper_summary: paper_summary: This paper proposes to use "natural language predicates" to explain text datasets. Authors develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradient descent and discretizes them by prompting language models. With this method, the authors can characterize the dataset with a distribution over predicates. The authors apply it to taxonomize real-world user chat dialogs and show how the text data's features can evolve across time, demonstrating the potential of the method in understanding text data.
strength_weakness: strength_weakness: 1. The motivation of the paper is somewhat unclear. It's not well explained in the Introduction section why people want to "Explain text datasets". Is it for controlling the dataset quality for pretraining? Or tailor the domain of dataset for finetuning LM? Although the authors showcase the method on a dataset of real-world user-ChatGPT dialogues, the results look arbitrary and not very insightful. Only "Taxonomizing User Applications via Clustering" and "Characterizing Temporal Trends via Time Series Modeling" are still not fully demonstrating the usefulness of the method.

2. The method should be compared with more baseline methods. It's unknown what's the benefit of predicate-conditioned distribution over other methods. The authors should also explain why the "predicate" is the key to explain text datasets, instead of other factors such as just verbs, nouns, text representations, etc.

3. The datasets being tested are relatively small and not showing the scalability of the method.
questions: questions: 1. Can you provide more insights on the motivation of the paper? Why do people want to explain text datasets? What are the potential applications of the method apart from the two demonstrated in the paper?

2. What is the benefit of using "predicate" to explain text datasets? How is the predicate better than other forms of explanations?
limitations: limitations: Limitations are being discussed in the paper.
paper_summary: paper_summary: The authors propose a general framework to generate explanations of different text datasets, by parameterizing the data distribution with textual predicates. The textual predicates, along with their weights, can be viewed as an explanation of the data. Three different tasks are used as examples: clustering, time-series modeling, and multi-class classification. To learn the textual predicates and their associated weights, the authors first iteratively learn the predicates in the embedding space along with the weights, and then use a pre-trained language model to discretize the predicate embeddings into textual space. Experiments are conducted on three datasets that have ground-truth predicates. A qualitative experiment with user-chatgpt dialogues is also presented.
strength_weakness: strength_weakness: 1. Missing pure LLM baselines. It is fairly common nowadays that LLMs are directly used to generate explanations given the data, which can be applied to any dataset and any task. More evidence is needed to show the advantages of the proposed method compared to direct prompting. For example, for the clustering task, what if you perform clustering first, and then prompt GPT4 to directly explain each cluster? 

2. The explanations generated are mostly topic descriptions, and the datasets used for qualitative results are quite simple. Can the proposed method generate more complex descriptions for more difficult datasets? For example, for a STEM problem-solving dataset (e.g. MATH), can the proposed method explain the dataset by the theorem used/subarea of the subject/similar solving strategy? 

In summary, I like the way the proposed method combines the classical Bayesian method with LLM prompting, but I'm not totally convinced that the proposed method is significant as there is no direct comparison with LLM prompting, or evaluation on harder datasets. I'm willing to raise my score if my concerns are properly addressed.
questions: questions: See weaknesses.
limitations: limitations: Yes.

Comments: title: Thanks for your support
Looking forward to your comment
Looking forward to your reply
Thanks for supporting our work
Response to rebuttal
comment: Thanks for reading our response and supporting our work!
We have run the experiments to address your reviews. Please feel free to let us know if you have any questions. Thanks a lot!!
Thank you for the rebuttal and the new experiments. I'll increase my score in light of adding these new results to the paper.
We have directly run the experiments mentioned in your review. Please feel free to let us know if you have any questions. Thanks a lot!!
Thanks for reading our response and supporting our work!
Thanks a lot for your work on the rebuttal.

You response confirms my initial idea about the high-quality and solid contribution of your work.

I confirm my score and vote for the inclusion of the paper at the conference